Integrating Intelligent Assistants into Human Teams Katia Sycara Michael Lewis The Robotics Institute School of Information Sciences Carnegie Mellon University University of Pittsburgh Pittsburgh, PA 15213 Pittsburgh, PA 15260 (412) 268-8825 (412) 624-9426 katia@cmu.edu ml@sis.pitt.edu www.cs.cmu.edu/˜softagents www.pitt.edu/˜cmlewis
Team Members CMU Prasad Chalasani Liren Chen Keith Decker Kostya Domashnev Somesh Jha Anadeep Pannu Onn Shehory Rande Shern Vandana Verma Dajun Zeng
Team Members U. of Pittsburgh Michael Lewis (PI) Terry Lenox Emily Roth
Talk Outline � Goals � Potential Impact for the Navy � Approach � Research Issues � Progress � Plan for Next Year
Overall Research Goal Increase the effectiveness of joint Command and Control Teams through the incorporation of Agent Technology in environments that are: � distributed � time stressed � uncertain � open (information sources, communication links and agents dynamically appear and disappear) Team members are distributed in terms of: � time and space � expertise
Impacts for Navy � Reduce time for a C2 team to arrive at a decision � Allow C2 teams to consider a broader range of alternatives � Enable C2 teams to flexibly manage contingencies (replan, repair) � Reduce time for a C2 team to form a shared model of the situation � Reduce individual and team errors � Support team cohesion and team work skills � Increase overall team performance
Transition Opportunities � Maritime Crisis planning � Target identification training � Air campaign planning � Strike planning � Aircraft maintenance
Overall Approach � develop an adaptive, self-organizing collection of Intelligent Agents (the RETSINA infrastructure) that interact with the humans and each other. – integrate multimedia information management and decision support – anticipate and satisfy human information processing and problem solving needs – perform real-time synchronization of human actions – notify about significant changes in the environment – adapt to user, task and situation � develop model libraries of individual and team tasks � develop verifiable useful human-agent interaction techniques
Overall Research Issues � Agents and Agent Interactions � Human Agent Interaction � Information Filtering and Integration
Overall Research Issues: Agents and Agent Interactions � interleaving planning, replanning, execution monitoring and information gathering in a multiagent setting � single agent architecture and self-awareness � agent coordination scheme � finding appropriate agents � agent interoperability � agent-to-agent task delegation protocols � learning through agent interactions
Overall Research Issues: Human Agent Interaction � agent-based team aiding � functional allocation between humans and agents (insert agents into military simulations and perform controlled experiments with human subjects to assess utility) � human-agent trust � development of task models (graphical task editor) � user-guided instantiation of agents (agent editor)
Insert TeamAiding.ppt
Overall Research Issues: Information Filtering and Integration � learning and tracking multiple interests of users � increase relevance of retrieved information (refinement key words, relevance feedback, summary of most important information in documents) � detecting “interesting” patterns from multiple data sources � information integration and conflict resolution
Retsina Functional Organization USER 1 USER 2 USER h Goals and Task Results Specifications Interface Agent 1 Interface Agent 2 Interface Agent k Task Task Proposed Solution TaskAgent 1 TaskAgent 2 TaskAgent j Conflict Resolution Information Information Integration Request Service Request Reply Advertisement InfoAgent 1 MiddleAgent 1 InfoAgent n query answer Info Source 1 Info Source 2 Info Source k
Characteristics of RETSINA Agents � Agents act autonomously to accomplish objectives – Goal-directed – Taskable – Running unassisted for long periods – Proactive & Reactive
Characteristics of RETSINA Agents (Contd.) � Agents engage in peer-to-peer interactions – Agents are taskable, i.e. users or other agents can delegate tasks to them, user acceptability and trust an important issue – Can interact as cooperative teams or self-interested individuals – Interaction protocols – Coordination Strategies – Negotiation Protocols � Agents adapt to their environment, user, task and each other – Adapt both at the individual level and at the societal level – Employ Alternate Methods – Learn from (and about) users and each other
Progress � RETSINA system infrastructure development – Java implementation � RETSINA agent architecture – increased planning sophistication in individual agents � Middle agents � Agent interaction protocols
Middle Agent Types preferences Capabilities initially known by initially known by provider provider + mid- provider + middle + re- only dle agent quester requester only (broadcaster) “front-agent” matchmaker/yellow- pages requester + mid- anonymizer broker recommender dle agent requester + mid- blackboard introducer/- arbitrator dle + provider bodyguard
Retsina Agent Architecture Control Knowledge Domain Facts Current Objectives Task Structures Schedule and Action Beliefs Database Action Execution Communications Planner Scheduler Monitor KQML Messages to & from Control Flow other agents Data Flow Plan Library
RETSINA Planning Mechanisms � hierarchical task network-based formalism � library of task reduction schemas – alternative task reductions – contingent plans, loops � incremental task reduction, interleaved with execution – information gathered during execution directs future planning � resource and temporal constraints
A task Structure (Advertisement Task Structure) Advertise OK Content Make OK Send KQML Receiver Advertisement Message Down Reply-With Do-It Get OK Middle-Agent Outcome Name Provision Parameter
Progress (Contd.) � Agent interoperability – language for capability advertisement (Aardvark) – agent name server and distributed matchmaking a � Human Agent Interaction – Task Editor – Agent Editor – Human Agent Trust – Team TANDEM experiments a www.cs.cmu.edu/˜softagents/retsina/ans
Insert Aardvark.ppt: language for capability advertisement
Insert Interact.ppt: Agent Editor
Progress (Contd.) � Applications – Information filtering: Webmate a , DVINA – Agents in team aiding: ModSAF , multiagent air patrol, agent-aided aircraft maintenance b a www.cs.cmu.edu/˜softagents/webmate b This application is done in collaboration with the CMU wearable computer project.
ModSAF Vision
USER 1 USER 2 USER 3 Interface Interface Interface Agent 1 Agent 2 Agent 3 Materiel Agent Team Agent Doctrine Agent Path Force Case-Based Infosphere Planning Analysis Agent Agent Agent WebMate Agent Weather Region III Region I Region II Agent Agent Agent Agent ModSAF
���������� �������������� ������ ������� ������ ������������� ��������� ������ �������� �������� ������� �������� ANS ���������� ������ ���� ������ ������� ������ ����� ������ ������ ��������� ������� ������ �������� ������� ������������� ������ ���������� ����� ������� ������ ������ ������ ��������� ������ ���������������������������� ������ ��������
Insert AirMain.ppt: Aircraft Maintenance Task
Overview of the WebMate System � Use the multiple TF-IDF vectors to keep track of user interests in different domains which are automatically learned � Use the trigger pair model to automatically extract relevant words for refining search � The user can provide multiple pages as relevance guidance for information search
Insert WebMate.ppt (more detailed description)
Insert WebMateDemo.ppt (detailed description of WebMate demo)
Overview of Informedia � One of the six Digital Libraries Initiative projects funded by the NSF , DARPA, NASA and others in collaboration with WQED � A multimedia library that will consist of over one thousand hours of digital video, audio, images, text and other related materials � Uses combined speech, language and image understanding technology to transcribe, segment and index the linear video.
Recommend
More recommend
