Multi-Robot Learning for Continuous Area Sweeping Peter Stone Joint - PowerPoint PPT Presentation

Multi-Robot Learning for Continuous Area Sweeping Peter Stone Joint work with Mazda Ahmadi Learning Agents Research Group (LARG) Department of Computer Sciences The University of Texas at Austin LAMAS, July 2005 Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Multiagent Learning in LARG Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Multiagent Learning in LARG Transfer Learning in Keepaway [Taylor, Wed., 10:30] Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Multiagent Learning in LARG Transfer Learning in Keepaway [Taylor, Wed., 10:30] Multiagent Traffic Management [Dresner, 10:45] Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Multiagent Learning in LARG Transfer Learning in Keepaway [Taylor, Wed., 10:30] Multiagent Traffic Management [Dresner, 10:45] General Game Playing [Kuhlmann, Dresner] Winner, 2005 RoboCup coach comp. [Kuhlmann, Knox] Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Multiagent Learning in LARG Transfer Learning in Keepaway [Taylor, Wed., 10:30] Multiagent Traffic Management [Dresner, 10:45] General Game Playing [Kuhlmann, Dresner] Winner, 2005 RoboCup coach comp. [Kuhlmann, Knox] Learning for Continuous Area Sweeping [Ahmadi, 2005] Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Multiagent Learning in LARG Transfer Learning in Keepaway [Taylor, Wed., 10:30] Multiagent Traffic Management [Dresner, 10:45] General Game Playing [Kuhlmann, Dresner] Winner, 2005 RoboCup coach comp. [Kuhlmann, Knox] Learning for Continuous Area Sweeping [Ahmadi, 2005] Mostly single-robot Initial multi-robot results Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Project Description Definitions Area sweeping Continuous area sweeping Examples: cleaning robots, surveillance robots. Non-uniform sweeping Multi-robot sweeping Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Project Description Definitions Area sweeping � � � � �� �� Continuous area Closet sweeping Examples: cleaning robots, surveillance robots. Bathroom Non-uniform sweeping Multi-robot sweeping Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Project Description Definitions Area sweeping � � � � �� �� Continuous area Closet sweeping Examples: cleaning robots , surveillance robots. Non-uniform Bathroom sweeping Multi-robot sweeping Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Project Description Definitions Area sweeping Continuous area Closet sweeping Examples: cleaning robots, ������ ������ ����� ����� ��� ��� �� �� �� �� ��� ��� surveillance ������ ������ ����� ����� ��� ��� �� �� �� �� ��� ��� ������ ������ ��� ��� �� �� �� �� ������ ������ ��� ��� robots. ��� ��� ������ ������ ��� ��� ��� ��� ������ ������ ��� ��� Non-uniform Bathroom ��� ��� ������ ������ ��� ��� ��� ��� ������ ������ �� �� ��� ��� sweeping ������ ������ �� �� ��� ��� ������ ������ �� �� ��� ��� Multi-robot sweeping Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Project Description Definitions Area sweeping Continuous area Closet sweeping Examples: cleaning robots, surveillance robots. Non-uniform Bathroom sweeping Multi-robot sweeping Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Project Description Definitions Area sweeping Continuous area Closet sweeping Examples: cleaning robots, surveillance robots. Non-uniform Bathroom sweeping Multi-robot sweeping Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Project Description Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Outline Introduction and Motivation 1 Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Outline Introduction and Motivation 1 Single Robot Problem Specification 2 Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Outline Introduction and Motivation 1 Single Robot Problem Specification 2 Exploration Algorithm 3 Learning Expected Rewards Planning Correctness Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Outline Introduction and Motivation 1 Single Robot Problem Specification 2 Exploration Algorithm 3 Learning Expected Rewards Planning Correctness Results 4 Simulation Results Results on Real Robots Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Outline Introduction and Motivation 1 Single Robot Problem Specification 2 Exploration Algorithm 3 Learning Expected Rewards Planning Correctness Results 4 Simulation Results Results on Real Robots 5 Multi-robot Extensions Overview Negotiation Algorithm Results Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Outline Introduction and Motivation 1 Single Robot Problem Specification 2 Exploration Algorithm 3 Learning Expected Rewards Planning Correctness Results 4 Simulation Results Results on Real Robots 5 Multi-robot Extensions Overview Negotiation Algorithm Results Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Assumptions Closet Bathroom The environment Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Assumptions Closet Bathroom The environment is divided into grid cells (G). Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Assumptions Closet Bathroom The orientations: east, west, north and south . Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Assumptions Closet 12’:13’’ 12’:30’’ 12’:50’’ 12’:18’’ 12’:42’’ 13’:02’’ Bathroom LV[G]: last time that robot has visited cell g. Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

Introduction Problem Specification Algorithm Results Multi-robot Learning Assumptions (cont.) Time is considered in sequence of discrete steps. imp e : importance of detecting event e . Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

The problem is defined as: (S, A ; T sa ; P eg ; CF ): Introduction Problem Specification Algorithm Results Multi-robot Learning Definitions Formal Definition Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

The problem is defined as: ( S , A ; T sa ; P eg ; CF ): Introduction Problem Specification � O � LV Algorithm Results Multi-robot Learning Definitions Formal Definition S : Set of states G Closet 12’:13’’ 12’:30’’ 12’:50’’ 12’:18’’ 12’:42’’ 13’:02’’ Bathroom Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping

; A ; T sa ; P eg ; CF ): Introduction Problem Specification Algorithm Results Multi-robot Learning Definitions Formal Definition The problem is defined as: ( S A : Set of possible actions Closet Bathroom Peter Stone, UT Austin Multi-Robot Continuous Area Sweeping