Emergent Verbal Behaviour in Human-Robot Interaction Kristiina - PowerPoint PPT Presentation

Emergent Verbal Behaviour in Human-Robot Interaction Kristiina Jokinen & Graham Wilcock University of Helsinki

Jokinen & Wilcock CogInfoCom, Budapest, 2011 2

Pyro: Python Robotics � Open source Python robotics toolkit � http://pyrobotics.org � For teaching and research � Simulators and real robots � Artificial intelligence and robotics � Reinforcement learning, fuzzy decisions, neural networks, genetic algorithms, ... Jokinen & Wilcock CogInfoCom, Budapest, 2011 3

Pyspeech: Python Speech � Open source Python speech interface � http://code.google.com/p/pyspeech � Speech input and output � Speech recognition functions � Text-to-speech functions � For Windows computers � Uses Microsoft Speech Engine Jokinen & Wilcock CogInfoCom, Budapest, 2011 4

Emergent Verbal Behaviour 1 � Autonomous non-verbal behaviour � Robot performs actions silently � Autonomous verbal behaviour � Robot explains its actions � Interactive verbal behaviour � Human requests specific actions Jokinen & Wilcock CogInfoCom, Budapest, 2011 5

Non-verbal behaviour • Autonomous behaviour • Wander randomly • Avoid obstacles • Follow a wall • etc. • Robot acts silently Jokinen & Wilcock CogInfoCom, Budapest, 2011 6

Verbal behaviour • Robot explains its own actions • ”object on right” • (therefore) ”turn left” • Monologue • One-way info • Can be irritating • ”clear, clear, clear...” Jokinen & Wilcock CogInfoCom, Budapest, 2011 7

Cooperative Verbal Behaviour � Autonomous verbal behaviour � Robot explains its actions � Interactive verbal behaviour � Human requests ”talk less”, ”talk more” � Cooperative verbal behaviour � Robot changes its verbosity level � No repeating, only says new things Jokinen & Wilcock CogInfoCom, Budapest, 2011 8

Emergent Verbal Behaviour 2 � Human-initiated verbal interaction � Human asks robot to do something � ”Book me a flight to Paris” � Robot-initiated verbal interaction � Goal: find out what the human wants � Method: the robot asks the questions � ”Classical” spoken dialogue systems Jokinen & Wilcock CogInfoCom, Budapest, 2011 9

Jokinen & Wilcock CogInfoCom, Budapest, 2011 10

Spoken Dialogue Systems � Example: flight reservations system � Find out origin city, destination city ... � Find out depart day, return day ... � Use finite state transitions � Well-known in spoken dialogue systems � Pyro includes finite state machines � Example implemented directly in Pyro Jokinen & Wilcock CogInfoCom, Budapest, 2011 11

Fixed-domain Dialogues � ”Classical” dialogue systems � Fixed-domain database � Flights, cities, days � Easily add new flights, new cities � Cannot easily switch domains Jokinen & Wilcock CogInfoCom, Budapest, 2011 12

Open-domain Dialogues � Web-based information retrieval � Open-domain, any topic � Example: Wikipedia articles � Question-answering systems � Not yet ”natural” conversations Jokinen & Wilcock CogInfoCom, Budapest, 2011 13

Emergent Verbal Behaviour 3 � Human-initiated open-domain dialogue � Human decides the topic � ”Tell me about Shakespeare” � Robot continues with the topic � Gets Wikipedia article about Shakespeare � Reads out the first paragraph � How to continue the conversation? Jokinen & Wilcock CogInfoCom, Budapest, 2011 14

Jokinen & Wilcock CogInfoCom, Budapest, 2011 15

Topic and NewInfo � Jokinen PhD thesis (1994) Response Generation in Information-seeking Dialogues � Jokinen & Wilcock (2003) Adaptivity and Response Generation in a Spoken Dialogue System Jokinen & Wilcock CogInfoCom, Budapest, 2011 16

Topic and NewInfo � Topic � Need to know the current topic � Need to keep track of topic shifts � NewInfo � Gives some new information about Topic � Dialogue response is based on NewInfo � Topic link may be explicit or implicit Jokinen & Wilcock CogInfoCom, Budapest, 2011 17

Topics in Wikipedia � Articles � Article titles identify major topics � Explicit disambiguation of similar titles � Sections � Section headings identify sub-topics � Topic trees created by human authors Jokinen & Wilcock CogInfoCom, Budapest, 2011 18

Jokinen & Wilcock CogInfoCom, Budapest, 2011 19

Topics in Spoken Wikipedia � Dialogue, not monologue � Don’t read out entire article (monologue) � This would be irritating (need ”talk less”) � Avoid inappropriate sections � ”Would you like to know about his Life, his Plays or his Poems?” � ”Would you like to know about his See also, his Notes or his References?” Jokinen & Wilcock CogInfoCom, Budapest, 2011 20

NewInfo in Wikipedia � Text paragraphs � Give new information about subtopics � Typically, one subtopic per paragraph � Hypertext links � Links identify major NewInfos � Links might become new Topics � Clicking on a link causes a topic shift Jokinen & Wilcock CogInfoCom, Budapest, 2011 21

Jokinen & Wilcock CogInfoCom, Budapest, 2011 22

NewInfo in Spoken Wikipedia � Hyperlinks, NewInfos and topic shifts � ”Shakespeare was born and raised in Stratford-upon-Avon” (NewInfo) � ”Stratford-upon-Avon?” (topic shift) � ”Stratford-upon-Avon is a market town and civil parish in Warwickshire, England” � ”Warwickshire?” Jokinen & Wilcock CogInfoCom, Budapest, 2011 23

Feedback in Spoken Wikipedia � Text paragraphs � Speak a paragraph, stop for feedback � Positive or negative? Interested or not? � Feedback may be non-verbal � Eye gaze � Facial expression � Body language Jokinen & Wilcock CogInfoCom, Budapest, 2011 24

Emergent Verbal Behaviour 4 � Robot-initiated open-domain dialogue � Robot suggests a topic � If human interested, robot continues � ”Interesting” topics from Wikipedia � Wikipedia front page layout � ”On this day” articles � ”Did you know ...?” articles Jokinen & Wilcock CogInfoCom, Budapest, 2011 25

Jokinen & Wilcock CogInfoCom, Budapest, 2011 26