speech processing 15 492 18 492
play

Speech Processing 15-492/18-492 Spoken Dialog Systems - Details of - PowerPoint PPT Presentation

Speech Processing 15-492/18-492 Spoken Dialog Systems - Details of Olympus modules - Dialog Task Design The Olympus Architecture Recog. Engine (SPHINX) Knowledge


  1. Speech Processing 15-492/18-492 Spoken Dialog Systems - Details of Olympus modules - Dialog Task Design

  2. The Olympus Architecture Recog. Engine (SPHINX) ����������� �������������� Knowledge ����������� ������� Source Phone / ��������������� �������������� Backend ������ ��������� Desktop ��������� ���������� �������� ������� Synth. Engine (SAPI/FLITE)

  3. RavenClaw Plan- -based dialog manager based dialog manager � Plan � Task- -independent engine independent engine � Task � � core Olympus library core Olympus library � � manage dialog by executing task specification manage dialog by executing task specification � � provides generic domain provides generic domain- -independent behavior independent behavior �  Help, repeat, … Help, repeat, …   Confirmation, non Confirmation, non- -understandings… understandings…  Dialog Task Specification � Dialog Task Specification � � dialog plan dialog plan � � interpretation context interpretation context �

  4. RavenClaw Architecture ������������������������������ ��������������������������� � Manages dialog by Manages dialog by � Captures all domain Captures all domain- - � � executing the dialog specific dialog (task) executing the dialog specific dialog (task) task specification logic using a task specification logic using a hierarchical description hierarchical description � Provides many Provides many � domain- -independent independent � Unique to each Unique to each domain � conversational application conversational application strategies strategies � Must be created for Must be created for � each application each application � Standard for most Standard for most � � Links to dialog engine Links to dialog engine applications applications � library library � No need to modify, just No need to modify, just � link shared library link shared library

  5. RavenClaw: Dialog Task Specification ���������� ��������� �� $������ �� ������������ ����������� �������� �� ���������� �� ���� �� ����� �� �� !�� �� �������%#������ �� ��"�� �� �� �������!��� �� ��#����� �� �� �%#� �� � Tree of dialog agents Tree of dialog agents � � Terminals: Inform, Request, Expect, Execute Terminals: Inform, Request, Expect, Execute � � Non Non- -terminals / Dialog agency: plans execution of child nodes terminals / Dialog agency: plans execution of child nodes � � Hierarchical Task Execution Network; each agent: Hierarchical Task Execution Network; each agent: � � Preconditions Preconditions � � Success & failure criteria Success & failure criteria � � Trigger (focus) criteria Trigger (focus) criteria � � Effects Effects �

  6. Sample Task Specification Code �������%#������ ����������� // /Madeleine/GeneralFeel �� ���������� �� ���� �� ����� DEFINE_AGENCY(CGeneralFeel, DEFINE_CONCEPTS( STRING_USER_CONCEPT(general_feeling, none)) DEFINE_SUBAGENTS( SUBAGENT(HowAreYou, CHowAreYou) SUBAGENT(Glad, CGlad) SUBAGENT(Sorry, CSorry)) SUCCEEDS_WHEN(COMPLETED(Glad) || COMPLETED(Sorry))) // /Madeleine/GeneralFeel/HowAreYou DEFINE_REQUEST_AGENT(CHowAreYou, REQUEST_CONCEPT(general_feeling) GRAMMAR_MAPPING("![Yes]>good, ![FeelingGood]>good, " "![FeelingSoSo]>soso, ![FeelingBad]>bad"))) // /Madeleine/GeneralFeel/Glad DEFINE_INFORM_AGENT(CGlad, PRECONDITION(C("general_feeling") == CString("good" )) PROMPT("inform glad_youre_good") ON_COMPLETION(FINISH(/Madeleine))) // /Madeleine/GeneralFeel/Sorry DEFINE_INFORM_AGENT(CSorry, PRECONDITION(C("general_feeling") != CString("good" )) PROMPT("inform sorry_youre_bad"))

  7. RavenClaw Task Specification Language (RCTSL) (Pseudo- -)declarative language )declarative language � (Pseudo � � Defines concept types Defines concept types � � Describes the task tree Describes the task tree � Set of C++ macros � Set of C++ macros � � Concept types and agents are classes Concept types and agents are classes � � Can use pure C++ code if necessary Can use pure C++ code if necessary � � Need to be recompiled when modified Need to be recompiled when modified �

  8. RCTSL Concepts Concepts are effectively RCTSL variables � Concepts are effectively RCTSL variables � � Store values for later use and manipulation Store values for later use and manipulation � Standard types � Standard types � � String, integer and String, integer and bool bool � User- -defined types defined types � User � � Structures and arrays Structures and arrays � Two main categories: � Two main categories: � � System System concepts concepts �  Store internal values, database results, etc. Store internal values, database results, etc.  � User User concepts concepts �  Capture entities obtained from the user Capture entities obtained from the user 

  9. How User Concepts get Values GRAMMAR_MAPPING directive directive � GRAMMAR_MAPPING � � Defines which grammar Defines which grammar slot(s slot(s) from Phoenix ) from Phoenix � are assigned to an expected concept are assigned to an expected concept // /MyBus/PerformTask/GetQuerySpecs/RequestOriginPl ace DEFINE_REQUEST_AGENT( CRequestOriginPlace, REQUEST_CONCEPT(origin) PROMPT("request origin_place") GRAMMAR_MAPPING("[origin_place], ![Place]") ) • Maps parsed value from grammar (slot Maps parsed value from grammar (slot • ) to concept origin origin ] ) to concept [origin_place origin_place] [

  10. Specifying Binding Scope Initiative can be controlled via binding scope � Initiative can be controlled via binding scope � � System vs. Mixed initiative System vs. Mixed initiative � Grammar mappings encode binding scope: � Grammar mappings encode binding scope: � � Special character before grammar slot name Special character before grammar slot name � � Strict (!): bind only when request agent is active Strict (!): bind only when request agent is active � � Open (@): bind always Open (@): bind always � � Default (Ø): bind only when request agent’s Default (Ø): bind only when request agent’s � subtask is active subtask is active // /MyBus/PerformTask/GetQuerySpecs/RequestOriginPl ace DEFINE_REQUEST_AGENT( CRequestOriginPlace, REQUEST_CONCEPT(origin) PROMPT("request origin_place") GRAMMAR_MAPPING("[origin_place], ![Place]") )

  11. RavenClaw Execution ����� ���������� ��������� �� $������ �� ������������ ����������� �������� �� ���������� �� ���� �� ����� �� �� !�� �� �������%#������ �� ��"�� �� �� �������!��� �� ��#����� �� �� �%#� �� ������������ ���������� ���� !�

  12. RavenClaw Execution ����� ���������� ��������� �� $������ �� ������������ ����������� �������� �� ���������� �� ���� �� ����� �� �� !�� �� �������%#������ �� ��"�� �� �� �������!��� �� ��#����� �� �� �%#� �� ������������ ���������� ���� !� "�!���� �

  13. RavenClaw Execution ����� ���������� ��������� �� $������ �� ������������ ����������� �������� �� ���������� �� ���� �� ����� �� �� !�� �� �������%#������ �� ��"�� �� �� �������!��� �� ��#����� �� �� �%#� �� ������������ ���������� ���� !� �����#� "�!���� �

Recommend


More recommend