Communication Knowledge Sasikumar M
Overview ● Communication key to tutoring ● Different modes impose different benefits and challenges ● Scaffolded inputs and generated text (in audio also) often used ● Brief look at VR, and language
Synthetic humans ● 3D human models ● Add speech input and output ● For body language/gesture ● Communication skills ● Marketing – real estate
VR ● NASA heavy investment ● Eg. training for living in outer space – Simulate lack-of-gravity effect ● Students for atomic/molecular space, and high speed phenomenon. ● Psychiatric treatment
Graphics techniques ● Facial animation for emotions ● Virtual characters (artificial life)
Social intelligence ● Human emotions integral to human existence. ● If a tutor is adaptive enough for a student to believe that he is interacting with a human, then it is “socially intelligent” ● Affect and mood recognition important, and to act on it. ● Detect boredom, doubt, motivation level, etc
SI: approach 1 ● Behavioural analysis – Problem solving time – Nature and number of mistakes – Help request ● Bayesian network – to convert the pieces into useful measure ● Can help infer, with machine learning – To decide on current mood – Motivation – Engagement ● One target for educational data mining
SI: approach 2 ● Visual recognition ● Camera based – stationary, and moving views – Eye-brow position – Head movement – nod, shake ● eye-tracker ● Posture sensing devices ● Often head orientation itself provides a lot of cues on “focus”
SI: approach 3 ● Metabolic indicators ● Skin-conductance, heart rate ● Pressure sensors – on keyboard, mouse, etc ● Speech cue based recognition
Language ● Easiest to use; can be laborious if typing, etc ● Ideal – speech input and speech output – With gestures and intonation ● Many challenges still on this.
Language ● Types – Mixed initiative ● Both system and user can drive the interaction ● Can seek clarification to synchronise the mental models of both. ● Need to handle turn taking, grounding, and evaluation of input. – Single initiative (system) ● User only answers questions to the point – Single initiative (user) ● “I want to go from Boston to Miami this weekend” – Directed ● Focussed, system driven, restricted dialogue
Issues ● Language generation – What to say when and how. ● Language understanding – What is said and meaning what ● Complexity of language – Structure, ambiguity, etc ● Mixed initiative is hard – role management, turn taking, information extraction ● Use with other technologies – Facial features, audio, intonation, gestures, etc
Language generation ● Dear $name, you are now working on exercise $ex of chapter $c ● As explained before, gravity is a force in the downward direction. ● These are simple template filling. Can we do concept to language? ● Explain a concept map or an equation?
Eliza ● Among the earliest natural language communication system – I am feeling very depressed? – Why are you feeling depressed? – My mother scolds me everday. – Who else scolds you? – My father does not like me. – Tell me more about your father. – He does not care. – Tell me more.
Eliza ● Simple pattern matches can go a long way (apparently) ● No learner model, domain model, etc ● Multiple options for a pattern, to sound more natural.
Eliza... – I am feeling very depressed ? – Why are you feeling depressed? – My mother scolds me everday. – Who else scolds you? – My father does not like me. – Tell me more about your father. – He does not care. – Tell me more.
Geometry tutor ● They are 180. ● Could you be more specific? What are they? ● The angles are 180. ● You are heading in the right direction. But more precision is needed. Where does 180 come from? Are all angles 180?
... ● The angles in a triangle are 180. ● Is each angle measure equal to 180? Precisely what is 180? ● The sum of angles in a triangle is 180. ● Correct.
Ms Lindquist ● Please write an expression for the total time taken for the trip. ● HELP! ● Let me ask you an easier question. Please compute the time actually on the bike, if the speed was 20 miles per hour, the break time was 2 hours, and the distance travelled was 100 miles. ● 7
... ● Correct. What math did you use to get it? ● 2+5 ● Okay, how did you get 5? ● 100/20 ● Correct. How did you get 7? ● 2+100/20 ● Correct. Can you use the variables given in the problem, instead of the numbers? ● b+m/s. ● Correct.
Review ● What weakness and nice things do you see? ● What does it take to implement such capabilities? ● What kind of knowledge is involved?
Auto-tutor
Language interaction ● Imagine the types of knowledge and processing involved in realising such interaction ● Domain model – words, meanings, etc ● Student model – current context, previous sentences ● Discourse knowledge ● Social psychology
Processing NL ● Syntax – Nouns, verbs, inflections ● Semantics – Meaning and content ● Pragmatics – Contextualised meaning ● Discourse – Multi-sentential text, turn taking, etc
Language interaction ● Highly interdisciplinary – Computer science – Cognitive science – Social psychology – Discourse processes – Etc ● Advantage – Restricted domain, ambiguity will be less. ● Negative – Students may not follow grammar well!
Syntax Processing ● Tokens – words – constituents – parse tree ● I had dinner with him on Sunday; he left for London then. ● Connect R32 across V1 to V2, in series with R15. ● Word structure – Inflections -> morphology -> analysis and synthesis – Enjoyable = enjoy + able – Misunderstanding = mis+understand+ing – mis+un+? mis+under+stand+ing
... ● Word category (part of speech – POS) – Pos-tagger – I always bank on you. – Where is the axis bank? – The bank of Yamuna is so dirty. ● Sentence structure – PP attachment – Phrase nesting – Complex sentences with/without connectives – I saw the boy on the hill with a telescope.
Parse Tree S VP Subj adv verb NP Art Noun PP is dirty Yamuna of The bank Tags and structure only indicative
Parsing ● Objective to get some handle on the structure of the sentence ● Deep parsing – full parse tree. – Usually very hard, in general ● Shallow parsing – chunks and broad structure.
Semantic processing ● Identifying the entities referred to, and their relationship with other entities. ● What is the information in the utterance? Correctness? ● What does he want? ● Need domain model, world knowledge, etc
Resources ● Domain ontology – concepts, key terms, relationships, etc ● Wordnet – general relations among words – Synonym, generalisation/specialisation, etc ● POS taggers and chunkers for many languages. ● Latent Semantic Analysis
Tutor Model has Overlay Model Architecture Student Domain model Model has Buggy ITS Model isa isa Perturbation Dialogue Model approach Based Based Tutor tutor Constraint Based Tutor
Meaning Representation ● We need meaning to map to the domain model? – Locate relevant rules, concepts, etc – And what processing is needed to answer/meet the expectation.
Latent Semantic Analysis ● Words carry and represent the unit of meaning. ● The co-occurrence of words also significant. ● How to represent a document. ● Word-document mapping -> matrix ● Reduce dimension of this, with minimal loss of information. ● LSA – reformulated vectors derived from this. ● Used heavily in text matching (document retrieval etc)
Pragmatics processing ● Common conventions, and expectations. ● Is there a way? ● Can you tell me the time? ● How are you? ● In ITS, this can mean intervention models, nature of hints, etc.
Discourse Processing ● Multi-sentence text with sentence to sentence information transfer. – Lal went to Dubai for shopping. He likes that place. ● Dialogues – Turns, different world models, etc – Additional language components in this case.
Discourse processing ● Insert/recognise discourse markers – By the way, – Did you notice that... – Let us come back to ... ● Respond—Lead on –Invite – “Correct. But will this work? What do you say?” ● Anaphora resolution – Where do you put that? ● General protocols. – Hmm, hi, yes, ok, etc.
Dialogue based tutoring ● Good for inquiry teaching domains. ● Usual classroom pattern: – Initiation Response Evaluation ● Extend to: collaboratively improve the answer, and Evaluation again. ● This can be teaching model.
Dialog moves ● Positive immediate feedback (yeah, right, etc) ● Neutral immediate feedback (hmm) ● Negative immediate feedback (no, not quite, etc) ● Pumping for more (tell me more.. what else) ● Prompting (this is K and ?) ● Hinting ● Elaborating ● Correcting ● Summarising ● Requestioning
Recommend
More recommend
