Real-life needs Individualized Individualized Feedback in ITS - PowerPoint PPT Presentation

ICALL: Part I ICALL: Part I Real-life needs Individualized Individualized Feedback in ITS Feedback in ITS Detmar Meurers Detmar Meurers Universit¨ at T¨ ubingen Universit¨ at T¨ ubingen Intelligent Computer-Assisted Language Learning Introduction Introduction Real-life needs/CALL Real-life needs/CALL Part I: Individualized Feedback in Intelligent Tutoring opportunity ◮ The time a student can spend with an instructor/tutor opportunity An opportunity for CALL An opportunity for CALL Systems From CALL to ICALL From CALL to ICALL typically is very limited. Intelligent Tutoring Systems Intelligent Tutoring Systems TAGARELA TAGARELA ◮ In consequence, work on form and grammar is often Activity types Activity types Feedback Feedback System Architecture deemphasized and confined to homework so that the System Architecture Detmar Meurers The three models The three models time with the instructor can be used for communicative Expert model: NLP Expert model: NLP (Universit¨ at T¨ ubingen) Annotation-based setup Annotation-based setup Activity model activities. Activity model Relevance for processing Relevance for processing Challenges Challenges 1. Constraining system 1. Constraining system based on joint research with ◮ The downside is that the learner has relatively few input input Luiz Amaral (UMass Amherst) 2. Task specification 2. Task specification opportunities to gain awareness of forms and rules and 3. Appropriate Feedback 3. Appropriate Feedback Two Evaluation Insights Two Evaluation Insights receive individual feedback on errors. On interpreting accented On interpreting accented characters characters On Tokenization On Tokenization European Summer School in Language, Logic, and Information Wrapping up Wrapping up Bordeaux. July 27–31, 2009 Conclusion Conclusion 1 / 61 2 / 61 ICALL: Part I ICALL: Part I Real-life needs An opportunity for CALL Individualized Individualized Feedback in ITS Feedback in ITS OSU practice confirming dilemma Detmar Meurers Detmar Meurers Universit¨ at T¨ ubingen Universit¨ at T¨ ubingen A series of interviews with Spanish/Portuguese language Introduction Introduction ◮ The situation seems like an excellent opportunity for Real-life needs/CALL Real-life needs/CALL instructors (cf., Amaral & Meurers 2005) finds that opportunity opportunity developing Computer-Assisted Language Learning An opportunity for CALL An opportunity for CALL From CALL to ICALL From CALL to ICALL ◮ it can be difficult to achieve the communicative goal of (CALL) tools to Intelligent Tutoring Systems Intelligent Tutoring Systems TAGARELA TAGARELA an activity when students have problems using the ◮ provide individual feedback on learner errors and Activity types Activity types appropriate language forms and sentence patterns. Feedback ◮ foster learner awareness of relevant language forms Feedback System Architecture System Architecture The three models and categories. The three models ◮ But class activities that focus on form or grammar Expert model: NLP Expert model: NLP Annotation-based setup Annotation-based setup patterns are perceived as problematic since Activity model ◮ But existing CALL systems which offer exercises Activity model Relevance for processing Relevance for processing ◮ they reduce the pace of a lesson, and Challenges ◮ typically are limited to uncontextualized multiple choice, Challenges ◮ individual differences make it impossible to have all 1. Constraining system 1. Constraining system input input point-and-click, or simple form filling, and students do the same tasks in exactly the same time. 2. Task specification 2. Task specification ◮ feedback usually is limited to yes/no or letter-by-letter 3. Appropriate Feedback 3. Appropriate Feedback Two Evaluation Insights Two Evaluation Insights ◮ While instructors were very sceptical of CALL tools matching of the string with a pre-stored answer. On interpreting accented On interpreting accented characters characters aiming to replace human interaction, they support tools On Tokenization ◮ Example: “Spanish Grammar Exercises” (B. K. Nelson) On Tokenization Wrapping up Wrapping up ◮ practicing receptive skills Conclusion Conclusion ◮ reinforcing acquisition of forms ◮ raising linguistic awareness in general 3 / 61 4 / 61

ICALL: Part I ICALL: Part I Making CALL tools aware of language: NLP Aspects of Linguistic Modeling Individualized Individualized Feedback in ITS Feedback in ITS Detmar Meurers Detmar Meurers Universit¨ at T¨ ubingen Universit¨ at T¨ ubingen ◮ String matching is the most common technique used in Introduction ◮ A range of potentially relevant aspects of linguistic analysis: Introduction Real-life needs/CALL Real-life needs/CALL opportunity ◮ tokenization: identify words opportunity CALL to analyze student input, which works well when An opportunity for CALL An opportunity for CALL ◮ morphological analysis: identify/interpret morphemes From CALL to ICALL From CALL to ICALL ◮ correct answers & potential errors are predictable & listable Intelligent Tutoring Systems Intelligent Tutoring Systems ◮ syntactic analysis: identify selection, government and ◮ there is no grammatical variation TAGARELA TAGARELA agreement relations and word order requirements Activity types Activity types ◮ envisaged errors correspond directly to intended feedback Feedback Feedback ◮ formal pragmatic analysis: identify coreference System Architecture System Architecture The three models The three models relations, information structure partitioning, . . . ◮ But what if Expert model: NLP Expert model: NLP Annotation-based setup Annotation-based setup ◮ possible correct answers are predictable but not Activity model Activity model ◮ Computational tools identifying such linguistic properties Relevance for processing Relevance for processing (conveniently) listable for a given activity Challenges Challenges need to be integrated into CALL systems to obtain ◮ errors can occur throughout a recursively built structure 1. Constraining system 1. Constraining system input input language-aware “Intelligent” CALL (ICALL). ◮ individualized feedback is desired which requires 2. Task specification 2. Task specification 3. Appropriate Feedback 3. Appropriate Feedback information about the learner input that can only be Two Evaluation Insights Two Evaluation Insights On interpreting accented ◮ What architecture can the NLP analysis be integrated in? On interpreting accented obtained through linguistic analysis characters characters On Tokenization On Tokenization ⇒ An Intelligent Tutoring System ⇒ Use NLP to analyze student input in such cases! Wrapping up Wrapping up Conclusion Conclusion 5 / 61 6 / 61 ICALL: Part I ICALL: Part I Intelligent Tutoring Systems Components of an ITS Individualized Individualized Feedback in ITS Feedback in ITS Detmar Meurers Detmar Meurers Universit¨ at T¨ ubingen Universit¨ at T¨ ubingen ◮ Expert Model: Introduction Introduction ◮ An Intelligent Tutoring System (ITS) is a computer Real-life needs/CALL Real-life needs/CALL ◮ the knowledge that the ITS has of its subject domain, in opportunity opportunity An opportunity for CALL An opportunity for CALL program that intelligently interacts with the learner. our case the linguistic knowledge From CALL to ICALL From CALL to ICALL Intelligent Tutoring Systems Intelligent Tutoring Systems ◮ An ITS should be able to: TAGARELA TAGARELA ◮ Student Model (= Learner Model) ◮ accurately diagnose the knowledge structures and skills Activity types Activity types Feedback ◮ the component of the system keeping track of the Feedback of the student System Architecture System Architecture student’s current state of knowledge The three models The three models ◮ adapt instruction accordingly Expert model: NLP Expert model: NLP ◮ It allows the ITS to infer the student’s understanding of ◮ provide personalized feedback Annotation-based setup Annotation-based setup Activity model Activity model the subject matter and to adjust the feedback to the Relevance for processing Relevance for processing ◮ Since Hartley & Sleeman (1973) an ITS is recognized Challenges student’s needs. Challenges 1. Constraining system 1. Constraining system as consisting of at least three components: input input 2. Task specification 2. Task specification ◮ Instruction Model: ◮ the expert model 3. Appropriate Feedback 3. Appropriate Feedback Two Evaluation Insights Two Evaluation Insights ◮ the student model ◮ the component that stores pedagogical information, On interpreting accented On interpreting accented characters characters ◮ the instruction model how to conduct instruction On Tokenization On Tokenization Wrapping up ◮ It helps define strategies to deliver appropriate feedback. Wrapping up Conclusion Conclusion 7 / 61 8 / 61