Constraint Based Models for ITS Sasikumar M
Background ● Ohlsson’s philosophy – refined over 1.5 decades. University of Canterbury, New Zealand is the primary home. ● Learning from performance errors. – Declarative knowledge converted into procedural, problem solving knowledge. ● A stage of error recognition, and then error correction: Error recognition needs declarative knowledge ● Constraints capture the essence of this knowledge to be learned. 2
Constraints ● Relevance condition – is the constraint applicable here? ● Satisfaction condition – is it violated in this case (assuming applicable) – If violated, problem to be flagged. – Else, go ahead. ● Different constraints to check for different aspects and types of knowledge. ● Specified in a simple pattern matching 3 language built around Lisp.
Constraints ● Broadly syntax constraints and semantic constraints – Syntax – properties of concepts – Semantic – relationships, domain knowledge, etc. – Syntax constraints easier to develop ● Granularity important – affects feedback. – For pedagogic effectiveness, must focus on a small aspect of the domain 4
Constraint example 5
Example constraints 6
…. ● Constraints divide the set of solution states into equivalence classes – All states violating same set of constraints are in one class ● Has same intervention options! – All correct solutions in one class (violates zero constraints) 7
Constraint based tutor ● No need for a problem solver, as in model based tutoring. ● Instead of focusing on mistakes, focus on constraints that all correct solutions must obey. ● Hence suited for ill-defined domains also. – Planning, designing kind of problems – Programming, e.g. 8
Domain model ● Primarily constraints – derived and manually created. ● Ontology defines the concepts. ● Problems and their ideal solutions – multiple solutions ok. ● Solution decomposed into components which are concept instances. – Constraints can refer to these. 9
Ontology ● Common idea from AI literature, but different definitions by different groups. ● CBM opts to use that to make the constraint generation easier. ● Capture the concepts and their properties. ● Identify the key concepts, identify important/relevant properties, and organise into a hierarchy/graph. – Properties: min value, max value, type, 10 number of values, mandatory?, etc.
Another example 11
Matching ● Simple pattern matching with information on ideal and student solution ● RETE algorithm for faster matching – Repeating patterns matched only once ● A solution has from 70 to many hundred relevance conditions satisfied. 12
Feedback Generation ● Feedback should tell: – Where the error is – What constitutes the error – Re-iterate the domain principle violated by the student ● Studies show such feedback is more effective. ● “Error is in the sum. Sum is given as 93, should be 100, because the values are percentages.” ● Evaluation on the full solution, or on request. 13 – Make learner in control.
Feedback on solution 14
SQL-tutor ● A collection of databases, database of problems, and a collection of constraints relevant to the domain. – Some are problem specific constraints – Some are general domain constraints ● The first tutor based on the constraint based modelling idea. Now many others. ● No problem solver here. Match the solutions or steps in a pre-planned way. 15
SQL-tutor 16
SQL-feedback ● Multiple levels: – Correct/incorrect, error flag (where is error), hint – All errors, partial solution, complete solution ● Only first three as a routine – every submit increments level. ● Last three only on explicit request. 17
Student model ● Every constraint has a percentage of correct use – defines the student’s comfort. ● Problems requiring use of constraints in which student is poor is selected –curriculum sequencing. ● Uses information only from last N problems – to handle changing knowledge level, forgetting, etc 18
…. ● Initial model was Overlay. – Overlay in terms of constraints ● More complex models explored including Bayesian networks – Estimate probability of violating a constraint based on student behaviour so far ● Used in problem selection: value of a problem depends on predicted number of violated constraints. 19
Open student model ● Open student model – Allow students to inspect the student model – Proper visualisation needed ● Bar-graph on knowledge-level of each constraint ● Ontology models? – Support for OSM explored through SQL-tutor 20
Other examples ● EER-tutor: EER modelling ● CAPIT: English language – For elementary school children – Uses probabilistic student model – Punctuation and capitalisation rules ● KERMIT: conceptual database design ● RDB normalisation – sequence of steps 21
… ● Collect UML – object oriented software design – Design UML diagrams from textual description ● J-LATTE – for learning Java – Concept mode – design the solution without code ● Pseudo code style, blocks like if-then, loop, etc. – Coding mode – write the code 22
Scope ● Well defined problems with a clear correct answer. ● Ill-defined problems with multiple correct answers. 23
Authoring tool ● WETAS ● ASPIRE 24
ASPIRE ● Steps in developing a tutoring system – Specify the domain characteristics – sub- domains, problem solving steps, etc – Compose the domain ontology – Model the problem and substructures – Design the student interface – Adding problems and solutions – Generate syntax and semantic constraints – Validate the constraints with expert 25 – Deploy
Problem solving steps ● For addition of fractions ● Find the lowest common denominator (LCD) ● Convert fractions to LCD as denominator ● Add the resulting fractions ● Simplify the result 26
Ontology Entities and Number their properties... Fractions whole-number Improper LCD Fraction 27
Model the problem Step Soln component Concept Find LCD LCD LCD Convert fractions to LCD Fra1 numerator Improper fraction Fra2 numerator Fra1 denominator Fra2 denominator Sum the improper Sum numerator Improper fraction fractions Sum denominator Final reduced sum Final sum whole number Reduced fraction 28 Final sum numerator Final sum denominator
Student interface Lowest common denominator + Fractions with LCD as denom Sum of fractions Reduced sum 29
Use of ontology ● WETAS-ontology includes a tool to define the domain ontology. ● From this many constraints can be auto- generated to a large extent. ● For fraction, 11 syntactic constraints, and 39 semantic constraints generated, and used. ● 1: – R: you are in step 1, computing LCD – S: LCD field must not be empty. ● 2: 30 – R: you are in step 1, computing LCD
Constraint generation ● General rules for concept: – Concepts in student solution -> must also be in ideal solution. – Instances of each concept must match between ideal and student soln 31
…. For features and values: – For each selection concept, student has supplied correct value. – For each feature, if ideal solution has a value, student must use that value. – For each feature, if student has given value, ideal solution must also have the same value (no extraneous things in solution) – If a feature value is “required”, then it must be same in student solution, and ideal solution. 32
Thank you… 33
Case Study: Design an ITS
Specification ● Teach conversion from active voice to passive voice. ● He took a lecture -> a lecture was taken by him. ● I was rotis. -> Rotis were being made by me. ● Language: English ● Assume: vocabulary, basic language constructs (verb, noun, etc), sentence structure, etc
Questions ● What is the teaching approach? ● Screen structure ● What kind of errors are possible? ● How to recognise the errors? ● What kind of feedback? ● Generation of exercises? ● What is the domain representation? ● Student model?
Transformation He took a lecture A lecture was taken by him
Answers....
Recommend
More recommend
