Defense Def ense and and Homeland S Homeland Secu ecurity rity - PowerPoint PPT Presentation

U.S. Army Research, Development and Engineering Command Learning in Intelligent Tutoring Environments (LITE) Lab personnel at USMA, April 2011 (L-R):  Dr. Robert Sottilare  Dr. Heather Holden  Mr. Keith Brawner  Mr. Benjamin Goldberg Defense Def ense and and Homeland S Homeland Secu ecurity rity Sim imula ulation tion Realti ltime me Cl Clusterin stering Of Unla labell lled Se Senso sory y Da Data ta For r Traine ainee e Sta State te Assessment ssessment Septe Se temb mber r 2011 LITE TE Lab *UNCLASSIFIED – FOUO*

Trainee State Assessment for ITS 2 2 • Student Actions • Sensor Data • Assessment and Classification • Instructional Strategy Decision? Sensors Sensors Sensors Trainee Instructional Trainee Assessment / Strategy Decision Classification *UNCLASSIFIED – FOUO*

Difficulties of Sensing 3 3 • Real-time processing • People are not • Deterministic Algorithms consistent • Datastream problems – Day to day – Baseline to Baseline • Infinite Length • Unsupervised learning • Concept Detection • Concept Drift • Concept Evolution *UNCLASSIFIED – FOUO*

Potential Solutions 4 4 • Incremental Clustering – K-means – Agglomerate • Growing Neural Gas • Adaptive Resonance Theory *UNCLASSIFIED – FOUO*

Incremental K -Means Clustering 5 5 Algorithm: For each point • Strengths Compare point to all known clusters – Benchmark approach If no cluster is within vigilance – Well supported create new cluster here else move matched cluster up to <delta> in the direction of the recent point • Weaknesses – Must know K – Responds to data frequency – Partitions poorly – NP-hard (general case) – Order sensitive (inc. case) *UNCLASSIFIED – FOUO*

Agglomerate Clustering 6 6 Algorithm: Move the closest centroid towards the • Strengths datapoint – Modified inc. k -means Merge the two closest centroids, if – Accounts for cluster merging appropriate – Order insensitive Creates one redundant centroid Set redundant centroid equal to the – Do not have to know k datapoint • Weaknesses – None ( a priori ) – Low coverage ( a posteriori) *UNCLASSIFIED – FOUO*

Adaptive Resonance Theory 7 7 Algorithm: • Strengths Apply new input pattern Compute activation of all neurons – Order insensitive Select winning neuron – Merges Vigilence test – Responds to new If vigilence is relevant, add new pattern concepts Else not, test next best neuron Else (no neurons), initialize new neuron • Weaknesses – Box-shapes – Parameterization issues – NN issues (trending) *UNCLASSIFIED – FOUO*

Growing Neural Gas 8 8 Algorithm: If appropriate (current point does not correspond to • Strengths known information) create new reference arc – Responds to new concepts store error – Order sensitive Else, increment age of all arcs in this area move existing arcs towards new data, establish new ages for arcs Remove Aged arcs • Weaknesses If any non-emanating arcs exist, remove – Order sensitive If it is the time to add a new point (timing) Add a new reference point, halve the – Data frequency response distances of the existing arcs to this point, scale the – Gradient Descent existing errors Compute path of all arcs • Slows with additional data For this point against each class: If there are few related nodes, compute the probability of the point belonging to the lowest error class Else determine the modified shape of the cluster it is most likely to belong to *UNCLASSIFIED – FOUO*

Raw Datasets 9 9 Clusters, Unordered Clusters, Ordered X: ECG X: STE Y: GSR Y: Engagement *UNCLASSIFIED – FOUO*

Performance – Ordered Shapes 10 10 Inc. K-means Agglomerate GNG ART *UNCLASSIFIED – FOUO*

Performance – Unordered Shapes 11 11 Inc. K-means Agglomerate GNG ART *UNCLASSIFIED – FOUO*

Performance – EEG (STE/Engagement) 12 12 Inc. K-means Agglomerate GNG ART *UNCLASSIFIED – FOUO*

Performance – ECG/GSR 13 13 Inc. K-means Agglomerate (Failed) GNG ART *UNCLASSIFIED – FOUO*

Performance – 4x speed movie 14 14 *UNCLASSIFIED – FOUO*

Conclusions / Future Work 15 15 • Use constraint-based approaches – Semi-supervised clustering – Requires selection of initial algorithm • Associate performance data with state data – More complete student picture • Evaluate against validated dataset – Determine sensors to use • Evaluate in an ITS system – Includes instructional strategy selection • Use clusters as states, forecast movement between them *UNCLASSIFIED – FOUO*