Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu - PowerPoint PPT Presentation

Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 2017-11-27 Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 1/19

Introduction Motivations ◮ Interruptions divided into self-interruptions and external interruptions ◮ Self-interruptions are more costly ◮ Self-interruptions can originate from a loss of focus ◮ Self-interruptions should be detectable using biometric sensors Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 2/19

Introduction Research Questions ◮ (RQ1) Does the low-cost eye tracker have the sufficient accuracy to track the current reading line? ◮ (RQ2) Does eye gaze behave differently before a self-interruption? ◮ (RQ3) Can we detect eye gaze patterns that occur before a self-interruption? Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 3/19

Experiment Setup Figure: GazeReader setup using the Tobii Eye Tracker 4C after the participant didn’t look at the screen for some time. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 4/19

Analysis Session data 2017-11-12T01:06:21.913Z|FIXATIONDATA|369.73,715.79;17.47%,8.83%;<TEXT_LINE> 2017-11-12T01:06:21.915Z|FIXATIONEND|332.62,721.53;11.03%,35.74%;<TEXT_LINE> 2017-11-12T01:06:21.915Z|HEAD|6.08,107.60,702.73;-0.27,0.19,-0.07 2017-11-12T01:06:21.918Z|GAZE|357.64,718.33;15.37%,20.74%;<TEXT_LINE> 2017-11-12T01:06:21.933Z|GAZE|326.13,723.11;9.91%,43.14%;<TEXT_LINE> 2017-11-12T01:06:21.938Z|HEAD|6.08,107.60,702.73;-0.27,0.19,-0.07 2017-11-12T01:06:21.986Z|HEAD|6.08,107.60,702.73;-0.27,0.19,-0.07 2017-11-12T01:06:32.174Z|BLUR| 2017-11-12T01:06:32.175Z|ACTIVE|GazeReader.exe;Dialog 2017-11-12T01:37:11.421Z|REASON|distraction 2017-11-12T01:37:11.440Z|FOCUS| 2017-11-12T01:37:11.449Z|GAZE|872.82,534.01;4.50%,7.96%<TEXT_LINE> 2017-11-12T01:37:11.453Z|GAZE|871.96,532.24;2.08%,-0.34%<TEXT_LINE> 2017-11-12T01:37:11.456Z|GAZE|871.53,528.94;0.06%,97.28%;<TEXT_LINE> 2017-11-12T01:37:11.458Z|FIXATIONDATA|871.52,532.08;0.85%,-1.09%<TEXT_LINE> 2017-11-12T01:37:11.462Z|FIXATIONDATA|871.00,529.17;-0.04%,98.36%;<TEXT_LINE> 2017-11-12T01:37:11.467Z|FIXATIONDATA|871.55,524.30;0.06%,75.53%;<TEXT_LINE> Figure: Example session data around a distraction event. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 5/19

Analysis Data segmentation Stripped data 0 1 During normal reading Before self-interruption BLUR FOCUS (non-reading related activities) 0 1 0 ⎧ ⎪ ⎨ ⎪ ⎩ ⎧ ⎪ ⎨ ⎪ ⎩ t i t r1 BLUR FOCUS (reading related activities) 0 0 ⎧ ⎨ ⎩ ⎧ ⎨ ⎩ t p t r2 Figure: Data segmentation scenario: Switch to another window which is reading related (top) or self interruption (bottom). Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 6/19

Analysis Data segmentation (continued) 0 1 Stripped data During normal reading Before self-interruption Look away Look back (select “self-interruption”) Pop-up 0 1 0 ⎧ ⎪ ⎨ ⎪ ⎩ ⎧ ⎨ ⎩ ⎧ ⎪ ⎨ ⎪ ⎩ t i t w t r1 Look away Look back (select “external interruption/take note”) Pop-up external 0 0 interruption reading 0 0 related ⎧ ⎨ ⎩ ⎧ ⎨ ⎩ ⎧ ⎪ ⎪ ⎪ ⎨ ⎩ t p t w ⎧ ⎨ ⎩ t r1 t r2 Figure: Data segmentation scenario: Looking away from the screen due to self interruption (top), or external interruption or reading related tasks (bottom). Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 7/19

Analysis Features Table: Features Features Sub features fixation duration mean median variance min max count saccade duration mean median variance min max length mean median variance min max angle mean median variance min max Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 8/19

Analysis Classification Data for (t i ) (Average over 5 runs) 2/3 1/3 Testing Testing Training Labels Calculate metrics (SVM) Data Data Calculate metrics (Knn) Calculate metrics (LR) Training Training Testing Predicted Features Labels Features Labels Calculate metrics (…) SVM K nearest neighbors Logistic regression … Figure: Process of classifications for a ( t i ) Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 9/19

Analysis Metrics Table: Confusion matrix. positive: self-interruption, negative: concentrating on reading Predicted positive Predicted negative Labeled positive True positive :-) False negative :- | Labeled negative False positive :-( True negative :- | ◮ Accuracy = true all true positive ◮ Precision = predicted positive true positive ◮ Recall = labeled positive ◮ F1 score = 2 · precision · recall precision + recall Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 10/19

Analysis Classifiers 1. AdaBoostClassifier 2. DecisionTreeClassifier 3. GaussianNB 4. GaussianProcessClassifier 5. KNeighborsClassifier 6. LogisticRegression 7. MLPClassifier 8. QuadraticDiscriminantAnalysis 9. RandomForestClassifier 10. SVC (non-linear) 11. SVC (linear) Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 11/19

Results Accuracy 1 0.9 0.8 0.7 0.6 accuracy 0.5 0.4 0.3 AdaBoost DecisionTree GaussianNB 0.2 KNeighbors LogisticRegression MLP 0.1 QuadraticDiscriminantAnalysis RandomForest SVC (linear) 0 0 10 20 30 40 50 60 t i /s Figure: Accuracy scores of different t i s and classifiers. Each line represents the result of a classifier. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 12/19

Results Precision 1 AdaBoost DecisionTree 0.9 GaussianNB KNeighbors LogisticRegression 0.8 MLP QuadraticDiscriminantAnalysis RandomForest 0.7 SVC (linear) 0.6 precision 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 60 t i /s Figure: Precision scores of different t i s and classifiers. Each line represents the result of a classifier. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 13/19

Results Recall 1 0.9 AdaBoost DecisionTree 0.8 GaussianNB KNeighbors LogisticRegression 0.7 MLP QuadraticDiscriminantAnalysis RandomForest 0.6 SVC (linear) recall 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 60 t i /s Figure: Recall scores of different t i s and classifiers. Each line represents the result of a classifier. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 14/19

Results F1 Score 1 AdaBoost DecisionTree 0.9 GaussianNB KNeighbors LogisticRegression 0.8 MLP QuadraticDiscriminantAnalysis RandomForest 0.7 SVC (linear) 0.6 f1 0.5 0.4 0.3 0.2 0.1 0 0 10 20 30 40 50 60 t i /s Figure: F1 scores of different t i s and classifiers. Each line represents the result of a classifier. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 15/19

Conclusion Contributions ◮ We prove that before a self-interruption, the eye movement is different compared to when the reader concentrates on reading. ◮ We demonstrate with proper classifiers, we can detect the incoming self-interruptions by gaining 5-second eye movement data before the interruption event. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 16/19

Conclusion Limitations ◮ Insufficient data, some participants rarely self-interrupt. ◮ Many participants usually prefer to read the print paper. ◮ Tobii 4C driver only runs on Windows and not accurate enough. ◮ Classification is binary. ◮ Do not know which features weight more. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 17/19

Conclusion Future work ◮ How long in advance can we predict? ◮ Use pupil dilation metrics as features. ◮ Analyzing sentences/words: functional words vs content words. Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 18/19

Q&A Questions? Detecting Self-Interruptions during Reading Jan Pilzer and Sam Liu 19/19