Support Vector Machine Classification and Psychophysiological - PowerPoint PPT Presentation

Support Vector Machine Classification and Psychophysiological Evaluation of Mental Workload and Engagement of Intuition- and Analysis-Inducing Tasks Presenter: Joseph Nuamah Department of Industrial and Systems Engineering Advisor: Younho Seong March 3, 2017 Presenter: Joseph Nuamah March 3, 2017 1 / 34

Outline 1 Introduction 2 EEG 3 Aim 4 Hypotheses 5 Methodology 6 Results 7 Discussion and Conclusion 8 Presenter: Joseph Nuamah March 3, 2017 2 / 34

Introduction: Human Factors(HF) Issues in Autonomous Vehicles(AV) Appropriate Levels of Automation Human operator in-the-loop for failure mode operations Fail-safe mechanisms into AV Usefulness of UV interfaces Presenter: Joseph Nuamah March 3, 2017 3 / 34

Introduction: Problems with Improper Design of Automation Increased monitoring demands Cognitive overload Mis-calibration of trust in automation Inability to resume manual control Loss of situation awareness Degraded manual skills due to lack of practice Presenter: Joseph Nuamah March 3, 2017 4 / 34

Introduction: System Monitoring Increased UV operator’s role of monitoring and supervising automation Increased monitoring requirements add to cognitive load Vigilance - Failure detection worse under passive monitoring than under active control (Wickens & Kessel, 1980) UVs likely to contain more displays and instruments: Camera-fed screens Screens following up flight plan Instruments indicating intact communication requirements between Air Traffic Control operator and UV operator Presenter: Joseph Nuamah March 3, 2017 5 / 34

Introduction: Human Supervisory Control Human interactions with environment mediated by technological interfaces Supervisory control - operators oversee automated process, and continuously determine basis need to re-enter control loop Ongoing assessment based on comparison of actual and intended system performance UV operator in supervisory role requires information about target parameters decides how automation should proceed to achieve targets communicate appropriate instructions monitor process to ensure commands are understood and executed Components of Human Supervisory Control System: Human operator Interface Automation Presenter: Joseph Nuamah March 3, 2017 6 / 34

Introduction: Decision Making Design of interface requires understanding of judgment characteristics that they are to support effect of design on operator judgment Task characteristics play important role in determining cognitive mode likely to be used Higher correspondence between task characteristics and cognitive characteristics correlate with operator’s judgment accuracy Dual process theories of intuition and analysis used to explicate human cognitive system Presenter: Joseph Nuamah March 3, 2017 7 / 34

Introduction: Decision Making (cont’d) Table: Attributes of Intuition and Analysis Cognition (adapted from Evans & Stanovich, 2013) Intuition Analysis Does not require working memory Requires working memory Fast Slow High capacity Capacity limited Parallel Serial Nonconscious Conscious Automatic Controlled Holistic Analytic Relatively undemanding of cognitive capacity Demanding of cognitive capacity Experience-based decision making Consequential decision making Table: Inducement of Intuition and Analysis by Task Conditions (adapted from Hammond et al., 1987) Task Characteristic Intuition-Inducing State of Task Characteristic Analysis-Inducing State of Task Characteristic Number of cues Large ( > 5) Small Measurement of cues Perceptual measurement Objective, reliable measurement Distribution of cue values Continuous highly variable distribution Unknown distribution; cues are dichotomous; values are discrete Redundancy among cues High redundancy Low redundancy Decomposition of task Low High Availability of organizing principle Unavailable Available Degree of certainty in task Low certainty High certainty Time period Brief Long Presenter: Joseph Nuamah March 3, 2017 8 / 34

Introduction: Decision Making (cont’d) Behavioral and subjective traditionally measures used to measure judgment and decision making performance May not produce much information on the operator’s state Physiological measurements may be used Continuously available and collection does not interfere with operator’s task performance Measures range from blood flow or neural activity in brain to heart rate variability and eye movements Methods include: Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (FNIRs), etc Skin conductance, cardiovascular responses, muscle activity, pupil diameter, eye blinks,eye movements, etc Presenter: Joseph Nuamah March 3, 2017 9 / 34

EEG EEG signals represent summed postsynaptic potentials of neurons firing a rate of milliseconds Graph of time varying voltage difference between active electrode attached to scalp and reference electrode Table: Lobes and corresponding electrode label Lobe Electrode Frontal F Temporal T Central C Parietal P Occipital O Presenter: Joseph Nuamah March 3, 2017 10 / 34

EEG (cont’d) Figure: Waveform showing several ERP components Figure: Time domain EEG signal Table: Lobes and corresponding electrode label Frequency Band (Hz) Associated Tasks & Behaviors Delta (0.1-3) Lethargic, not moving, not attentive Theta (4-8) Creative, intuitive, distracted, unfocused Alpha (8-12) Meditation, no action Beta (12-30) Mental activity Gamma ( > 30) High-level information Figure: Main EEG waves Presenter: Joseph Nuamah March 3, 2017 11 / 34

EEG Indexes Spectral composition of EEG changes in response to changes in task difficulty or level of alertness Alpha, theta, beta all related to task engagement Task Engagement Index (TEI) is given by beta power alpha power + theta power Task Load Index (TLI) is given by frontal midline theta parietal alpha Presenter: Joseph Nuamah March 3, 2017 12 / 34

Aim Employ TLI to provide insight into cognitive load Employ TEI to provide insight into engagement Employ SVM to discriminate EEG signals recorded during execution of intuition-inducing and analysis-inducing tasks Employ objective measures (reaction time and percent correct), and subjective measure (NASA-Task Load Index) to validate objective EEG measures (TLI and TEI) Presenter: Joseph Nuamah March 3, 2017 13 / 34

Hypotheses Engagement required for analysis-inducing tasks would be different from that required for intuition-inducing tasks Mental effort required for analysis-inducing tasks would be different from that required for intuition-inducing tasks Presenter: Joseph Nuamah March 3, 2017 14 / 34

Methodology: Materials and Method Participants: Six participants (1 female, 5 males) Ages between 18 and 35 years All right-handed Normal vision No history of neuropsychiatric disorders Equipment: g.HIamp-256 channel biosignal amplifier g.GAMMAcap Electrode Type: AgCl Active electrode connector box comes with 64 channels g.Recorder used to record the EEG signals Presentation Software for stimuli delivery Presenter: Joseph Nuamah March 3, 2017 15 / 34

Methodology: Stimuli Baseline: Participants were instructed to relax and fixate on a blank screen for 60 s Intuition-inducing task: For each stimulus, two objects presented: fixation on left, and flashing face on right Participants were instructed to press LEFT mouse button if they thought face on RIGHT was a happy face, or press RIGHT mouse button if they thought face on RIGHT was face of someone who was afraid Stimuli taken from FACE database established by Ebner et al. (2010) Stimulus duration was approx. 6 s Inter Stimulus Time (ISI) was approx. 2 s Two blocks, each containing 30 trials Presenter: Joseph Nuamah March 3, 2017 16 / 34

Methodology: Stimuli (cont’d) Analysis-inducing task: For each stimulus, two multiplications were presented Participants instructed to determine which of two multiplications was larger Participants instructed to press LEFT mouse button if they thought multiplication on LEFT was larger or press RIGHT mouse button if Presenter: Joseph Nuamah March 3, 2017 17 / 34 they thought

Methodology: NASA-TLX Subjective workload assessment tool Overall workload score based on weighted average of ratings on six subscales: Mental Demand Physical Demand Temporal Demand Performance Effort Frustration Presenter: Joseph Nuamah March 3, 2017 18 / 34

Methodology: Procedure Sign informed consent and complete demographic questionnaire Fit g.GAMMAcap on scalp: 20 electrodes used, ear lobe as reference Calibrate electrode impedance Present experimental conditions Record Response time (RT) Complete NASA-TLX questionnaire Presenter: Joseph Nuamah March 3, 2017 19 / 34

Methodology: Signal Preprocesssing Raw EEG signals recorded at sampling rate of 256 Hz with Butterworth filter (0.01Hz high pass - 100Hz low pass) Notch filter with 60 Hz cutoff frequency to remove line noise Data re-referenced to average EEG epochs time-locked to stimulus presentation Data with amplitudes outside of range of -50 µ V to +50 µ V rejected Independent component analysis (ICA) correct EEG data contaminated by signals of non-neural origin SASICA software used to reject artifact independent components before EEG data analysis Presenter: Joseph Nuamah March 3, 2017 20 / 34

Methodology: Signal Preprocesssing Presenter: Joseph Nuamah March 3, 2017 21 / 34