Development & evaluation of human-centered technology basic - PDF document

1/13/17 Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer BCI Research @NCSU Summary Brain-Computer Interfaces (BCIs): Research in Communication, Control and Human Cognition January 13, 2017 Chang S. Nam, Ph.D., CHFP Associate Professor, Director Brain-Computer Interface (BCI) Lab Edward P. Fitts Department of Industrial & Systems Engineering North Carolina State university Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer BCI Research @NCSU Summary Agenda Human-centered computing research @ NCSU 1 2 Motivations for interfacing human brain to computer • Unlocking the locked-in • Decoding people’s thoughts • Factoring in human factors BCI research @ NCSU 3 • P300-based BCIs for communication • Control by BCIs ü SSVEP-based BCI to support collaborative work ü SMR BCI-controlled FES for hand-wrist motor function ü SSSEP-based hybrid BCI for behaviorally non-responsive patients • Neural correlates of human cognition Summary 4 • Require active involvement of HF/E Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU 1

1/13/17 Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer BCI Research @NCSU Summary Development & evaluation of human-centered technology …basic & applied research on human factors/ergonomics (HF/E) issues associated with design, development, and evaluation of human-centered technology Brain-Computer Interface & Affective Computing Neuroergonomics Model-based Remote Healthcare Intelligent Adaptive User Interface Human-Automation Interaction Human-Centered Computing Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer Unlocking the locked-in BCI Research @NCSU Decoding people’s thoughts Summary Factoring in human factors Suffering from severe motor disabilities… q People who are totally paralyzed, or “locked-in” Ø Nearly 2 million Amyotrophic lateral sclerosis (ALS), or Lou Gehrig's disease in USA, affecting 5 out of every 100K people worldwide ü 1 in 50 people living with paralysis – approximately 6 million people Ø Approx. 800K/yr people suffering from a new or recurrent stroke in USA, and 33 million worldwide (Mozaffarian et al., 2015) Ø Socio-economic impacts ü Impossible to do routine tasks such as going up steps, getting out of a chair, or swallowing ü Reduced independence & quality of life Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU 2

1/13/17 Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer Unlocking the locked-in BCI Research @NCSU Decoding people’s thoughts Summary Factoring in human factors How it feels like to be paralyzed? Try it! Loss of voluntary muscle control, but Cognitively intact !!! Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer Unlocking the locked-in BCI Research @NCSU Decoding people’s thoughts Summary Factoring in human factors Brain-computer interface (BCI): General architecture A non-muscular communication and control system that does not depend on the brain’s normal output pathways of peripheral nerves and muscles Brain Spatial Extracting Classifier Signal Filtering Features Building Temporal Features Classifier Feedback Filtering Signal Signal Processing Classification Feedback Acquisition Training Task Online Task Offline Analysis Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU 3

1/13/17 Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer Unlocking the locked-in BCI Research @NCSU Decoding people’s thoughts Summary Factoring in human factors Neural mechanisms of human cognition Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer Unlocking the locked-in BCI Research @NCSU Decoding people’s thoughts Summary Factoring in human factors Still long way to go: HF/E matters Lack of understanding of, or inattention to user’s interaction with BCI systems: Effects of task, environmental & individual differences 8 Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU 4

1/13/17 Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer P300-based BCIs for communication BCI Research @NCSU Control by BCIs Summary Neural correlates of human cognition [1] How does a P300-based BCI work? q P300 ERP: Physical & Behavioral Properties Ø A positive peak in voltage of an event-related potential (ERP ) at a maximum of roughly 300 ms at Pz > Cz > Fz ü Oddball paradigm , where two stimuli are presented with different probabilities: frequent standard stimuli and infrequent task stimuli ü Related to the engagement of attention and the processing of novelty ü Fairly stable in locked in patients; No initial training (Serby et al., 2005) Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer P300-based BCIs for communication BCI Research @NCSU Control by BCIs Summary Neural correlates of human cognition Effects of task, environmental & individual differences Ø To assess how background noise and screen size affect task performance , neural Research activity and cortical integration of users with and without severe motor Goal disabilities � Can they use BCIs in a mall food court, city park, or street café? q Participants Ø 10 healthy subjects: 8M & 2F (M = 27.9 / 3.6 yrs) Ø 7 CP and 3 ALS patients with speech difficulties (M = 35.8 / 13.3 yrs) q Data acquisition and preprocessing Ø To spell 6, 10-character phrases, using 16 Chs. Ø Referenced/grounded to right/left mastoid with amplifier Ø Sampling rate of 256Hz; band-pass filtered 0.5 - 30Hz Source : Li et al., (2011); Li & Nam (2011); Nam et al. (2009; 2010; 2012) Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU 5

1/13/17 Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer P300-based BCIs for communication BCI Research @NCSU Control by BCIs Summary Neural correlates of human cognition Effects of task, environmental & individual differences q EEG Coherence Ø Synchronization of oscillatory cortical activity between brain regions within a certain frequency band Ø An increase in coherence is thought to reflect reduced cortical differentiation and specialization, when two coherences are compared q Cxy( ω ): coherence value between signals x and y § Φ xy( ω ): value of cross-correlation power spectrum of signals x, y § Φ xx( ω ): value of auto-correlation power spectrum of signal x § Φ yy( ω ): value of auto-correlation power spectrum of signal y Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer P300-based BCIs for communication BCI Research @NCSU Control by BCIs Summary Neural correlates of human cognition Effects of intra-individual differences on neural activity q Major findings Ø Task and environmental factor effects on real-world applicability Ø Differences between people with (~75%) and without (~95%) severe motor disabilities Ø Variations among people with motor disabilities Performance ALS (Good) CP (Moderate) CP (Bad) r 2 Topography P300 Pattern Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU 6

1/13/17 Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer P300-based BCIs for communication BCI Research @NCSU Control by BCIs Summary Neural correlates of human cognition Effects of inter-individual differences on coherence q Severe motor disability affected functional cortical integration Results Ø Significant difference in coherence between the two groups. Implications Ø Participants with severe neuromuscular impairments, as compared with the able-bodied group, were obliged to recruit more cortical regions, reflecting a less efficient operating strategy for the task. Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU Human-Centered Computing Research @NCSU Motivations for Interfacing Brain to Computer P300-based BCIs for communication BCI Research @NCSU Control by BCIs Summary Neural correlates of human cognition [2] How does a motor imagery-based BCI work? Temporal Dynamics q Sensory motor rhythm (SMR) at C3, Cz, & C4: An indicator of cortical activation/deactivation Ø Reduction/Enhancement in 8-14Hz alpha band and 15-25Hz beta band Spatial Dynamics Source : Jeon et al (2011); Nam et al (2011) Edward P. Fitts Dept of Industrial & Systems Engineering Brain-Computer Interface (BCI) @ NCSU 7