A novel EEG-based spelling system using N100 and P300 Hikaru Sato - PowerPoint PPT Presentation

A novel EEG-based spelling system using N100 and P300 Hikaru Sato and Yoshikazu Washizawa The University of Electro-Communications, Tokyo, Japan. September 2, 2014 Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 1 / 30

Table of Contents Motivation 1 Brain computer interfaces Event-related potentials Previous studies (P300-speller) and technical issues Proposed method 2 N100 ERP components New design of stimulus Experiment and Result 3 Summary and future plans 4 Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 2 / 30

Introduction What is BCI? Brain Computer Interface ⊲ Without physical movement ⊲ By only brain signal ⊲ Mental task Support human by using brain signal Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 3 / 30

Introduction ERP; Event-related potential ERP ⊲ Caused by cognition and mental task ⊲ ERP components; N100, N200, P300 P300 ⊲ Peak is roughly 300ms ∼ 500 ms ⊲ Observed in the oddball task Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 4 / 30

Introduction ERP; Event-related potential P300 waveform −6 x 10 6 4 Amplitude [V] 2 0 −2 −4 −6 0 0.2 0.4 0.6 Time [s] Averaged brain signal of Subject A for each 16 channel in the proposed system Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 4 / 30

Previous work P300-speller (Farwell et al,1988) Feature components ⊲ P300 → Detect the target row and column Stimuli ⊲ 12 images (6 rows and 6 columns) ⊲ Flash row and column in random order How to input ⊲ Respond the row and column which include the target character Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 5 / 30

Previous work How to detect the target character “Row” × “Column” ⇒ “Target character” Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 6 / 30

Previous work Technical issues of P300-speller Require at least 12 flashes to input one character ⊲ Restrict the improvement of typing speed ⊲ Make the user fatigue At least one character flashes twice in a row Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 7 / 30

Proposed method Proposed method: Purpose and Approach ⋆ Purpose ⊲ Increase Information transfer rate and typing speed ⊲ Same stimulus never flashes twice in a row ⊲ Reduce the user’s fatigue ⋆ Approach ⊲ Uniquely designed stimulus images ⊲ P300 ⇒ Detect the target stimulus image ⊲ N100 ⇒ Detect the user’s gazing position Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 8 / 30

Proposed method Characteristics of visual N100 Elicited without user’s cognitive process Peak is around 150ms after the stimulation Amplitude is larger for intended location than for non-intended location N100 waveform −6 3x 10 2 Amplitude [V] 1 0 −1 −2 −3 0 0.1 0.2 0.3 Time [s] Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 9 / 30

Proposed method Detection of gazing position using N100 Detect the gazing position by N100 response Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 10 / 30

Proposed method Peoposed method Feature components ⊲ P300 → Detect the target image ⊲ N100 → Detect the gazing position Stimuli ⊲ 9 images (4 characters and 2 blanks) ⊲ Flash in random order How to input ⊲ Gaze at the position presenting the target character and respond the target character. Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 11 / 30

Proposed method How to detect the gazing position Gaze at position No. 1. Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 12 / 30

Proposed method How to detect the target character “Image” × “Position” ⇒ “Target character” Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 13 / 30

Proposed method Advantages of the proposed method Reduce the number of stimulus images ⊲ 12 in P300-speller → 9 in proposed method ⊲ Increase typing speed ⊲ Reduce the user’s fatigue Every character is never presented twice in a row ⊲ Easy to conduct mental task ⊲ Only one response in a loop (Two in P300-speller) Extend the distance between characters ⊲ Prevent the user from responding neighbors of target by mistake Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 14 / 30

Experimental condition Experimental condition Method P300-speller Proposed method Stimulus images Total number of flashes 24 (12 × 2 loops) 18 (9 × 2 loops ) Flash duration 125 ms + 62.5 ms Trial duration 4.5 sec 3.375 sec Sampling rate : 512 Hz EEG electrodes : 16 channels Subjects : 10 healthy males Trials : 40 trials Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 15 / 30

Analysis Feature extraction range P300 ： 125ms - 625ms N100 ： 100ms - 250ms Averaged waveform (Cz) Averaged waveform (FCz) −6 −6 x 10 x 10 P300 N100 6 6 non−P300 non−N100 4 4 Amplitude [V] Amplitude [V] 2 2 0 0 −2 −2 −4 −4 −6 −6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 Time [s] Time [s] Range for P300 Range for N100 Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 16 / 30

Analysis Classification and Evaluation Preprocessing ⊲ 1-13Hz bandpass Butterworth filter ⊲ 49-51Hz bandstop Butterworth filter ⊲ Downsampling: 512Hz → 64Hz Classification method ⊲ Linear Support Vector Machine (Linear SVM) Evaluation ⊲ Five-fold cross-validation ⊲ Information transfer rate (ITR) [bit/sec] is given by ITR = 1 1 − P { } log 2 N + P log 2 P + (1 − P ) log 2 (1) T N − 1 T : Input time P : Accuracy N : The number of character , subject to P > 1 / N Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 17 / 30

Result Averaged signal waveform of P300 −6 −6 Averaged waveform (Cz) (P300−speller) Averaged waveform (Cz) (Proposed) x 10 x 10 P300 P300 6 6 non−P300 non−P300 4 4 Amplitude [V] Amplitude [V] 2 2 0 0 −2 −2 −4 −4 −6 −6 0 0.2 0.4 0.6 0 0.2 0.4 0.6 Time [s] Time [s] P300-speller Proposed mthod Averaged waveform of Subject A Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 18 / 30

Result Averaged signal waveform of N100 −6 Averaged waveform (FCz)(Proposed) x 10 N100 6 non−N100 4 Amplitude [V] 2 0 −2 −4 −6 0 0.2 0.4 0.6 Time [s] Averaged waveform of Subject A Red line : Gaze at characters Blue line : Gaze at blanks Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 19 / 30

Result Detection performance of ERP components Averaged classification accuracy and standard deviation of 10 subjects P300 [%] N100 [%] P300-speller 80.9 ± 7.4 – Proposed 74.0 ± 18.4 88.5 ± 10.4 P300 classification P300-speller Proposed The number of training samples 64 32 The number of classes 6 9 N100 detection performance ⊲ The number of averaging N100: 6 times P300: 2 times Hikaru Sato and Yoshikazu Washizawa (The University of Electro-Communications, Tokyo, Japan.) A novel EEG-based spelling system using N100 and P300 September 2, 2014 20 / 30