Topics in Brain Computer Interfaces Topics in Brain Computer - PowerPoint PPT Presentation

Topics in Brain Computer Interfaces Topics in Brain Computer Interfaces CS295- -7 7 CS295 Professor: M ICHAEL B LACK TA: F RANK W OOD Spring 2005 Michael J. Black - January 2005 Brown University

From what part of the brain should we record? Michael J. Black - January 2005 Brown University

Motor Systems SMA: Primary motor cortex (M1) involved in Posterior parietal cortex the planning Supplementary M1: directly motor cortex of complex involved in (SMA) movements producing muscle Posterior Parietal and in two- contraction. Cortex: involved handed in transforming movements. visual information to motor commands. Premotor Cortex: involved in the sensory guidance of movement and Premotor cortex motor planning. (PMA) Michael J. Black - January 2005 Brown University

Brown University Primary motor cortex (M1) Larynx Tongue Motor System Face Hand Arm Trunk Foot Hip Michael J. Black - January 2005

What is represented? Using wrist and fingers Using elbow as fulcrum Using shoulder as fulcrum (outstretched arm) Adapted from R. Shadmehr Michael J. Black - January 2005 Brown University

Signing Your Name Prefrontal Cortex: I’ll sign my name. Posterior Parietal: combine visual and somatosensory information to localize pen wrt body. Premotor cortex: plan motion of hand wrt target path. Cerebellum: formulate details of movement in terms of dynamics. Primary Motor Cortex: sends motor commands down spinal cord. Brain Stem maintains stable posture during writing. Michael J. Black - January 2005 Brown University

Summary Posterior Parietal Cortex: Basal Ganglia: Transforms visual cues into plans Learning movements, for voluntary movements. initiating movements. Motor cortex: Initiating and Thalamus directing voluntary movements Cerebellum: Learning movements and coordination Brainstem Centers: Postural control. Visual cues Spinal Cord: Reflex coordination Motor neurons Skeletal Muscles Michael J. Black - January 2005 Brown University

Brown University Motor Control Michael J. Black - January 2005

Controlling a Motor Prosthesis MI arm area of motor cortex. * know that activity of cells related to hand motion * accessible (in monkeys and humans) * hypothesis: natural for controlling continuous motion of a prosthesis Michael J. Black - January 2005 Brown University

How can we record the neural signals? Michael J. Black - January 2005 Brown University

Sensing the Brain Slow(sec) Source: Matt Fellows Non-invasive Invasive fMRI ~ 10 3 neurons TIME Optical imaging EEG 10 2 10 1 10 4 Fast (msec) 10 3 LFP 10 0 Spikes MEG SPACE Course(mm) Fine(microns) Michael J. Black - January 2005 Brown University

Cyberkinetics Array SEM image Extra-cellular recording 100 “ideal” microelectrodes 10x10 grid, 4x4 mm platform 1 or 1.5 mm long, Si shafts, Pt coated tips Glass separation Parylene insulation coating Michael J. Black - January 2005 Brown University

Array Utah = Bionic = Cyberkinetics array. Fixed electrode depths – can’t move them to get a better signal. Take what you get and make the most of it. Inventor: Richard Normann, Univ. of Utah. Michael J. Black - January 2005 Brown University

Signal Out Implant Surgical Procedure Connector skin Bone cap Bone Arachnoid Dura I III Cortex V 400 µm 500 µm VI White Matter J. Donoghue 1/2001 Michael J. Black - January 2005 Brown University

Surgical Implantation WARNING: Graphic images of surgical procedure follow. Michael J. Black - January 2005 Brown University

Preclinical Safety: Removal and Re- -implantation implantation Removal and Re Explant Second Implant First Implant F2 F2 + 4 wks Removed F1 Original Implant F1 Original Implant + 4 wks Removed F3 +3 months F3 +3 months Arrays can be removed and reimplanted. Successful recordings can be obtained from reimplanted arrays. Donoghue Lab Michael J. Black - January 2005 Brown University

Surgical Methods Bone flap fixation Skin closure Intended to follow human neurosurgical Percutaneous procedures and methods. Connector • Limit duration • Eliminate most foreign materials • use established surgical methods Michael J. Black - January 2005 Brown University

sig003a;SNR=5.539294 sig004a;SNR=8.510092 sig005a;SNR=5.858280 sig013a;SNR=15.437233 sig013b;SNR=5.333427 sig015a;SNR=13.058348 sig020a;SNR=5.916720 sig021a;SNR=4.573570 sig021b;SNR=5.601879 100 20 50 20 20 40 100 50 20 50 0 20 0 0 0 0 0 0 0 -20 0 -20 -50 -50 -100 -20 -20 -20 -40 -40 -50 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 -40 sig006a;SNR=5.993156 sig007a;SNR=5.306515 sig008a;SNR=6.905757 sig015b;SNR=8.720314 sig015c;SNR=6.930796 sig016a;SNR=9.538304 0 20 40 0 20 40 0 20 40 sig023a;SNR=3.877040 sig024a;SNR=10.029772 sig024b;SNR=5.714142 50 40 50 40 60 20 20 20 10 50 40 20 0 0 0 0 0 0 0 20 0 -10 0 -20 -20 -20 -50 -20 -50 -20 -40 -50 -20 -40 -100 -40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 sig010a;SNR=7.021763 sig011a;SNR=8.703303 sig012b;SNR=3.918911 sig016b;SNR=4.044562 sig017a;SNR=12.788792 sig018a;SNR=5.623947 0 20 40 0 20 40 0 20 40 sig024c;SNR=3.930796 sig025a;SNR=6.354530 sig025b;SNR=4.965227 50 20 10 20 50 20 0 50 20 0 0 0 0 0 0 -10 0 -20 0 -10 -50 -50 -20 -40 -20 -20 -20 -50 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 sig029a;SNR=5.305835 sig029b;SNR=4.215226 sig030a;SNR=4.760648 sig035a;SNR=7.124755 sig035b;SNR=7.190937 sig036a;SNR=5.132120 sig053b;SNR=5.631417 sig063a;SNR=4.964285 sig063b;SNR=7.248083 20 40 50 50 40 20 20 20 20 10 20 0 0 0 0 0 0 0 0 0 -20 -20 -20 -10 -20 -20 -40 -20 -50 -50 -40 -20 -40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 sig031a;SNR=4.598879 sig032a;SNR=9.948935 sig032b;SNR=10.123838 sig037a;SNR=6.818353 sig037b;SNR=6.569363 sig043a;SNR=10.220711 sig065a;SNR=6.912202 sig066a;SNR=12.661345 sig066b;SNR=7.682087 20 20 40 50 20 50 50 20 20 20 0 0 0 0 0 0 0 0 0 -20 -20 -20 -20 -20 -20 -50 -50 -50 -40 0 20 40 0 20 40 0 20 40 sig032c;SNR=6.934851 sig033a;SNR=10.746377 sig034a;SNR=4.974278 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 sig043b;SNR=6.420375 sig047a;SNR=5.598402 sig053a;SNR=6.838001 sig067a;SNR=7.024523 sig067b;SNR=5.579144 sig070a;SNR=5.273455 50 20 50 20 20 20 20 20 10 0 0 0 0 0 0 0 0 0 -10 -20 -20 -20 -20 -20 -50 -20 -40 -20 -50 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 0 20 40 sig070b;SNR=5.236397 sig074a;SNR=7.190510 sig074b;SNR=5.264159 Recorded waveforms 57 units 10 10 20 0 0 0 -10 -10 -20 -20 -20 0 20 40 0 20 40 0 20 40 Michael J. Black - January 2005 Brown University

Chronic Implants � From: Selim Suner * 39 implants in 17 macaque monkeys (February 1996-April 2003) * Recordings for 1098 days Many neurons every day (19 tests over 110 days) n = 80± 7 in Blue - no recording 3 recent MI implants Donoghue Lab Red - best recordings Michael J. Black - January 2005 Brown University

Implant Challenges • Electronics – Miniaturization – Encapsulation – Telemetry – Heat dissipation – Low power – On board signal processing and Nurmikko and Patterson spike sorting Chip-scale integration of array and electronics. Michael J. Black - January 2005 Brown University

Long term vision Nurmikko and Patterson Michael J. Black - January 2005 Brown University

Brown University Michael J. Black - January 2005

What do the neural signals encode? Michael J. Black - January 2005 Brown University

Language of the Brain Language of the Brain “If spikes are the language of the brain, we would like to be provide a dictionary… perhaps even providing the analog of a thesaurus.” Rieke, et al 1997. Michael J. Black - January 2005 Brown University

Some Terminology Sequence of spikes from a single neuron = “spike train” time Interspike Interval (ISI) ISI Distribution (normalized histogram) Michael J. Black - January 2005 Brown University

Neural “Coding” • How do cells represent information? • ie, how is representation “coded” in action potentials. • If we understand the encoding then we can tackle the “decoding” problem. • inference – from activity to encoded property Michael J. Black - January 2005 Brown University

Neural Coding What are the possibilities? You’ve got action potentials and now you want to represent “move the hand to the right”. How might you do it? Michael J. Black - January 2005 Brown University

Neural Coding What are the possibilities? 1. Localist encoding in on/off response . 2. Rate coding. 3. Precise timing – pattern of spiking carries information. 4. Ensembles code information that individuals can’t. 5. Synchronous firing within and across ensembles (it is the interdependencies that matter). Michael J. Black - January 2005 Brown University