Classification of fMRI-based cognitive states Stephen LaConte - PowerPoint PPT Presentation

Classification of fMRI-based cognitive states Stephen LaConte Department of Neuroscience

“Organization” • Background and motivation • Basic principles • Evaluation of predictive models • Model interpretation • fMRI considerations • Supervised learning-based real-time fMRI • Tools and resources Stephen LaConte July 25, 2008

Background and Motivation • Complements univariate approaches (Friston, 1995; McIntosh, 1996; Strother, 2002; Moeller and Habeck 2006) • Early demonstration (Dehaene, 1998) • Methodology and validation (Strother, 2002; LaConte, 2003; Shaw, 2003; Mitchell 2004; Mourão-Miranda, 2005; Martinez-Ramon, 2006) • Representation of different classes of stimuli (Haxby, 2001; Cox and Savoy, 2003; Haynes & Rees, 2005; Kamitani & Tong 2005) • Detecting and tracking cognitive states (Polyn, 2005) • Natural representation for real-time fMRI (LaConte, 2007) • Image identification (Kay, 2008) • Data analysis competitions HBM ’06 and ‘07 Stephen LaConte July 25, 2008

Background and Motivation • Complements univariate approaches (Friston, 1995; McIntosh, 1996; Strother, 2002; Moeller and Habeck 2006) • Early demonstration (Dehaene, 1998) • Methodology and validation (Strother, 2002; LaConte, 2003; Shaw, 2003; Mitchell 2004; Mourão-Miranda, 2005; Martinez-Ramon, 2006) • Representation of different classes of stimuli (Haxby, 2001; Cox and Savoy, 2003; Haynes & Rees, 2005; Kamitani & Tong 2005) • Detecting and tracking cognitive states (Polyn, 2005) • Natural representation for real-time fMRI (LaConte, 2007) • Image identification (Kay, 2008) • Data analysis competitions HBM ’06 and ‘07 Stephen LaConte July 25, 2008

Background and Motivation • Complements univariate approaches (Friston, 1995; McIntosh, 1996; Strother, 2002; Moeller and Habeck 2006) • Early demonstration (Dehaene, 1998) • Methodology and validation (Strother, 2002; LaConte, 2003; Shaw, 2003; Mitchell 2004; Mourão-Miranda, 2005; Martinez-Ramon, 2006) • Representation of 1.0 different classes of stimuli (Haxby, 2001; Cox and Savoy, 2003; Haynes & Rees, 2005; Kamitani & Tong 2005) 0.8 • Detecting and tracking cognitive states mean prediction accuracy (Polyn, 2005) 0.6 • Natural representation for real-time fMRI (LaConte, 2007) 0.4 Air Alignment De-trend • Image identification DC Low High None (Kay, 2008) 100 PCs 75 PCs 50 PCs 25 PCs 10 PCs Smooth Low • Data analysis competitions 0.2 High HBM ’06 and ‘07 No Alignment DC De-trend, No Smooth: 0.0 0.30 0.40 0.50 0.60 mean reproducibility Stephen LaConte July 25, 2008

Background and Motivation • Complements univariate approaches (Friston, 1995; McIntosh, 1996; Strother, 2002; Moeller and Habeck 2006) • Early demonstration (Dehaene, 1998) • Methodology and validation (Strother, 2002; LaConte, 2003; Shaw, 2003; Mitchell 2004; Mourão-Miranda, 2005; Martinez-Ramon, 2006) • Representation of different classes of stimuli (Haxby, 2001; Cox and Savoy, 2003; Haynes & Rees, 2005; Kamitani & Tong 2005) • Detecting and tracking cognitive states (Polyn, 2005) • Natural representation for real-time fMRI (LaConte, 2007) • Image identification (Kay, 2008) • Data analysis competitions HBM ’06 and ‘07 Stephen LaConte July 25, 2008

Background and Motivation • Complements univariate approaches (Friston, 1995; McIntosh, 1996; Strother, 2002; Moeller and Habeck 2006) • Early demonstration (Dehaene, 1998) • Methodology and validation (Strother, 2002; LaConte, 2003; Shaw, 2003; Mitchell 2004; Mourão-Miranda, 2005; Martinez-Ramon, 2006) • Representation of different classes of stimuli (Haxby, 2001; Cox and Savoy, 2003; Haynes & Rees, 2005; Kamitani & Tong 2005) • Detecting and tracking cognitive states (Polyn, 2005) • Natural representation for real-time fMRI (LaConte, 2007) • Image identification (Kay, 2008) • Data analysis competitions HBM ’06 and ‘07 Stephen LaConte July 25, 2008

Background and Motivation • Complements univariate approaches (Friston, 1995; McIntosh, 1996; Strother, 2002; Moeller and Habeck 2006) • Early demonstration (Dehaene, 1998) • Methodology and validation (Strother, 2002; LaConte, 2003; Shaw, 2003; Mitchell 2004; Mourão-Miranda, 2005; Martinez-Ramon, 2006) • Representation of different classes of stimuli (Haxby, 2001; Cox and Savoy, 2003; Haynes & Rees, 2005; Kamitani & Tong 2005) • Detecting and tracking cognitive states (Polyn, 2005) • Natural representation for real-time fMRI (LaConte, 2007) • Image identification (Kay, 2008) • Data analysis competitions HBM ’06 and ‘07 Stephen LaConte July 25, 2008

Background and Motivation • Complements univariate approaches (Friston, 1995; McIntosh, 1996; Strother, 2002; Moeller and Habeck 2006) • Early demonstration (Dehaene, 1998) • Methodology and validation (Strother, 2002; LaConte, 2003; Shaw, 2003; Mitchell 2004; Mourão-Miranda, 2005; Martinez-Ramon, 2006) • Representation of different classes of stimuli (Haxby, 2001; Cox and Savoy, 2003; Haynes & Rees, 2005; Kamitani & Tong 2005) • Detecting and tracking cognitive states (Polyn, 2005) • Natural representation for real-time fMRI (LaConte, 2007) • Image identification (Kay, 2008) • Data analysis competitions HBM ’06 and ‘07 Stephen LaConte July 25, 2008

Background and Motivation • Complements univariate approaches (Friston, 1995; McIntosh, 1996; Strother, 2002; Moeller and Habeck 2006) • Early demonstration (Dehaene, 1998) • Methodology and validation (Strother, 2002; LaConte, 2003; Shaw, 2003; Mitchell 2004; Mourão-Miranda, 2005; Martinez-Ramon, 2006) • Representation of different classes of stimuli (Haxby, 2001; Cox and Savoy, 2003; Haynes & Rees, 2005; Kamitani & Tong 2005) • Detecting and tracking cognitive states (Polyn, 2005) • Natural representation for real-time fMRI (LaConte, 2007) • Image identification (Kay, 2008) • Data analysis competitions HBM ’06 and ‘07 Stephen LaConte July 25, 2008

“Organization” • Background and motivation • Basic principles • Evaluation of predictive models • Model interpretation • fMRI considerations • Supervised learning-based real-time fMRI • Tools and resources Stephen LaConte July 25, 2008

Supervised learning applied to fMRI Step 1: Train with labeled data Step 2: Use model to predict/decode y y t Data Estimated labels (y) Visual display Visual display Visual display label Time-labeled Time-labeled for time t scans scans Stimulus Stimulus ^ y t Data acquisition Data acquisition Data acquisition Image data I t I t Model Supervised Image data learning y represents the stimulus/behavioral categories for each volume. For classification, there is a set of stimulus categories y = {0, 1, 2, …, N}. Stephen LaConte July 25, 2008

Mathematical Representation of fMRI Data [ ] � X X X 1 2 N Stephen LaConte July 25, 2008

Mathematical Representation of fMRI Data Observations/Time/Intervals � � � � X X X 11 12 1 N � � � � X X X � � 21 22 2 N � � � � � � � � � � � X X X 1 1 M M MN Variables/Features/Space Stephen LaConte July 25, 2008

Classification with individual volumes x 2 � � x 1 � � x � � 2 � � � � � � � x N x 1 experiment time Stephen LaConte July 25, 2008

Temporal Regression x 2 y 10 10 8 8 6 6 4 4 2 2 0 0 x x 1 2 x 1 fMRI experiment y T = β + β f ( x ) x 9 0 6 3 L ( y , f ( x )) 1 time Stephen LaConte July 25, 2008

Classification scalar (or very high dimensional low dimensional) input space decision Classifier ˆ x g Stephen LaConte July 25, 2008

Training Data and Decision Boundary Stephen LaConte July 25, 2008

Training Data and Decision Boundary Stephen LaConte July 25, 2008

Training Data and Decision Boundary Stephen LaConte July 25, 2008

Multi-class Training Data Individual 2-class models 1 vs. 4 2 vs. 4 1 vs. 3 2 4 3 3 vs. 4 2 vs. 3 1 vs. 2 1 Stephen LaConte July 25, 2008

Multi-class Individual 2-class models 4-Class Model 1 vs. 4 2 vs. 4 1 vs. 3 2 4 3 3 vs. 4 2 vs. 3 1 vs. 2 1 Stephen LaConte July 25, 2008

Classification scalar (or very high dimensional low dimensional) input space decision Classifier ˆ x g Stephen LaConte July 25, 2008

Canonical Variates Analysis reduced scalar (or very high dimensional dimensional low dimensional) input space feature space decision ˆ x z g U T x Lz PCA rotation linear weights from feature based on eigenvectors of space training data class covariances vector Stephen LaConte July 25, 2008

PCA/CVA time voxels Data Matrix: X PCA via SVD: T T = Λ = U X V Q Truncate Q (model complexity) CVA: * = T * = C LQ LU X Columns of L are determined by the eigenvectors of W -1 B. W is the within class variance and B the between class variance, and both are obtained from Q. Stephen LaConte July 25, 2008