O e Overview & Aim e & Investigating common properties of - PowerPoint PPT Presentation

O e Overview & Aim e & Investigating common properties of across-channel processing in time, space and speech perception i i ti d h ti Shuji Mori K Kyushu University h U i it JSPS Kiban A Grant Kickoff Workshop August 2, 2013

Across-channel processing Across channel processing Channel – Independent information pathway 1600Hz 800Hz 400Hz 200Hz 200Hz 100Hz 50Hz 25Hz ‘frequency’ channels Auditory processing across channels Auditory processing across channels – Time: between-frequency gap detection – Space: between-ear gap detection S b t d t ti – Speech: voiced-voiceless categorical boundary 1 Psychophysics Lab. Department of Informatics Faculty of ISEE

Between-frequency gap detection Between frequency gap detection Gap detection Measurement of auditory temporal resolution (Moore & Glasberg, 1988; Plomp, 1964, etc.) Traditional, ‘within-frequency’ task y equency Fre No gap 5 ms Time Gap thresholds below 5 msec (Scharf & Buus, 1986) 2 Psychophysics Lab. Department of Informatics Faculty of ISEE

Between-frequency gap detection Between frequency gap detection Between-frequency task ency No gap 5 5 ms 10 10 ms 80 80 ms Frequ Time (Phillips et al., 1997, JASA, pp.3697, Fig.2) 3 Psychophysics Lab. Department of Informatics Faculty of ISEE

Between-ear gap detection Between ear gap detection de R R Ear/Sid L Leading Trailing marker Time Dependence on leading marker duration (Phillips et al., 1997, 1998) ( ) Headphones Speakers shold (ms) Gap thres (Adapted from (Adapted from Phillips et al., p , Phillips et al., p , 1998, JASA, 1997, JASA, Leading marker duration (ms) pp.3702, Fig.5) pp.2066, Fig.2) 4 Psychophysics Lab. Department of Informatics Faculty of ISEE

Between-ear gap detection Between ear gap detection Similarity to between-frequency thresholds (Phillips et al., 1997) 1997) (Adapted from Phillips et al Phillips et al., 1997, JASA, pp.3698, Fig.3) Leading marker duration (ms) Headphones Speakers shold (ms) Gap thres (Adapted from (Adapted from Phillips et al., p , Phillips et al., p , 1997, JASA, 1998, JASA, Leading marker duration (ms) pp.2066, Fig.2) pp.3702, Fig.5) 5 Psychophysics Lab. Department of Informatics Faculty of ISEE

Voiced-voiceless categorical boundary Voiced voiceless categorical boundary Phoneme contrasts of stop consonants /b/ - /p/, /d/ - /t/, /g/ - /k/ Voice onset time (V0T) as perceptual cues (Liberman et al., 1957; Lisker & Abramson, 1964) l 1957 Li k & Ab 1964) 100 /ba/ /ba/ percentage 75 /pa/ /ba/ ntification p 50 /pa/ 25 Consonant Consonant Iden VOT Vowel 0 19 30 41 52 63 74 85 Time Time VOT (msec) (Based on Oyama, 2013, pp.11, Fig. 6) 6 Psychophysics Lab. Department of Informatics Faculty of ISEE

Voiced-voiceless categorical boundary Voiced voiceless categorical boundary Similarity to between-frequency thresholds (Phillips et al., 1997 Elango an & St art 2008) 1997; Elangovan & Stuart, 2008) cy requenc Fr Time (Adapted from Elangovan & Stuart, 2008, Ear & Hearing, pp.769, Fig.6) 7 Psychophysics Lab. Department of Informatics Faculty of ISEE

Research questions Research questions 1. Common properties in time, space, and speech Does same processing mediate those three domains of auditory perception? What really is across-channel processing? 2. Brain mechanism Where in auditory pathway does across-channel processing take place? 3. Possible operation in other domains Vision Perception of geminate consonants 8 Psychophysics Lab. Department of Informatics Faculty of ISEE

JSPS Kiban A Grant JSPS Kiban A Grant ‘Investigating common properties of across-channel processing in time, space and speech perception’   FY 2013 to 2017 FY 2013 to 2017  Shuji Mori (chief), Kaoru Sekiyama, Keiji Iramina, Nobuyuki Hirose Willy Wong Makiko Sadakata Nobuyuki Hirose, Willy Wong, Makiko Sadakata, Takako Mitsudo 9 Psychophysics Lab. Department of Informatics Faculty of ISEE

Common properties Common properties Oyama (2013) Correlations of individual gap thresholds and /b/ - /p/ boundaries (cf. Elangovan & Stuart, 2008) • Between-frequency ency • Between-ear • /ba/ /pa/ identification • /ba/-/pa/ identification Frequ 21 native Japanese listeners 21 native Japanese listeners Time 10 Psychophysics Lab. Department of Informatics Faculty of ISEE

Common properties Common properties Between-frequency and -ear positively correlated shold (msec) shold (msec) shold (msec) 50 50 50 40 40 40 30 30 30 20 20 20 20 20 20 Gap Thres Gap Thres Gap Thres 10 10 10 2000Hz 4000Hz 1000Hz 0 0 0 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 Gap Threshold (msec) Gap Threshold (msec) Gap Threshold (msec) /b/-/p/ boundary correlated with between-frequency, but not with between ear not with between-ear ry (msec) ry (msec) 40 40 30 30 VOT Boundar VOT Boundar 20 20 10 10 Between-ear 2000Hz 0 0 0 10 20 30 40 50 0 10 20 30 40 50 Gap Threshold (msec) Gap Threshold (msec) 11 Psychophysics Lab. Department of Informatics Faculty of ISEE

Across-channel processing Across channel processing Relative timing (Phillips, 1999) Monitoring offset of leading and onset of trailing marker Reflects central computation Discontinuity detection Performed peripherally Performed peripherally 12 Psychophysics Lab. Department of Informatics Faculty of ISEE

Across-channel processing Across channel processing Distinction between two operations? (Phillips et al., 1997, JASA, pp.3697, Fig.2) Why takes longer for larger separation? 13 Psychophysics Lab. Department of Informatics Faculty of ISEE

Across-channel processing Across channel processing Attention shift Attention dwell time (Fitzgibbons et al., 1974) Imprecise time-stamping in unattended channel g (Phillips et al., 1997) Attention shifts fast to a cued frequency (Scharf et al., 2007) Attention disruption does NOT explain prolonged gap thresholds 14 Psychophysics Lab. Department of Informatics Faculty of ISEE

Psychometric function Psychometric function Within-frequency 1.0 esponses s (Florentine et al., tion of 1990) 0.5 0.5 Between-frequency Between-frequency Proport ‘Yes’ re Attention disrupt. Prediction 0 0 0 1 10 100 Gap duration (msec) Internal noise 15 Psychophysics Lab. Department of Informatics Faculty of ISEE

Brain mechanism Brain mechanism MEG MEG ABR (Coren et al., 1994, Sensation & perception, pp.204, Fig.6-17) 16 Psychophysics Lab. Department of Informatics Faculty of ISEE

Other domains Other domains Perceptual channel Frequency, ear/side, phoneme (vowels, consonants), … Vision Spatial frequency, color, orientation, … Temporal gap detection task (Yeshurun, 2004) p g p ( , ) Gap Time 17 Psychophysics Lab. Department of Informatics Faculty of ISEE

Other domains Other domains Geminate consonants /kk/, /tt/, /ss/, … Stop geminates contain silent intervals of over 100 msec g y equency Fre Time Time Largely cognitive (Sadakata & Sekiyama, 2011) but … 18 Psychophysics Lab. Department of Informatics Faculty of ISEE

Next … Next … Psychophysical theory of speech perception Speech is special (Liberman & Mattingly,1985; Fowler, 1986) Natural psychophysical boundary (Kuhl & Miller, 1978) VOT boundary identical for humans and nonhumans All phonetic contrasts are psychophysical in nature Identifying psychophysical correlates of phonetic contrasts 19 Psychophysics Lab. Department of Informatics Faculty of ISEE

Ultimately … Ultimately … Psychophysics of language Psychophysics of Japanese reading (with K. Sekiyama, W. Teramoto ) Combining psychophysical studies of speech and reading 20 Psychophysics Lab. Department of Informatics Faculty of ISEE

Thank you for your attention y y 21 Psychophysics Lab. Department of Informatics Faculty of ISEE