Hearing & Music (Chapter 10-11a) Lecture 18 Jonathan Pillow Sensation & Perception (PSY 345 / NEU 325) Spring 2019 1
Harmonics • Objects tend to vibrate at multiple “resonant frequencies” (integer multiples of some fundamental frequency) • most vibrations die down, but some persist because their wavelength is reinforced by the object’s physical properties • Auditory system acutely sensitive to harmonics Example: guitar string Fundamental F1 2nd harmonic F2 3rd harmonic F3 (1st harmonic) (2 x F1) (3 x F1) 2
Many sounds, including voices, are harmonic 3
If the fundamental of a harmonic sound is removed, listeners will still hear its pitch demo: https://oup-arc.com/access/content/sensation-and-perception-5e-student-resources/ sensation-and-perception-5e-activity-10-2?previousFilter=tag_chapter-10 4
Missing Fundamental • only 3 harmonics are needed 5
Only three harmonics are needed to hear a missing fundamental 2 • all harmonics are aligned at the fundamental freq. 3 • fundamental could therefore be conveyed by 4 temporal code (“phase locking”) 2+3+4 6
Only three harmonics are needed to hear a missing fundamental 2 • Could also be conveyed by 3 “pattern matching” of the place code on the cochlea 4 2+3+4 7
Complex Sounds Timbre : Psychological sensation by which a listener can judge that two sounds with the same fundamental loudness and pitch are dissimilar • conveyed by harmonics and other frequencies • Perception of timbre depends on context in which sound is heard https://oup-arc.com/access/content/sensation-and-perception-5e-student-resources/ Timbre demo: sensation-and-perception-5e-activity-10-3?previousFilter=tag_chapter-10 8
Figure 10.20 Timbre 9
Auditory Scene Analysis What happens in natural situations? • Acoustic environment can be a busy place with multiple sound sources • How does the auditory system sort out these sources? � Source segregation - processing an auditory scene consisting of multiple sound sources into its separate sources 10
Waveforms from all sounds are summed into a single waveform arriving at the ears 11
Cocktail party effect • We can “select out” and attend to one conversation even when many are present simultaneously • first documented by Colin Cherry, 1953 12
Cocktail party effect Cherry’s findings: • Same voice speaking, Presented to Both ears ⇒ Very Difficult • Same voice speaking, Separate ears ⇒ Easy 13
Cocktail party effect However, subjects: • couldn’t identify a single phrase from non-attended ear • couldn’t say for sure if it was English • didn’t notice a change to German • didn’t notice speech being played backward • Did notice : change from male to female speaker 14
Cocktail party effect • Suggests we can easily use spatial, timing, and spectral cues to separate sound streams, but cannot attend to multiple sound streams at the same time! 15
Continuity and Restoration Effects How do we know that listeners hear sounds as continuous? • Principle of good continuation : in spite of interruptions, one can still “hear” a sound • Experiments (e.g., Kluender and Jenison, 1992) suggest that missing sounds are restored and encoded in the brain as if they were actually present! 16
Continuity Effects 17
Also true for speech: Adding noise can improve comprehension original speech speech w/ gaps gaps filled by noise 18
Brain automatically fills in sound that is missing due to noise gap filled by noise (cough) speech with a gap Q: Can you tell which phoneme is missing? 19
Brain automatically fills in sound that is missing due to noise gap filled by noise (cough) speech with a gap Q: Can you tell which phoneme is missing? 20
Continuity and Restoration Effects in Music Beat-box tutorial: http://www.youtube.com/watch?v=8D7hCqGm0X0 21
Yanny vs. Laurel https://www.youtube.com/watch?time_continue=11&v=7X_WvGAhMlQ 22
Summary • Interaural timing differences (ITD) • Interaural level differences (ILD) • MSO, LSO • cone of confusion • head-related transfer function (HRTF) • harmonics • missing fundamental • timbre • auditory scene analysis • cocktail party effect • continuity and restoration effects 23
11 Music and Speech Perception 24
Music • Universal: found across cultures • Also distinctive of cultures • Can have strong emotional effects • Has strong physiological effects • Pythagoras: Numbers and music intervals • Strong link between music and mathematics 25
The sounds of music (i.e. “pitch”) extend across a frequency range from about 25 to 4200 Hz • 7 octaves • hard to identify octaves above this limit 26
Musical Intervals Octave : The interval between two sound frequencies having a ratio of 2:1 • Example: Middle C (C 4 ) has a fundamental frequency of 261.6 Hz; • notes that are one octave from middle C: � 130.8 Hz (C 3 ) (below) � 523.2 Hz (C 5 ) (above) 27
Helix showing two characteristics of musical pitch: 1) tone height - increasing with frequency 2) tone chroma - which note within a scale (determines “feel” of note) 28
Helix showing two characteristics of musical pitch: Octave determines pitch more strongly than frequency • C 3 (130.8 Hz) sounds more similar to C 4 (261.6 Hz) than to E 3 (164.8 Hz) • There is more to musical pitch than just frequency! 29
Chords G - B - D G Major Chord, several heights Certain combinations are considered pleasing, or “consonant” Varies across cultures, but tends to focus on even ratios of frequencies (F1): “fifth” - 2:3 “major third” - 4:5 30
Dissonant : Less elegant ratios of note frequencies e.g., diminished fifth or “tritone”: 7:5 (six semitones) Disapproved of since the middle ages: The name diabolus in musica ("the Devil in music") has been applied to the interval from at least the early 18th century. Johann Joseph Fux cites the phrase in his seminal 1725 work Gradus ad Parnassum. Georg Telemann in 1733 notes, "' mi against fa ', which the ancients called 'Satan in music'.” Because of that original symbolic association with the devil and its avoidance, this interval came to be heard in Western cultural convention as suggesting an "evil" connotative meaning in music. Today the interval continues to suggest an "oppressive", "scary", or "evil" sound. 31
• Melody : An arrangement of notes or chords in succession • Examples: “Twinkle, Twinkle, Little Star” or “Baa Baa Black Sheep” • defined by relationship between notes (i.e., relative pitch, not absolute pitch) • Key : refers to the reference or starting note (Same melody can be played starting from a different note) 32
Absolute vs. Relative pitch Most people have relative pitch - they can identify the interval from one note to another (eg, an octave or a fifth). absolute pitch - very rare ability (1 / 10,000 in US) to identify or create a musical note without any external reference • more prevalent among people who grew up in east Asia (influence of tonal languages?) • may also have genetic component (currently unknown) • musical training during early critical period seems to have influence • more common in people with autism / Aspberger’s • may make it difficult to play music transposed to different key 33
Emotional content Emotional content of music is: 1. psychological, not physical : - same set of notes can sound “happy” or “sad” depending on where you start (C major vs. A minor scale) 2. culture-dependent: - emotional coloration of sounds (even consonant and dissonant intervals) are not the same across cultures 34
Cultural differences • Some relationships between notes, such as octaves (2:1), are universal; • fifth (3:2) is next most common interval (harmonic series is a common inspiration for scales) However, cultures divide up the octave quite differently: • Western music: 12 semi-tone (equal-tempered) scale (but “blue note” in jazz/blues falls in between ) 7-note (eg major) scales for composition: do-re-mi-fa-so-la-ti • Arabic music: 24 quarter-tones • Japan, China, India many others: 5-note (pentatonic) scale • Indonesian pentatonic scale: pelog https://en.wikipedia.org/wiki/Pelog 35
Recommend
More recommend
