CSE6335 Topics in Virtual Reality Wednesday March 25, 2009 Auditory Perception and Audition Addition of audio cues to VR environments Primary goal is to provide 3D audio cues (simulate audio coming from a particular direction) Technical problem is to do this with a limited array of speakers or a single set of headphones. Sound in its simplest form compression & rarefraction Wavelength/frequency Sound pressure level
Sound is a frequency-temporal process (can be represented as set of different frequencies at different times)
Threshold varies with frequency Ear as a filter
Middle Ear Operates as an impedance transformer Innear Ear Organs of balance Cochlea Pressure pathway through the Cochlea
Cochlea analyzes sound Auditory perception -> Audio in VR Direction azimuth and elevation Distance Spaciousness Presence and envelopment So how do we know where sounds come from?
Duplex Theory We use ITD’s and IID’s to localize targets. Clearly not a complete theory (but predicts many effects). Cannot account for Long wavelength sounds (sound diffracts around the head). Geometric confusions.. Cone of confusion Geometry shows a large number of points with the same ITD’s. So its more complex than just geometry Effect of head rotation Head rotations can be used to ‘solve’ front-back ambiguities
Head-related transfer function Sound source localization is dependent upon the frequency distribution of sound sources. Each individual ear filters frequencies in different directions and we use this information to localize sounds. Note: makes assumptions about the distribution of energy in the sound source. HRTFs Can measure these individually, construct ‘approximate’ ones, etc. KEMAR is a freely available one. HRTF Measurement
So, This suggests how to simulate 3d sound in VR Generate appropriate IID and ITD cues Generate appropriate HRTF cues Present these cues to the left and right ears. Headphone systems One ‘speaker’ to each ear. Invalidates the effects of the pinnae. Suffers from ‘inside the head’ localization. Speaker systems 2, 4, 5.1 and higher channel systems exist Question is ‘how do you spread sound over these speakers so that the sound comes from a specific direction?’ The first big problem is crosstalk
Crosstalk How do you ‘cancel out’ the unwanted sound? Reducing crosstalk Generate inverted (negative) version of appropriately delayed sound to ‘cancel’ out as much as possible the unwanted sound. Can do this recursively. Distributing sound over a speaker array Basic task is to apply amplitude gains (and possibly temporal delays) to sound given the sound location and distribution of speakers. Many different approaches, we will look at two
Ambisonics Amplitude-panning approach Sound has a gain factor at each microphone alpha is the angular separation between source and speaker i Four channel system Vector base amplitude panning Triplet-based system Direction to speaker n Direction to sound source p is a linear sum
Audition and VE VE relies on portaudio to actually talk to k speakers You write code that takes m input audio channel and distributes it over the k output channels. Your code is executed in real time as needed Audition and VE Simple left-right % mixer shown Summary Goal is to simulate 3d sound ITD, IID and HRTF cues Headphone and speaker array systems Basic approach is to distribute acoustic cues over the available sound sources
Recommend
More recommend
