Towards usability of Higher Order Ambisonics in Digital Audio - PowerPoint PPT Presentation

Towards usability of Higher Order Ambisonics in Digital Audio Workstations Matthias Kronlachner Institute of Electronic Music and Acoustics University of Music and Performing Arts, Graz 2nd International Conference MUSIC AND TECHNOLOGIES Kaunas University of Technology, 8-10 November 2012, Kaunas, Lithuania

Osaka, 1970 Pischelsdorf (AT), 2011-?? 2

What is Ambisonic(s)? • Surround recording and playback technique, Michael Gerzon 1970s • Little commercial success so far, patents expired • Independent of playback loudspeaker configuration, scaleable • 2D and 3D sound-fields can be synthesized 3

What is Ambisonic(s)? • Not using phantom sources (VBAP), but trying to recreate original sound-field (WFS) Vector Based Amplitude Panning (VBAP) Wave Field Synthesis (WFS) 4

What is Ambisonic(s)? • Comparable to M/S stereo recording technique • A sound-field is “sampled” using spherical-harmonics First Order Ambisonics MS Stereo recording W (omni directional) Left = M+S X, Y, Z (figure of eight) Right = M-S 5

What is Ambisonic(s)? • 4 channel signal (W, X, Y, Z) is called B-Format • B-Format does not represent a speaker feed neither a microphone signal • trying to recreate original sound-field in a “sweet-spot” 2 2 1 1 y [m] 0 0 − 1 − 1 − 2 − 2 − 2 − 1 0 1 2 x [m] (h) HOA 28 2000Hz B-Format (1st order Ambisonics) plane wave from 0° (M. Frank) 6

What is Ambisonic(s)? • Loudspeaker feeds are a linear combination of the B- Format signals (Decoder Matrix) e.g.:  � p j = 1 W · ( 1 2) + X · (cos φ j cos θ j ) + Y · (sin φ j cos θ j ) + Z · (sin θ j ) √ L (3.12) • ALL speakers work together x to synthesize the sound-field LS1 LS2 LS8 φ LS7 LS3 y LS6 LS4 LS5 Regular 2D loudspeaker placement B-Format (1st order Ambisonics) 7

What is Ambisonic(s)? Higher Order Ambisonics (HOA) extends this approach to a bigger set of spherical harmonics 0th order (W) 1st order (X, Z, Y) 2nd order 3rd order 4th order 5th order Spherical harmonics up to 5th order 3D 8

Higher Order Ambisonics • more accurate spatial resolution in a larger area (“sweet-spot”) • i need more transmission channels, loudspeakers 9

Ambisonics in practice mic. calibration data rotation, Loudspeaker configuration Ambisonics zoom MicEnc Microphone Mono Sound N Encoder L L Manipulation Ambisonics Source Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation N ≥ L Independent from Loudspeaker configuration L 3 D = ( M + 1) 2 L 2 D = 2 M + 1 M… Ambisonics Order L… Number of Ambisonics Channels N… Number of Loudspeakers Overview of an Ambisonics Playback System 10

Microphone arrays Eigenmike (32 microphone capsules) for Higher Order Recordings mic. calibration data rotation, Loudspeaker configuration Ambisonics zoom MicEnc Microphone Mono Sound N Encoder L L Manipulation Ambisonics Source Loudspeaker si (optional) Decoder Mono Sound Encoder Core Sound Tetramic, Soundfield and Source azimuth, DIY First Order Microphone (4 cardioids) elevation N ≥ L Independent from Loudspeaker configuration 11 + 1

Encoder (Panner) Mono Signal sources can be positioned defining their position with azimuth ( φ ) and elevation ( θ ). Channel Routing in Reaper - Mono Source Input - 3rd order 3D output: L 3 D = ( M + 1) 2 = 16 L 2 D = 2 M + 1 = 7 Wigware VST Plug-Ins (Win, Mac OS) mic. calibration data rotation, Loudspeaker configuration Ambisonics zoom MicEnc up to 3rd order 3D Microphone Mono Sound N Encoder L L Manipulation Ambisonics Source Loudspeaker si (optional) Decoder Mono Sound Encoder Source azimuth, elevation N ≥ L Independent from Loudspeaker configuration 12 + 1

Ambisonics Domain the recording or streaming of Ambisonics signals is independent from the final loudspeaker placement... mic. calibration data rotation, Loudspeaker configuration Ambisonics zoom MicEnc Microphone Mono Sound N Encoder L L Manipulation Ambisonics Source Loudspeaker si (optional) Decoder Mono Sound Encoder Source azimuth, elevation N ≥ L Independent from Loudspeaker configuration 13 + 1

Loudspeaker configuration 2D Loudspeaker Ring @ BBC mic. calibration data rotation, Loudspeaker configuration Ambisonics zoom MicEnc Microphone Mono Sound N Encoder L L Manipulation Ambisonics Source Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, 29 Loudspeakers in half sphere elevation N ≥ L Independent from Loudspeaker configuration @ Mumuth Graz 14

Decoder • Computing loudspeaker signals from B-Format L N loudspeaker signals B-Format Ambdec (Linux, Mac OS) Standalone Jack Application mic. calibration data rotation, Loudspeaker configuration Ambisonics zoom MicEnc Microphone Mono Sound N Encoder L L Manipulation Ambisonics Source Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation N ≥ L VST Plug-In Decoder (Windows, Mac OS) Independent from Loudspeaker configuration 15

Problems with existing solutions • Operating system compatibility (Win, Mac OS, Linux) Plug-In Standards (VST, AU, AAX, RTAS, LV2) • Listening/producing at home? • jumps in angular representation between -180° and 180° (especially when encoding - panning) • Different standards about channel sequence and channel scaling 16

Ambisonics standards Problem: different standardization of channel order and normalization (scaling) of channels Furse-Malham Ordering ACN Order 1st order: W,X,Y,Z (Ambisonics Channel Numbering) 2nd order: R,S,T,U,V > should be new standard 3rd order: K,L,M,N,O,P ,Q de facto standard for order ≤ 3 17

Different standards adjust scaling and channel order in every stage mic. calibration data rotation, Loudspeaker configuration Ambisonics zoom MicEnc Microphone Mono Sound N Encoder L L Manipulation Ambisonics Source Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, elevation N ≥ L Independent from Loudspeaker configuration 18 + 1

Jumping angular representation • Visual representation of automation curve not intuitive 19

Listening at home Listening to recordings without a large number of loudspeakers possible using headphones and headtracking head tracking virtual individual or general (Open Sound Control) loudspeaker configuration HRTFs left HRTF N L L Soundfield Ambisonics Ambisonics headphone Convolution B-Format Rotation Decoder signals Filter right virtual loudspeaker L… Number of Ambisonics Channels signals N… Number of Virtual Loudspeakers HRTF.. Head Related Transfer Function spk8 spk1 Ambisonics Binaural Decoder F8 F1 spk2 spk7 F7 F2 mic. calibration data rotation, Loudspeaker configuration Ambisonics F3 zoom MicEnc Microphone F6 spk3 spk6 Mono Sound N Encoder L L F4 Manipulation Ambisonics Source F5 Loudspeaker signals (optional) Decoder Mono Sound Encoder Source azimuth, spk5 spk4 elevation N ≥ L loudspeakers - left ear transfer path Independent from Loudspeaker configuration 20

Binaural Decoder L L Stereo 21

Things to do... ... for me • more sophisticated panning options and visualization • order > 3 • CPU efficiency ... for the community • standardize channel sequence and scaling for order > 3 • Affordable DIY Higher Order Microphones 22

A č i ū !