Virtual Navigation of Ambisonics- Encoded Sound Fields Containing - PowerPoint PPT Presentation

Virtual Navigation of Ambisonics- Encoded Sound Fields Containing Near-Field Sources Joseph G. Tylka Final Public Oral Examination Mechanical and Aerospace Engineering Princeton University Advisor: Edgar Y. Choueiri 9 May 2019 � 1

Virtual Navigation � 2

Virtual Navigation � 2

Virtual Navigation � 2

Ambisonics Encoding & Binaural Rendering https://en.wikipedia.org/wiki/Ambisonics Convolve ∗ Decode to Encode to Binaural Capture virtual Ambisonics signals speakers HRTFs Xie ( 2013 ), Head-Related Transfer https://developers.google.com/vr/concepts/spatial-audio Function and Virtual Auditory mh acoustics Eigenmike Display , Fig. 2.5. � 3

Near-Field Sources Sound source Ambisonics mic. R Region of validity � 4

Virtual Navigation of Ambisonics- Encoded Sound Fields Containing Near-Field Sources • Several navigational methods exist - Unclear how they perform, how to compare - When should each method be used? • Region of validity is a well-known mathematical issue - Unclear how it manifests in terms of audio � 5

Outline A. Metrics for evaluating navigational methods • Auditory models and objective metrics - Spectral coloration and perceived source localization • Subjective validation through listening experiments B. Effect of microphone validity • Proposed method: valid microphone interpolation (VMI) • Simple benchmark: weighted-average interpolation (WAI) C. Comparisons of navigational methods • State-of-the-art: time-frequency analysis (TFA) • Practical considerations Summary and conclusions � 6

A. Metrics for Evaluating Navigational Methods � 7

Listening Experiments � 8

Experiment 1: Coloration � 9

Coloration Test • Collect subjective ratings of coloration & compute objective metrics • Perform multiple linear regression between ratings and metrics values • MU ltiple S timuli with H idden R eference and A nchor (ITU-R BS.1534-3) • Reference : no navigation, pink noise • Anchor : 3.5 kHz low-passed version of Ref . • Test samples : vary navigational method and distance • User rates each sample from 0–100: 100 = Ref .; 0 = Anchor • Coloration score = 100 − MUSHRA rating: 0 = Ref .; 100 = Anchor � 10

Coloration GUI � 11

Coloration Metric η ( f c ) • Averaged spectral distortions in Magnitude (dB) narrow frequency bands (Schärer and Lindau, 2009) |A ( f ) |/|A 0 ( f ) | • Critical bands approximated by ERB-spaced gammatone filter bank ✓ R | H Γ ( f ; f c ) || A ( f ) | 2 d ◆ f η ( f c ) = 10 log 10 | H Γ ( f ; f c )| Magnitude (dB) R | H Γ ( f ; f c ) || A 0 ( f ) | 2 d f ρ η = max η ( f c ) − min η ( f c ) c c • More sophisticated coloration metrics exist (Boren et al., 2015) Frequency (Hz) Schärer and Lindau ( 2009 ). “Evaluation of Equalization Methods for Binaural Signals.” Boren et al. ( 2015 ). “Coloration metrics for headphone equalization.” � 12

Regression Results Proposed: r = 0.84 Kates: r = 0.72 Avg. Measured Coloration Score 100 100 Test details: • 27 test samples 50 50 • 4 trained listeners • Pink noise signal 0 0 Legend 0 50 100 0 50 100 Data/model Predicted Coloration Score Predicted Coloration Score y = x − − — y = x ± 20 Pulkki et al.: r = 0.77 Wittek et al.: r = 0.77 Avg. Measured Coloration Score 100 100 50 50 0 0 0 50 100 0 50 100 Avg. Predicted Coloration Score Avg. Predicted Coloration Score Tylka and Choueiri ( 2017 ). “Models for evaluating navigational techniques for higher-order ambisonics.” � 13

Experiment 2: Localization � 14

Localization Test … … 10 11 12 13 14 15 5 cm 127 cm θ Recording/encoding Interpolation 10 cm � 15

Localization Test � 16

Localization Metric 1.Transform to plane-wave Plane-wave IR impulse responses (IRs) 2.Split each IR into wavelets High-pass 3.Threshold to find onset times Find peaks 4.Compute average amplitude in each critical band Window 5.Compute perceptually-weighted “energy vector” in each band Wavelets 6.Compute average vector over all bands Stitt et al. ( 2016 ). “Extended Energy Vector Prediction of Ambisonically Reproduced Image Direction at Off-Center Listening Positions.” � 17

Localization Test Results Test details: • 70 test samples Legend • 4 trained listeners Data/model • Speech signal y = x − − — y = x ± 5° Pearson correlation coefficient: r = 0.85 Mean absolute error: ε = 3.67° Tylka and Choueiri ( 2017 ). “Models for evaluating navigational techniques for higher-order ambisonics.” � 18

Summary - Part A. • Developed objective metrics for coloration and localization • Constructed experimental setup to conduct listening tests • Conducted subjective listening tests to validate the metrics • Finding : the chosen coloration metric is a dominant factor in human perception of coloration • Finding : the structure of the localization metric is valid for giving reasonable predictions of perceived localization � 19

B. Effect of Microphone Validity � 20

Ambisonics Interpolation Sound source Amb. mic. 2 Listening Amb. mic. 1 position Amb. mic. 3 � 21

Ambisonics Interpolation Sound source Amb. mic. 2 Listening Amb. mic. 1 position Amb. mic. 3 Region of validity � 21

Ambisonics Interpolation Sound source Amb. mic. 2 Amb. mic. 1 Listening position Amb. mic. 3 Region of validity � 21

Ambisonics Interpolation Sound source Amb. mic. 2 Amb. mic. 1 Listening position Amb. mic. 3 Region of validity � 21

Ambisonics Interpolation Sound source Amb. mic. 2 Listening Amb. mic. 1 position Amb. mic. 3 Region of validity � 21

Valid Microphone Interpolation Amb. signals from mic. 1 Discard signals Apply Interpolated 
 from invalid interpolation amb. signals microphones filter matrix Amb. signals from mic. P Compute Determine valid Detect and locate near- interpolation microphones field sources filter matrix Listening Microphone position positions Tylka and Choueiri ( 2019 ). “A Parametric Method for Virtual Navigation Within an Array of Ambisonics Microphones.” Under Review. � 22

Numerical Simulations • x ~ ~ ~ s 1 s 0 s 2 s 0 ' 1 ' 0 γ = ' 2 ∆ / 2 y � � ∆ Tylka and Choueiri ( 2019 ). “A Parametric Method for Virtual Navigation Within an Array of Ambisonics Microphones.” Under Review. � 23

Results - Coloration Weighted-Average Interpolation (WAI) Valid Microphone Interpolation (VMI) Finding : excluding the invalid microphone improves coloration performance for interior sources Tylka and Choueiri ( 2019 ). “A Parametric Method for Virtual Navigation Within an Array of Ambisonics Microphones.” Under Review. � 24

Results - Localization Weighted-Average Interpolation (WAI) Valid Microphone Interpolation (VMI) Finding : excluding the invalid microphone improves localization performance for interior sources Tylka and Choueiri ( 2019 ). “A Parametric Method for Virtual Navigation Within an Array of Ambisonics Microphones.” Under Review. � 25

Summary - Part B. • Developed the valid microphone interpolation (VMI) method which excludes invalid microphones from the interpolation • Compared this method to a simple benchmark: weighted-average interpolation (WAI) • Finding : excluding the invalid microphone significantly improves coloration and localization performance for interior sources � 26

C. Comparisons of Navigational Methods � 27

Time-Frequency Analysis Source Triangulation Point-Source Modeling s 0 ŝ 1 ŝ 2 Thiergart et al. ( 2013 ). “Geometry-Based Spatial Sound Acquisition Using Distributed Microphone Arrays.” � 28

Results - Coloration Time-Frequency Analysis (TFA) Valid Microphone Interpolation (VMI) Finding : VMI achieves superior coloration performance for interior sources and/or large array spacings Tylka and Choueiri ( 2019 ). “Domains of Practical Applicability for Parametric Interpolation Methods for Virtual Sound Field Navigation.” Under Review. � 29

Results - Localization Time-Frequency Analysis (TFA) Valid Microphone Interpolation (VMI) Finding : TFA achieves superior localization performance only for interior sources with large array spacings Tylka and Choueiri ( 2019 ). “Domains of Practical Applicability for Parametric Interpolation Methods for Virtual Sound Field Navigation.” Under Review. � 30

Domains of Practical Application Coloration Localization ������� ������� ������� ������� ��� ��� ��� ����� ������ ����� ������ ����� ����� ����� ����� ��� ��� ��� �������� ������� �������� ������� Tylka and Choueiri ( 2019 ). “Domains of Practical Applicability for Parametric Interpolation Methods for Virtual Sound Field Navigation.” Under Review. � 31