A Study on Gesture Interaction with a 3D Audio Display Georgios - PowerPoint PPT Presentation

A Study on Gesture Interaction with a 3D Audio Display Georgios Marentakis Stephen Brewster http://www.audioclouds.org

Motivation for Gesture Controlled Spatial Audio Displays Research • Visual Display Limitations • Use Audio, Spatial Audio • Conventional Input Devices (Mouse, Keyboard) are hard to use when mobile • Use gestures

Literature Review • Cohen et al, Audio Windows, Interacting with spatial sound, pointing, moving and grabbing sound, filtears • Schmandt et al, Nomadic Radio, Audio Hallway, Dynamic Soundscape • Brewster et al, Auditory Pie Menus

Interface Design • Substitute/Augment Visual Display with a gesture controlled Auditory Display Background Radio task Weather forecast Announcement

Spatial Sound -> Ambiguity • Auditory Perception of Space is less accurate than the Visual Counterpart • Non-Individualized Ear Measurements • Performance improves with experience, most experiments done with trained listeners Ideally …

Browsing & Selecting Experiment Features • Real Time Orientation Update • On Target Audio Feedback/No Feedback • Three interaction techniques for browsing and selecting a sound. We look for • Effect of Interaction Technique • Effect of Sound Position • Effect of Feedback • Effective Selection Angles

Interaction Browsing Selecting Getting Feedback Orientation Update On Target Confirmation

Experiment Design Dependent Variables Independent Variables Effective Selection • Three interaction Angle (Two Down - One Up methods Procedure) • Feedback vs. No Feedback (Between Subjects) • 8 sound positions, random order Angular Deviation from Target

Procedure Twelve participants: five females, seven males Age span: 19 to 30. Participants had no experience in interacting with a spatial audio display

Effective Selection Angle - Feedback Interaction Technique 14 F 2, 22 = 10.777, p< 0.001 12 Sound Location not Mean Effective Angle 10 Significant – Participants were facing the sound 8 Tablet significantly 6 more accurate Tablet 4 No difference Hand between hand and 2 Head 0 45 90 135 180 225 270 360 head LOCATION Only 1 out of 3 participants could complete the no feedback case. It took much longer, but accuracy was comparable.

Deviation analysis 30 Interaction Method F 2,192 = 7.463, p < 0.001 20 Mean Deviation +- 1 SD Sound Position 10 F 7,672 = 7.987, p < 0.001 0 -10 225° != 0°, 45°, 90°, 135°, -20 270°, 315° Tablet0 Tablet180 Hand0 Hand180 Head0 Head180 Tablet90 Tablet270 Hand90 Hand270 Head90 Head270 180° != 45°, 270°, 315°

Ease of Use & Comfort 7.5 7.5 7.0 7.0 Mean Ease of Use Ratings Mean Comfort Ratings 6.5 6.5 6.0 6.0 5.5 5.5 5.0 5.0 4.5 4.5 Tablet Hand Head Tablet Hand Head Tablet significantly harder to use No statistical difference between hand and head

Conclusion • Effective angle is interaction method dependent • Tablet significantly better but not preferable from users, unnatural. • Hand based interaction ineffective for sound locations in the back of a user • On Target Cue very Important • Real Time Orientation Update Cue quite demanding

Future Work • Sound Design • Display Design • Multiple Sounds • Feedback • Navigation