resound
play

RESound Interactive Sound Rendering in Dynamic Virtual Environments - PowerPoint PPT Presentation

Dec 25, 2022 •205 likes •958 views

RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak Lakulish Antani, Dinesh Manocha University of North Carolina ACM MM 09 1 October 19-23, 2009 Sound rendering and applications Details of

  1. RESound Interactive Sound Rendering in Dynamic Virtual Environments Micah Taylor, Anish Chandak Lakulish Antani, Dinesh Manocha University of North Carolina ACM MM 09 1 October 19-23, 2009

  2. ● Sound rendering and applications ● Details of propagation ● Our system: RESound ACM MM 09 2 October 19-23, 2009

  3. ● Sound rendering and applications ● Details of propagation ● Our system: RESound ACM MM 09 3 October 19-23, 2009

  4. Sound Rendering ● Three main steps ● Signal input ● Sound propagation ● Audio output ACM MM 09 4 October 19-23, 2009

  5. Sound Rendering: Signal Input [Matt Hileo] [Raghuvanshi 2006] ● Recorded sample ● Synthesized sound ● Simple and fast ● Physics simulation generates sound ● Played with events ● Matches virtual events ● Static ACM MM 09 5 October 19-23, 2009

  6. Sound Rendering: Signal Input ● Synthesized sound ● Uses physical models [Florens et al. 1991] ● Interactive rates with many objects [Raghuvanshi et al. 2006] ● Correlates closely with visual scene [Ren et al. 2009] ACM MM 09 6 October 19-23, 2009

  7. Sound Rendering: Propagation ● Goal: Model environment influences ● Echoes ● Delay from distance ● Attenuation from distance ● Frequency shifts ● Output: Impulse response ● Represents room's effect on input signal ACM MM 09 7 October 19-23, 2009

  8. Sound Rendering: Propagation ● Common methods ● No propagation - direct path only ● Geometric simulation ● Numerical simulation ACM MM 09 8 October 19-23, 2009

  9. Sound Rendering: Audio Output ● Goal ● Combine many sounds from environment ● Apply any needed effects ● Output to user's audio device ● Uses the output from prior steps ● Input signal ● Room impulse response ACM MM 09 9 October 19-23, 2009

  10. Sound Rendering: Audio Output ● Common output methods ● Mono – Fast, simple – No spatialization ● Stereo – Fast, simple, left+right spatialization ● 3d sound – Head Related Transfer Functions (HRTF) – Complex, very good spatialization ACM MM 09 10 October 19-23, 2009

  11. Applications ● Video games ● Helps player avoid monsters ● Provides sound cues to environment size ● Used in most 3d video games ACM MM 09 11 October 19-23, 2009

  12. Applications ● Training simulators ● Improves realism ● Decreases incorrect training ● Current uses ● Tactical training ● EMT training ACM MM 09 12 October 19-23, 2009

  13. Applications ● Multimedia ● Auditory displays – Enhance data visualization ● Telephony and Video conferencing ACM MM 09 13 October 19-23, 2009

  14. Applications ● Computer aided design ● Relay cues about environment design ● Preview room acoustics before construction ACM MM 09 14 October 19-23, 2009

  15. ● Sound rendering and applications ● Details of propagation ● Our system: RESound ACM MM 09 15 October 19-23, 2009

  16. Propagation ● Simplest method: ● Direct path between source and listener ● Add echoes with post- process filter ● Fast ● Widely used ACM MM 09 16 October 19-23, 2009

  17. Propagation ● However ● Not physically based ● Spatialization incorrect ● Echoes do not match environment ACM MM 09 17 October 19-23, 2009

  18. Propagation ● Acoustic simulations ● Numerical – Solves acoustic wave equation – Slow, but getting faster [Raghuvanshi et al. 2009] ● Geometric – High frequency approximation – Very fast – interactive – Models sound as ray ACM MM 09 18 October 19-23, 2009

  19. Propagation ● Specular reflection ● Mirror-like reflections ● Reflections decrease amplitude ● Longer paths, longer delays ACM MM 09 19 October 19-23, 2009

  20. Propagation ● Specular reflection ● Mirror-like reflections ● Reflections decrease amplitude ● Longer paths, longer delays ● Often many reflection paths ACM MM 09 20 October 19-23, 2009

  21. Propagation ● Diffuse reflection ● Scattering reflections ACM MM 09 21 October 19-23, 2009

  22. Propagation ● Diffuse reflection ● Scattering reflections ● Scattered sound reaches listener ACM MM 09 22 October 19-23, 2009

  23. Propagation ● Diffraction ● Sound 'bends' around corners ● Can change phase ACM MM 09 23 October 19-23, 2009

  24. Propagation ● Diffraction ● Sound 'bends' around corners ● Can change phase ● Often many diffraction paths ACM MM 09 24 October 19-23, 2009

  25. Propagation ● Combine ● Direct ● Specular ● Diffuse ● Diffraction ● Early contributions ● 4-5 recursions ACM MM 09 25 October 19-23, 2009

  26. Propagation ● Reverberation ● Late contributions ● Impulses decays over time ● Hundreds of recursions ● Gives 'feel' of the room ACM MM 09 26 October 19-23, 2009

  27. Propagation ● Specular reflections ● Image-source method [Allen et al. 1979] ● From source ● Reflect against all scene triangles – Creates image-sources – Is listener visible ● Reflect image sources – and so on... ACM MM 09 27 October 19-23, 2009

  28. Propagation ● However ● Very compute intensive ● Need to accelerate ● Graphic acceleration ● Remove non-visible triangles ● Sound acceleration ● Remove non-reflecting triangles ACM MM 09 28 October 19-23, 2009

  29. Propagation ● Accelerated by ● Ray tracing [Vorlander 1989] ● Beam tracing [Funkhouser et al. 1998] ● Frustum tracing [Lauterbach et al. 2007] ● And others... ● Often require precomputation ● Non-moving source ACM MM 09 29 October 19-23, 2009

  30. Propagation ● Diffuse reflections ● Often modeled by ray tracing [Dalenbaeck 1996] ● Radiosity [Siltanen et al. 2004] ● Compute intensive ● Fixed source and receiver ● No scene movement ACM MM 09 30 October 19-23, 2009

  31. Propagation ● Diffraction ● Added to – Beam tracing [Tsingos et al. 2001] – Ray tracing [Stephenson et al. 2007] – Frustum tracing [Taylor et al. 2009] – Image source [Shroeder et al. 2009] ACM MM 09 31 October 19-23, 2009

  32. Propagation ● Reverberation ● Ray tracing – Slow, accurate [Hodgson 1990] ● Statistical – Fast, some error [Savioja et al. 1999] ACM MM 09 32 October 19-23, 2009

  33. ● Sound rendering and applications ● Details of propagation ● Our system: RESound ACM MM 09 33 October 19-23, 2009

  34. RESound ● Simulates all mentioned effects ● Interactive update rates ● Dynamic scenes ● Handles propagation and output ● Given input sound + environment ● Propagates sound through environment ● Renders signal at receiver's position ACM MM 09 34 October 19-23, 2009

  35. RESound System overview ACM MM 09 35 October 19-23, 2009

  36. RESound ● Early contributions by simulation ● Specular + diffraction ● Diffuse reflection ● Late contributions by statistics ● 3d audio output ACM MM 09 36 October 19-23, 2009

  37. RESound ● Unified engine ● Frustum tracing ● Ray tracing ● Ray primitive ● Single acceleration structure ● Bounding Volume Hierarchy ● Allows dynamic scenes ● Fast ray tracing ACM MM 09 37 October 19-23, 2009

  38. RESound ● Scene acceleration hierarchy ● Bounding Volume Hierarchy [Lauterbach et al. 2006] – Fast construction times – Allows interactive visual ray tracing – Allows dynamic scene changes ● Can accelerate frustum and ray tracing ACM MM 09 38 October 19-23, 2009

  39. ● Specular + diffraction ● Diffuse ● Frustum tracing ● Ray tracing ● Volumetric, finds most ● Shares scene paths structure ● Dynamic scenes ● Dynamic scenes ● Fast ● Fast ACM MM 09 39 October 19-23, 2009

  40. RESound ● Frustum tracing ● Specular reflection ACM MM 09 40 October 19-23, 2009

  41. RESound ● Frustum tracing ● Specular reflection ● Frustum is bounded by rays ACM MM 09 41 October 19-23, 2009

  42. RESound ● Frustum tracing ● Specular reflection ● Check if receiver is inside bounded volume ACM MM 09 42 October 19-23, 2009

  43. RESound ● Frustum tracing ● Specular reflection ● Bounding rays can be reflected ACM MM 09 43 October 19-23, 2009

  44. RESound ● Frustum tracing ● Specular reflection ● Sound path is linear combination of rays ACM MM 09 44 October 19-23, 2009

  45. RESound ● Diffraction ● Covers more area ● Allows smooth transitions – Fades out ACM MM 09 45 October 19-23, 2009

  46. RESound ● Diffraction ● Covers more area ● Allows smooth transitions – Fades out ● First step ● Find diffracting edges ACM MM 09 46 October 19-23, 2009

  47. RESound ● Frustum tracing ● Edge diffraction ACM MM 09 47 October 19-23, 2009

  48. RESound ● Frustum tracing ● Edge diffraction ● From source – Trace many frusta ACM MM 09 48 October 19-23, 2009

  49. RESound ● Frustum tracing ● Edge diffraction ● Receiver is hidden from source ACM MM 09 49 October 19-23, 2009

  50. RESound ● Frustum tracing ● Edge diffraction ● But diffracting edge is visible ACM MM 09 50 October 19-23, 2009

  51. RESound ● Frustum tracing ● Edge diffraction ● Create diffraction frustum ACM MM 09 51 October 19-23, 2009

  52. RESound ● Frustum tracing ● Edge diffraction ● Diffracting sound reaches the receiver ACM MM 09 52 October 19-23, 2009

Recommend

More recommend