Daedalus: Summer 2017 Internship Project Final Report Ashish - PowerPoint PPT Presentation

Daedalus: Summer 2017 Internship Project Final Report Ashish Tondwalkar August 3, 2017

Who am I? Senior - TJHSST Class of 2018 Hobbyist, Chess player, Game Enthusiast Snr. Research Plan: Locate an object's position using Sound Interning @ USC-ISI under Dr. Andrew Schmidt to research Microphone Array Beamforming for Sound Source Localization - Project DAEDALUS

Daedalus Overview: Research methods to improve accuracy of localization Possible applications: • Military: • Locating long range sharp shooters • Inform squads of firefight locations Possible areas to research accuracy: • Microphone Distance • Array Shape • Microphone Count

Setup • Pynq - Z1 Board • Jupyter to run python • OLED Display to output status & data • Microphones in triangle configuration • Servo placed in the middle to provide direction • Speaker to calibrate microphone recordings OLED MIC0 MIC2 Servo PYNQ Speaker MIC1 4

Project Plan 1. Time Sound thresholds 2. Record Small intervals of sound from microphones 3. Convert data to manipulatable format 4. Filter Noise by Butterworth 5. Calibrate Recordings using speaker sound 6. Cross Correlation for Time Phase Delay 7. Direction: Plug Time Differences into Matrix Equation & retrieve unit vector components 8. Distance: Plug Time Differences into Algebraic Equation & retrieve estimated circle radius 9. Output Vector Data through display and servos 5

DirecEon Given: • Position Vectors of Microphones • Sound time delay for each Microphone Unknown: • Direction of Sound 6

DirecEon 7

Distance Curviness of arc estimated by registering time delay of microphone input 8

ImplementaEon 9

ImplementaEon Issues: • Eq Ax = b requires A to be square and Invertible • Cannot Implement more than 1+D Microphones • Solving using SciPy augmented matrix yields incorrect answer; does not like floats On average solution requires less than 1 ms to calculate 10

SimulaEng SoluEon 11

Reconfiguring Board • Change System Block Design for more microphones, removing unnecessary blocks, reassigning port inputs • Made mistakes in top level source file -> Vivado taking 30 minutes every time to tell me I made a mistake… • Messed up address assignment which forced me to redownload original source files and redo necessary changes Issues: • Mic Input is very sharp; requires load sounds - IOB issues? - Mics are bad? - Pull Down in constraints file? - Misinterpretation of Design? 12

ReformaMng & BuOerworth • PDM -> PCM ~ 30 seconds per 1 second clip • • Bu>erworth centers audio ~ 10 seconds per 1 second clip • • ~2 Minutes to reformat 3 recordings of 1 second 13

AlternaEves • Using raw data from audio buffer - 192000 Hz sampling rate! • Manipulate FFT Array to filter audio • iFFT and continue to GCC FFT RAW DATA iFFT 14

Parallel Recording • Did not use hardware to record in sync • Python Threading to record in parallel; not in sync Calibration Solution: • Record speaker in center of mic array initially, then make sound • Realign recording using GCC of speaker recording segment • GCC modified recordings Speaker initially played to detect offset of mic recordings Segment of Recording to detect target’s sound 15

Threading 16

Speaker Placement • Microphones located at 3 vertices of a square • Speaker Placed in center of square • Speaker Pointed upward for even sound propagation 17

Hardware RestricEon • Microphones not very sensitive, cannot pickup conversational speech • Microphones not omnidirectional Testing Calibration without all microphones pointing directly at speaker Calibration NOT ALIGNED! Speaker playing twice 18

Hardware RestricEon • Audio Segments of speaker played twice are better aligned when microphones are pointed toward the speaker Calibration • Microphones not omnidirecEonal & not very sensiEve through causes inaccuracies in posiEon calculaEons 19

CalibraEon GCC Time Difference Shifting Recording Arrays 20

Servo Output • PWM signal • Duty Cycle Determines Position • Smaller Period -> larger range for duty cycle -> better precision • Period of 2 ms • Angle: [0, π /2] ~ Duty: (21%, 55%) 21

OLED Output • Status of the Program • Displays which Phase program is in • Target Position Output 22

Conclusion Project Conclusion: • Proof of concept completed • Demonstration of Project available • Able to indicate direction with fair accuracy • Final Program ~ 400 Lines of Python • Total Notebook Code ~ 1800 Lines of Python Areas for further improvement: • Implementing simultaneous recording through hardware • Higher quality omnidirectional microphones • Find a method to improve distance estimation • Research accuracy improvement 23

Conclusion Some of what I have gained from the internship: • Learned Python and Jupyter • Major experience in using an FPGA • Far better understanding of Verilog and Vivado • Beginner level research experience • Interest in pursuing Research over Industry • Had Fun 24

Senior Research • Larger budget will allow for better microphones • More time will allow me to implement hardware enabled recording Build Up from Internship Project • Sound bar mimicry • Time-delay Speaker Beam forming to produce stereo sound at target location 25

Sources W. Jiang, Z. Cai, M. Luo, and Z. Yu. A Simple Microphone Array For Source Direction and Distance Estimation. J. Valin, F. Michaud, J. Rouat, and D. Letourneau. Robust Sound Source Localization Using a Microphone Array on a Mobile Robot. N. Kwok, J. Buchholz, G. Fang, and J. Gal. Sound Source Localization: Microphone Array Design and Evolutionary Estimation. M. Omologo, and P. Svaizer. Use of the Crosspower-Spectrum Phase in Acoustic Event Location. Special Thanks to: Dr. Schmidt Other RCG Members 26