Eulerian video magnification: engineering applications of a V1-like - PowerPoint PPT Presentation

Eulerian video magnification: engineering applications of a V1-like image representation Bill Freeman Professor and Associate Dept. Head, Electrical Engineering and Computer Science Massachusetts Institute of Technology Joint work with Michael Rubinstein, Neal Wadhwa, Fredo Durand. Friday, January 3, 14

Steerable pyramid image representation input output even and odd phase basis functions (odd-phase filters shown are from a di fg erent figure) steerable pyramid image representation (only the even phase outputs are shown here) Friday, January 3, 14

Phase-based Pipeline (SIGGRAPH’13) Amplitude Phase Temporal filtering on phases Complex steerable pyramid [Simoncelli and Freeman 1995] Friday, January 3, 14

4 Friday, January 3, 14

Motion magnification original motion magnified Friday, January 3, 14

Vibration Modes of PVC pipe Source (20000 FPS) 480Hz (x200) 1200Hz (x400) 2400Hz (x1200) Sequences courtesy of Justin Chen, Civil Engineering, MIT “Piping Vibration Analysis” [Wachel et al. 1990] Friday, January 3, 14

original motion magnified (x40, 6 - 11 Hz) Friday, January 3, 14

still-frames from original and motion magnified (40x, 6 - 11 Hz) Friday, January 3, 14

For a translating image I ( x , t ) = f ( x + δ ( t )) A size microscope, with magnification M, gives this: f ( x + δ ( t )) → f ( Mx + M δ ( t )) A motion microscope gives this: f ( x + δ ( t )) → f ( x + M δ ( t )) And we often “focus” the amplification on a particular subband of the temporal signal: → f ( x + δ 1 ( t ) + M δ 2 ( t )) Friday, January 3, 14

original size magnified by 40 motion magnified by 40 Friday, January 3, 14

Engine photography Friday, January 3, 14

400 fps video of car engine Friday, January 3, 14

400 fps video of car engine, 30x motion magnified Friday, January 3, 14

Original vs motion magnified Friday, January 3, 14

Visualization of Vocalization Motions Power 0 500 1000 Frequency (Hz) Source (2 KHz) Friday, January 3, 14

high-speed video, high-speed video motion magnified x100 Power 0 500 1000 Frequency (Hz) Fundamental frequency: ~100Hz Source (2 KHz) 100 Hz Amplified x100 Source (2 KHz) Friday, January 3, 14

high-speed video, motion magnified x100 high-speed video Power 0 500 1000 Frequency (Hz) First harmonic Source (2 KHz) 220 Hz Amplified x100 Source (2 KHz) 225 Hz Amplified x100 Friday, January 3, 14

Frequency sweep showing motion components of a person at rest. 0.1-0.5 Hz amplified original 2.5-4.5 Hz amplified amplified 1-2 Hz Friday, January 3, 14

Eulerian Video Magnification in the wild Friday, January 3, 14

VideoScope by Quanta Research Cambridge Friday, January 3, 14

EVM in the Wild: Guinea Pig! Source Motion-magnified “SuperCreaturefan”: “Guinea pig Ti fg any is the first rodent on Earth to undergo Eulerian Video Magnification.” http://www.youtube.com/watch?v=uXOSJvNwtIk Friday, January 3, 14

EVM in the Wild: Pregnancy Original Processed “Tomez85” https://www.youtube.com/watch?v=J1wvFmWv7zY Friday, January 3, 14

by Erin Knutson (Graphic Design student at Yale) Friday, January 3, 14

Ways to use the method • Coding in Matlab –Matlab code available for last-year’s method • Upload your own videos –Upload site for phase-based method. • Potential collaborations. • To reach project page, google “Eulerian video magnification” 24 Friday, January 3, 14

For research project pages, with pointers to code or video upload site: http://people.csail.mit.edu/mrub/vidmag/ http://people.csail.mit.edu/nwadhwa/phase-video/ Joint work with Michael Rubinstein, Hao-Yu Wu, Eugene Hsu, Neal Wadhwa, . Friday, January 3, 14