A Computational Mo del for Represen tation of Image V elo - PowerPoint PPT Presentation

A Computational Mo del for Represen tation of Image V elo citi es Eero Simoncelli Computer and Information Science Departmen t Univ ersit y of P ennsylv ania Da vid Heeger Psyc hology Departmen t Stanford Univ ersit y AR V O� ��� �

The Mo del Stage 1 Stage 2 Input: normalized STE Input: image intensities . . . • • • • • • • + + . . . . . . -2 -2 σ e σ v • • • • . . . . . . + + . . . . . . ÷ ÷ ÷ ÷ Output: normalized STE Output: normalized VE First stage computes spatio�temp oral energy �STE�� � Second stage computes �v elo cit y energy� �VE�� � Simple computation� linear com bination� squaring� normalization� � AR V O� ��� �

Spatio�temp oral Energy �STE� Stage t ω t ω y y ω x x Eac h unit is based on a linear com bination of image in tensities� � Unit resp onse is lo caliz ed in spatio�temp oral frequency � � F requency domain is �co v ered� b y a minimal n um b er of units� � d � Interme diate r esp onses may b e exactly interp olate AR V O� ��� �

STE Unit vs� Complex Cell Stim ulus Cell Resp onse Mo del Resp onse Grating Plaid P olar plots of resp onse vs� stim ulus direction of mo v emen t� � Single cell recordings replotted from Mo vshon et� al� ������ � AR V O� ��� �

STE Unit Is Not V elo cit y�T uned ω t ω y ω x Unit resp onds equally w ell to a whole family of v elo cit i es �ap erture � problem�� Equiv alen tly � in the F ourier domain� a family of planes cut through � the unit� AR V O� ��� �

V elo cit y Energy �VE� Stage ω t ω y v y v x ω x Eac h unit is based on a linear com bination of STE resp onses �as � in Albrigh t ���� Smith et al ����� STE resp onses are from previous stage� interp olate d � Unit resp onse is lo caliz ed in v elo cit y space� � V elo cit y domain is �co v ered� b y a minimal n um b er of units� � d � Interme diate r esp onses may b e exactly interp olate AR V O� ��� �

VE Unit vs� MT Cell Stim ulus Cell Resp onse Mo del Resp onse Grating Plaid P olar plots of resp onse vs� stim ulus direction of mo v emen t� � MT cell recordings replotted from Mo vshon et� al� ������ � AR V O� ��� �

VE Unit vs� MT Cell Cell Resp onse Mo del Resp onse 1.0 1.0 0.9 0.8 0.8 0.7 0.6 0.6 0.5 0.4 0.4 0.3 0.2 0.2 0.1 0.0 0.0 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 log sp eed log sp eed Resp onse to an orien ted bar as a function of log sp eed� � MT cell recordings replotted from Maunsell and V an Essen ������� � AR V O� ��� �

VE P opulation Resp onse� Single Motion v y v x v Stim ulus is a translating random dot pattern� � F ull v elo cit y�space is in terp olated from resp onses of a small n um� � b er of units� Resp onse is unimo dal� � AR V O� ��� �

VE P opulation Resp onse� T ransparen t Dots v y v x Stim ulus is t w o random dot patterns translating in di�eren t di� � rections� VE Resp onse is bimo dal� indicating presence of t w o motions� � AR V O� ��� �

VE P opulation Resp onse� T ransparen t Noise P atterns v y v x Stim ulus is t w o additiv ely com bined fractal noise patterns� mo ving � in di�eren t directions� VE Resp onse is bimo dal� indicating presence of t w o motions� � AR V O� ��� ��

VE P opulation Resp onse� Sine Grating Plaids v y v x v y v x Steep er plaids lo ok more transparen t� � Consisten t with Adelson � Mo vshon ������� � AR V O� ��� ��

VE P opulation Resp onse� Square Grating Plaids v y v x v y v x T ransparency p ercept is in�uenced b y luminance of in tersections� � Consisten t with Stoner et al ������� � AR V O� ��� ��

Conclusions Simple t w o�stage distributed computation� F or eac h stage� � Linear op erators� squared and normalized� � Resp onse space is minimally sampled� � Resp onses smo othly co v er the space� � In termediate resp onses ma y b e exactly in terp olated� � Mo del is consisten t with ph ysiology of Complex � MT cells� � Mo del is capable of represen ting m ultiple motions� � Mo del is consisten t with plaid transparency p erception� � AR V O� ��� ��