tl v II ii.bn n be bottleneck Encode BTK from D Zi M - PDF document

content from W MML Notes Reduction Book on Dimensionality Deisenroth Faisal Ong Given X Xa Xz Xz TI 0 R E Xi There Key low dimensional Assumption exists a representation a Aline snapping Beran Bix ERM Zi from X z CD M This B and is Projection a orthonormal is an matrix He k IFK T orthogonal 0 be by tl v II ii.bn n be bottleneck Encode BTK from D Zi M d.ms Ki _B2i B Btxi Decode m so 1 I Dim Mil

So of Xi Zi is a Compressed representation Xi And Is a reconstruction What function here should be our objective One The Maximize Variance perspective in lower X dimensional Space Leads Cool With Connection To Eigenvalues first Consider Iz dimension cbix.it v LDx71xD Assuming 5 0 Remember b'If.tn ixixiIb un S Covariance Matrix'd bISb µ to find be Want to Maximize this This but is Trivial without additional constraint in i

have We problem a constrained Lagrangian Opt Re write as bISbztXC1 bib H Llb 2b S 2dbI bib 1 Set both 0 to b By definition is an be of S q eigenvector the X is eigenvalue he In The first general is principal Component I bazzi bz b'IX c E IRD P p D dimensions dimension 1 To The Captured by information Compress not bz this we can procedure repeat on X babies in reconstruction

This bz.bz where bz will yield maximizes Variance remaining This iterative approach not actually necessary The 5 of need Eigenvectors all we are ott Iii II p s Let's This I In Class implement see exercise PCA A probabilistic perspective on III 9Etitm.ae Be x 2113 P m 02 P x12 B E Plz ar We This can use to generate Images from the latent space notebook see

Projection perspective Ii 132 B orthonormal again Want to minimize reconstruction error f II Hx Jm Till What The for Xi Coordinates 2 are optimal B w.r.ir JIN JI dei J2ji dki daji II x sitter f CI zmibm b So JJm x Htb f

it may bi bi 0 x II am b Tb f 2 bib f f x b x b 2 bjXi Set 0 to Zji for A of similar choice B C basis argument be made can The M yielding again largest Eigenvectors see reading