Hypothesis Testing with Kernels Zolt´ an Szab´ o (Gatsby Unit, UCL) PRNI, Trento June 22, 2016 Zolt´ an Szab´ o Hypothesis Testing with Kernels
Motivation: detecting differences in AM signals Amplitude modulation: simple technique to transmit voice over radio. in the example: 2 songs. Fragments from song 1 „ P x , song 2 „ P y . Zolt´ an Szab´ o Hypothesis Testing with Kernels
Motivation: detecting differences in AM signals Amplitude modulation: simple technique to transmit voice over radio. in the example: 2 songs. Fragments from song 1 „ P x , song 2 „ P y . Question: P x “ P y ? Zolt´ an Szab´ o Hypothesis Testing with Kernels
Motivation: discrete domain - 2-sample testing How do we compare distributions? Given: 2 sets of text fragments (fisheries, agriculture). x 1 : Now disturbing reports out of y 1 : Honourable senators, I have a question Newfoundland show that the fragile snow for the Leader of the Government in the crab industry is in serious decline. First the Senate with regard to the support funding west coast salmon, the east coast salmon to farmers that has been announced. Most and the cod, and now the snow crabs off farmers have not received any money yet. Newfoundland. y 2 : On the grain transportation system we x 2 : To my pleasant surprise he responded have had the Estey report and the Kroeger that he had personally visited those report. We could go on and on. Recently wharves and that he had already programs have been announced over and announced money to fix them. What over by the government such as money for wharves did the minister visit in my riding the disaster in agriculture on the prairies and how much additional funding is he and across Canada. going to provide for Delaps Cove, . . . Hampton, Port Lorne, . . . . . . Zolt´ an Szab´ o Hypothesis Testing with Kernels
Motivation: discrete domain - 2-sample testing How do we compare distributions? Given: 2 sets of text fragments (fisheries, agriculture). x 1 : Now disturbing reports out of y 1 : Honourable senators, I have a question Newfoundland show that the fragile snow for the Leader of the Government in the crab industry is in serious decline. First the Senate with regard to the support funding west coast salmon, the east coast salmon to farmers that has been announced. Most and the cod, and now the snow crabs off farmers have not received any money yet. Newfoundland. y 2 : On the grain transportation system we x 2 : To my pleasant surprise he responded have had the Estey report and the Kroeger that he had personally visited those report. We could go on and on. Recently wharves and that he had already programs have been announced over and announced money to fix them. What over by the government such as money for wharves did the minister visit in my riding the disaster in agriculture on the prairies and how much additional funding is he and across Canada. going to provide for Delaps Cove, . . . Hampton, Port Lorne, . . . . . . Do t x i u and t y j u come from the same distribution, i.e. P x “ P y ? Zolt´ an Szab´ o Hypothesis Testing with Kernels
Motivation: discrete domain - independence testing How do we detect dependency? (paired samples) x 1 : Honourable senators, I have a question y 1 : Honorables s´ enateurs, ma question for the Leader of the Government in the s’adresse au leader du gouvernement au Senate with regard to the support funding S´ enat et concerne l’aide financi´ ere qu’on a to farmers that has been announced. Most annonc´ ee pour les agriculteurs. La plupart farmers have not received any money yet. des agriculteurs n’ont encore rien reu de cet argent. x 2 : No doubt there is great pressure on provincial and municipal governments in y 2 : Il est ´ evident que les ordres de relation to the issue of child care, but the gouvernements provinciaux et municipaux reality is that there have been no cuts to subissent de fortes pressions en ce qui child care funding from the federal concerne les services de garde, mais le government to the provinces. In fact, we gouvernement n’a pas r´ eduit le have increased federal investments for early financement qu’il verse aux provinces pour childhood development. les services de garde. Au contraire, nous avons augment´ e le financement f´ ed´ eral . . . pour le d´ eveloppement des jeunes enfants. . . . Zolt´ an Szab´ o Hypothesis Testing with Kernels
Motivation: discrete domain - independence testing How do we detect dependency? (paired samples) x 1 : Honourable senators, I have a question y 1 : Honorables s´ enateurs, ma question for the Leader of the Government in the s’adresse au leader du gouvernement au Senate with regard to the support funding S´ enat et concerne l’aide financi´ ere qu’on a to farmers that has been announced. Most annonc´ ee pour les agriculteurs. La plupart farmers have not received any money yet. des agriculteurs n’ont encore rien reu de cet argent. x 2 : No doubt there is great pressure on provincial and municipal governments in y 2 : Il est ´ evident que les ordres de relation to the issue of child care, but the gouvernements provinciaux et municipaux reality is that there have been no cuts to subissent de fortes pressions en ce qui child care funding from the federal concerne les services de garde, mais le government to the provinces. In fact, we gouvernement n’a pas r´ eduit le have increased federal investments for early financement qu’il verse aux provinces pour childhood development. les services de garde. Au contraire, nous avons augment´ e le financement f´ ed´ eral . . . pour le d´ eveloppement des jeunes enfants. . . . Are the French paragraphs translations of the English ones, or have nothing to do with it, i.e. P XY “ P X P Y ? Zolt´ an Szab´ o Hypothesis Testing with Kernels
Outline RKHS based metric on probability distributions. 1 2-sample testing: 2 Nonparametric. Distance between distribution representations. Zolt´ an Szab´ o Hypothesis Testing with Kernels
Outline RKHS based metric on probability distributions. 1 2-sample testing: 2 Nonparametric. Distance between distribution representations. Independence testing: 3 Dependency detection. Distance between joint ( P XY ) and product of marginals ( P X P Y ). Zolt´ an Szab´ o Hypothesis Testing with Kernels
Kernels Zolt´ an Szab´ o Hypothesis Testing with Kernels
Kernels on numerous data types Kernels exist on essentially any data type: images, texts, graphs, time series, dynamical systems, . . . ñ distribution representation, hypothesis testing: on all these domains. Zolt´ an Szab´ o Hypothesis Testing with Kernels
Towards representations of distributions: E X Given: 2 Gaussians with different means. Solution: t -test. Zolt´ an Szab´ o Hypothesis Testing with Kernels
Towards representations of distributions: E X 2 Setup: 2 Gaussians; same means, different variances. Idea: look at the 2nd-order features of RVs. Zolt´ an Szab´ o Hypothesis Testing with Kernels
Towards representations of distributions: E X 2 Setup: 2 Gaussians; same means, different variances. Idea: look at the 2nd-order features of RVs. ϕ x “ x 2 ñ difference in E X 2 . Zolt´ an Szab´ o Hypothesis Testing with Kernels
Towards representations of distributions: further moments Setup: a Gaussian and a Laplacian distribution. Challenge: their means and variances are the same. Idea: look at higher-order features. Let us consider feature representations! Zolt´ an Szab´ o Hypothesis Testing with Kernels
Kernel: similarity between features Given: x and x 1 P X objects (images, texts, . . . ). Zolt´ an Szab´ o Hypothesis Testing with Kernels
Kernel: similarity between features Given: x and x 1 P X objects (images, texts, . . . ). Question: how similar they are? Zolt´ an Szab´ o Hypothesis Testing with Kernels
Kernel: similarity between features Given: x and x 1 P X objects (images, texts, . . . ). Question: how similar they are? Define features of the objects: ϕ x : features of x , ϕ x 1 : features of x 1 . Kernel: inner product of these features k p x , x 1 q : “ � ϕ x , ϕ x 1 � H . Zolt´ an Szab´ o Hypothesis Testing with Kernels
Kernel examples X “ R d : k G p x , y q “ e ´ γ } x ´ y } 2 k p p x , y q “ p � x , y � ` γ q p , 2 , 1 k e p x , y q “ e ´ γ } x ´ y } 2 , k C p x , y q “ 1 ` . γ } x ´ y } 2 2 Zolt´ an Szab´ o Hypothesis Testing with Kernels
Kernel examples X “ R d : k G p x , y q “ e ´ γ } x ´ y } 2 k p p x , y q “ p � x , y � ` γ q p , 2 , 1 k e p x , y q “ e ´ γ } x ´ y } 2 , k C p x , y q “ 1 ` . γ } x ´ y } 2 2 X = texts, strings: bag-of-word kernel, r -spectrum kernel: # of common ď r -substrings. Zolt´ an Szab´ o Hypothesis Testing with Kernels
Kernel examples X “ R d : k G p x , y q “ e ´ γ } x ´ y } 2 k p p x , y q “ p � x , y � ` γ q p , 2 , 1 k e p x , y q “ e ´ γ } x ´ y } 2 , k C p x , y q “ 1 ` . γ } x ´ y } 2 2 X = texts, strings: bag-of-word kernel, r -spectrum kernel: # of common ď r -substrings. X = time-series: dynamic time-warping. Zolt´ an Szab´ o Hypothesis Testing with Kernels
Two-sample testing Zolt´ an Szab´ o Hypothesis Testing with Kernels
Ingredient: maximum mean discrepancy Zolt´ an Szab´ o Hypothesis Testing with Kernels
Recommend
More recommend
