Transfer learning for unsupervised infmuenza-like illness models - PowerPoint PPT Presentation

Transfer learning for unsupervised infmuenza-like illness models from online search data Bin Zou Vasileios Lampos Ingemar J. Cox Department of Computer Science University College London ( lampos.net )

From online searches to infmuenza-like illness rates Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19. 1/29 12 10 ILI percentage 8 6 4 2 0 2004 2005 2006 2007 2008 Year

From online searches to infmuenza-like illness rates Google Flu Trends ( discontinued ) popularising an established idea Ginsberg et al. (2009) Eysenbach (2006); Polgreen et al. (2008) Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19. 1/29

From online searches to infmuenza-like illness rates Task abstraction • input – frequency of search queries over time: • output – corresponding infmuenza-like illness (ILI) rate: Modelling • originally proposed models were evidently not good solutions • new families of methods seem to work OK in various geographies Cook et al. (2011); Olson et al. (2013); Lazer et al. (2014) Lampos et al. (2015a); Yang et al. (2015); Lampos et al. (2017); Wagner et al. (2018) Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19. 2/29 X ∈ R n × s y ∈ R n • regression task , i.e. learn f : X → y

From online searches to infmuenza-like illness rates Task abstraction • input – frequency of search queries over time: • output – corresponding infmuenza-like illness (ILI) rate: Modelling Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19. 2/29 X ∈ R n × s y ∈ R n • regression task , i.e. learn f : X → y • originally proposed models were evidently not good solutions 1 • new families of methods seem to work OK in various geographies 2 1 Cook et al. (2011); Olson et al. (2013); Lazer et al. (2014) 2 Lampos et al. (2015a); Yang et al. (2015); Lampos et al. (2017); Wagner et al. (2018)

Why estimate ILI rates from online search statistics? Common arguments for: • complements traditional syndromic surveillance • applicable to locations that lack an established health system oxymoron ( supervised learning ) motivated this paper 3/29 ✓ timeliness ✓ broader demographic coverage, larger cohort ✓ broader geographical coverage ✓ not afgected by closure days or national holidays ✓ lower cost Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19.

Why estimate ILI rates from online search statistics? Common arguments for: • complements traditional syndromic surveillance • applicable to locations that lack an established health system motivated this paper 3/29 ✓ timeliness ✓ broader demographic coverage, larger cohort ✓ broader geographical coverage ✓ not afgected by closure days or national holidays ✓ lower cost ✓ oxymoron ( supervised learning ) Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19.

Why estimate ILI rates from online search statistics? Common arguments for: • complements traditional syndromic surveillance • applicable to locations that lack an established health system 3/29 ✓ timeliness ✓ broader demographic coverage, larger cohort ✓ broader geographical coverage ✓ not afgected by closure days or national holidays ✓ lower cost ✓ oxymoron ( supervised learning ) ✓ motivated this paper Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19.

Our contribution in a nutshell Main task • train a model for a source location where historical syndromic surveillance data is available, and • transfer it to a target location where syndromic surveillance data is not available or, in our experiments, ignored Transfer learning steps 1. Learn a linear regularised regression model for a source location 2. Map search queries from the source to the target domain (languages may difger) 3. Transfer the source weights to the target domain (might involve weight re-adjustment) 4/29 Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19.

Our contribution in a nutshell Main task • train a model for a source location where historical syndromic surveillance data is available, and • transfer it to a target location where syndromic surveillance data is not available or, in our experiments, ignored Transfer learning steps 1. Learn a linear regularised regression model for a source location 2. Map search queries from the source to the target domain (languages may difger) 3. Transfer the source weights to the target domain (might involve weight re-adjustment) 4/29 Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19.

S and Transfer learning task defjnition Target domain T , estimate Aim: Given 5/29 for a location Source domain # query j issued during ∆ t i query frequency x ij = # all queries issued during ∆ t i { } • D S = ( x i , y i ) , i ∈{ 1 , ... , n } • x i ∈ R s = { x ij } , j ∈{ 1 , ... , s } : frequency of source queries • y i ∈ R : ILI rate for time interval i • D T = { x ′ i } , i ∈{ 1 , ... , m } • x ′ i ∈ R t : frequency of target queries • note that t need not equal s Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19.

Transfer learning task defjnition for a location Source domain Target domain 5/29 # query j issued during ∆ t i query frequency x ij = # all queries issued during ∆ t i { } • D S = ( x i , y i ) , i ∈{ 1 , ... , n } • x i ∈ R s = { x ij } , j ∈{ 1 , ... , s } : frequency of source queries • y i ∈ R : ILI rate for time interval i • D T = { x ′ i } , i ∈{ 1 , ... , m } • x ′ i ∈ R t : frequency of target queries • note that t need not equal s ✞ ☎ Aim: Given D S and D T , estimate y ′ i ✝ ✆ Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19.

Step 1 – Learn a regression function in the source domain Source domain Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19. 6/29 • x i ∈ R s = { x ij } , j ∈{ 1 , ... , s } : frequency of source queries • y i ∈ R : ILI rate for time interval i Elastic net 1 ( constrained ) )) 2 n ( s s s ( ∑ ∑ ∑ ∑ w 2 argmin y i − β − x ij w j + λ 1 | w j | + λ 2 j w ,β i =1 j =1 j =1 j =1 subject to w ≥ 0 1 Zou and Hastie (2005)

Step 1 – Learn a regression function in the source domain Elastic net ( constrained ) Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19. consistency under collinearity • few training instances • more straightforward to transfer Why use elastic net? 7/29 )) 2 n ( s s s ( ∑ ∑ ∑ ∑ w 2 argmin y i − β − x ij w j + λ 1 | w j | + λ 2 j w ,β i =1 j =1 j =1 j =1 subject to w ≥ 0 • previous successful application 1 • combines ℓ 1 - and ℓ 2 -norm regularisation: sparse solution, model 1 Lampos et al. (2015a,b); Zou et al. (2016); Lampos et al. (2017)

Step 1 – Learn a regression function in the source domain Elastic net ( constrained ) • better performance at the target location • but enables a more comprehensive transfer • worse performing model for the source location Why apply a non-negative weight constraint? 7/29 )) 2 n ( s s s ( ∑ ∑ ∑ ∑ w 2 argmin y i − β − x ij w j + λ 1 | w j | + λ 2 j w ,β i =1 j =1 j =1 j =1 subject to w ≥ 0 • ( how? ) coordinate descent restricting negative updates to 0 Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19.

Step 1 – Learn a regression function in the source domain Selecting queries prior to applying elastic net • fjlter out queries with either or (corr. with ILI) S : remaining queries after applying elastic net Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19. 8/29 • hybrid feature selection similarly to previous work 1 • derive query embeddings e q using fastText 2 • defjne a fmu context/topic: T = { ‘ fmu ’, ‘ fever ’ } • compute each query’s similarity to T using g ( q , T ) = cos ( e q , e T 1 ) × cos ( e q , e T 2 ) cos( · , · ) is mapped to [0 , 1] 1 Zou et al. (2016); Lampos et al. (2017); Zou et al. (2018) 2 Bojanowski et al. (2017)

Step 1 – Learn a regression function in the source domain Selecting queries prior to applying elastic net Zou, Lampos , Cox. Transfer learning for unsupervised fmu models from online search . WWW ’19. 8/29 • hybrid feature selection similarly to previous work 1 • derive query embeddings e q using fastText 2 • defjne a fmu context/topic: T = { ‘ fmu ’, ‘ fever ’ } • compute each query’s similarity to T using g ( q , T ) = cos ( e q , e T 1 ) × cos ( e q , e T 2 ) cos( · , · ) is mapped to [0 , 1] • fjlter out queries with either g ≤ 0 . 5 or r ≤ 0 . 3 (corr. with ILI) ✞ ☎ Q S : remaining queries after applying elastic net ✝ ✆ 1 Zou et al. (2016); Lampos et al. (2017); Zou et al. (2018) 2 Bojanowski et al. (2017)