Regret Bounds for Lifelong Learning Pierre Alquier Groupe de - PowerPoint PPT Presentation

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Probabilistic setting for LTL Possible probabilistic setting : P 1 , . . . , P M i.i.d from P , ( X t , 1 , Y t , 1 ) , . . . , ( X t , n M , Y t , n M ) i.i.d from P t , R t ( f ) = E ( X , Y ) ∼ P t [ ℓ ( Y , f ( X ))] , quantitative criterion to minimize w.r.t I � � R LTL ( I ) = E P ∼P min f ∈C E ( X , Y ) ∼ P [ ℓ ( Y , f ( I , X ))] . Note the strong Bayesian flavor... Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example of LTL : dictionary learning Example taken from : Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example of LTL : dictionary learning Example : dictionary learning. The X t , i ∈ R K , but all the relevant information is in DX t , i ∈ R k , k ≪ K . The matrix D is unknown. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example of LTL : dictionary learning Example : dictionary learning. The X t , i ∈ R K , but all the relevant information is in DX t , i ∈ R k , k ≪ K . The matrix D is unknown. β 1 , . . . , β M i.i.d from P , Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example of LTL : dictionary learning Example : dictionary learning. The X t , i ∈ R K , but all the relevant information is in DX t , i ∈ R k , k ≪ K . The matrix D is unknown. β 1 , . . . , β M i.i.d from P , ( X t , 1 , Y t , 1 ) , . . . , ( X t , n , Y t , n ) i.i.d from P β t : Y = β T t DX + ε, Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example of LTL : dictionary learning Example : dictionary learning. The X t , i ∈ R K , but all the relevant information is in DX t , i ∈ R k , k ≪ K . The matrix D is unknown. β 1 , . . . , β M i.i.d from P , ( X t , 1 , Y t , 1 ) , . . . , ( X t , n , Y t , n ) i.i.d from P β t : Y = β T t DX + ε, R t ( β, ∆) = E ( X , Y ) ∼ P β t [ ℓ ( Y , β T ∆ X )] , Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example of LTL : dictionary learning Example : dictionary learning. The X t , i ∈ R K , but all the relevant information is in DX t , i ∈ R k , k ≪ K . The matrix D is unknown. β 1 , . . . , β M i.i.d from P , ( X t , 1 , Y t , 1 ) , . . . , ( X t , n , Y t , n ) i.i.d from P β t : Y = β T t DX + ε, R t ( β, ∆) = E ( X , Y ) ∼ P β t [ ℓ ( Y , β T ∆ X )] , quantitative criterion to minimize w.r.t M � � �� ℓ ( Y , β T ∆ X ) R LTL (∆) = E β ∼P . E ( X , Y ) ∼ P β Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example of LTL : dictionary learning Maurer, Pontil and Romera-Paredes propose : M n � � ˆ ℓ ( Y t , i , β T D = arg min arg min t ∆ X t , i ) ∆ � β t � 1 ≤ α t = 1 i = 1 Theorem (Maurer et al ) Under suitable assumptions, with probability at least 1 − δ ,  �  � 1 � � � log 1 1 R LTL ( ˆ  . δ D ) ≤ inf ∆ R LTL (∆)+ C  α k M + + α M n Note that C can depend on ( k , K ) or not, depending on assumptions on the distribution of X under P β ... Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Going online : lifelong learning Lifelong learning (LL) Online version of learning-to-learn ? Recent work with The Tien Mai and Massimiliano Pontil. Objectives : Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Going online : lifelong learning Lifelong learning (LL) Online version of learning-to-learn ? Recent work with The Tien Mai and Massimiliano Pontil. Objectives : consider that tasks can be revealed sequentially. Use the tools of online learning theory : avoid probabilistic assumptions. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Going online : lifelong learning Lifelong learning (LL) Online version of learning-to-learn ? Recent work with The Tien Mai and Massimiliano Pontil. Objectives : consider that tasks can be revealed sequentially. Use the tools of online learning theory : avoid probabilistic assumptions. if possible, define a general strategy that does not depend on the learning algorithm used within each task. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Transfer learning, multitask learning, lifelong learning... 1 A strategy for lifelong learning, with regret analysis 2 Open questions 3 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Massimiliano Pontil The Tien Mai (UCL, IIT) (U. of Oslo) Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , loss function ℓ . Lifelong-learning problem (LL) Propose initial g . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , loss function ℓ . Lifelong-learning problem (LL) Propose initial g . For t = 1 , 2 , . . . , Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , loss function ℓ . Lifelong-learning problem (LL) Propose initial g . For t = 1 , 2 , . . . , 1 propose initial h t . For i = 1 , . . . , n t Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , loss function ℓ . Lifelong-learning problem (LL) Propose initial g . For t = 1 , 2 , . . . , 1 propose initial h t . For i = 1 , . . . , n t x t , i revealed, 1 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , loss function ℓ . Lifelong-learning problem (LL) Propose initial g . For t = 1 , 2 , . . . , 1 propose initial h t . For i = 1 , . . . , n t x t , i revealed, 1 predict ˆ y t , i = h t ◦ g ( x t , i ) , 2 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , loss function ℓ . Lifelong-learning problem (LL) Propose initial g . For t = 1 , 2 , . . . , 1 propose initial h t . For i = 1 , . . . , n t x t , i revealed, 1 predict ˆ y t , i = h t ◦ g ( x t , i ) , 2 y t , i revealed, suffer loss ˆ ℓ t , i := ℓ ( y t , i , ˆ y t , i ) , 3 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , loss function ℓ . Lifelong-learning problem (LL) Propose initial g . For t = 1 , 2 , . . . , 1 propose initial h t . For i = 1 , . . . , n t x t , i revealed, 1 predict ˆ y t , i = h t ◦ g ( x t , i ) , 2 y t , i revealed, suffer loss ˆ ℓ t , i := ℓ ( y t , i , ˆ y t , i ) , 3 update h t . 4 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Setting objects in X , labels in Y , set of functions G : X → Z and H : Z → Y , loss function ℓ . Lifelong-learning problem (LL) Propose initial g . For t = 1 , 2 , . . . , 1 propose initial h t . For i = 1 , . . . , n t x t , i revealed, 1 predict ˆ y t , i = h t ◦ g ( x t , i ) , 2 y t , i revealed, suffer loss ˆ ℓ t , i := ℓ ( y t , i , ˆ y t , i ) , 3 update h t . 4 2 udpate g . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Within-task algorithm For t = 1 , 2 , . . . , 1 Solve a usual online task, input z t , i = g ( x t , i ) , output y t , i . 2 udpate g . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Within-task algorithm For t = 1 , 2 , . . . , 1 Solve a usual online task, input z t , i = g ( x t , i ) , output y t , i . 2 udpate g . We can do it using any online algorithm. Will be refered to as “within-task algorithm”. For many algorithms, bounds are known on the (normalized)-regret : n t n t R t ( g ) = 1 − 1 � � ℓ ( y t , i , ˆ y t , i ) inf ℓ ( y t , i , h ( z t , i )) . n t n t h ∈H i = 1 i = 1 � �� � � nt = 1 i = 1 ˆ ℓ t , i =ˆ L t ( g ) nt Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Examples of within-task algorithms Online gradient for convex ℓ Initialize h = 0. Update h ← h − η ∇ f = h ℓ ( y t , i , f ( z t , i )) . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Examples of within-task algorithms Online gradient for convex ℓ Initialize h = 0. Update h ← h − η ∇ f = h ℓ ( y t , i , f ( z t , i )) . Many variants and improvements (projected gradient, online Newton-step, ...). R t ( g ) in 1 / √ n t or 1 / n t depending on assumptions on ℓ . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Examples of within-task algorithms Online gradient for convex ℓ Initialize h = 0. Update h ← h − η ∇ f = h ℓ ( y t , i , f ( z t , i )) . Many variants and improvements (projected gradient, online Newton-step, ...). R t ( g ) in 1 / √ n t or 1 / n t depending on assumptions on ℓ . EWA (Exponentially Weighted Aggregation) Prior ρ 1 = π , initialize h ∼ ρ 1 . Update ρ i + 1 ( d f ) ∝ exp[ − ηℓ ( y t , i , f ( z t , i ))] ρ i ( d f ) , h ∼ ρ i + 1 . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Examples of within-task algorithms Online gradient for convex ℓ Initialize h = 0. Update h ← h − η ∇ f = h ℓ ( y t , i , f ( z t , i )) . Many variants and improvements (projected gradient, online Newton-step, ...). R t ( g ) in 1 / √ n t or 1 / n t depending on assumptions on ℓ . EWA (Exponentially Weighted Aggregation) Prior ρ 1 = π , initialize h ∼ ρ 1 . Update ρ i + 1 ( d f ) ∝ exp[ − ηℓ ( y t , i , f ( z t , i ))] ρ i ( d f ) , h ∼ ρ i + 1 . E [ R t ( g )] in 1 / √ n t under boundedness assumption. Integrated variant : R t ( g ) in 1 / n t if ℓ is exp-concave. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions EWA for lifelong learning EWA-LL Prior π = ρ 1 on G . Draw g ∼ π . For t = 1 , 2 , . . . 1 run the within-task algorithm on task t . Suffer ˆ L t ( g ) . 2 update ρ t + 1 ( d f ) ∝ exp[ − η ˆ L t ( f )] ρ t ( d f ) . 3 draw g ∼ ρ t + 1 . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions EWA for lifelong learning EWA-LL Prior π = ρ 1 on G . Draw g ∼ π . For t = 1 , 2 , . . . 1 run the within-task algorithm on task t . Suffer ˆ L t ( g ) . 2 update ρ t + 1 ( d f ) ∝ exp[ − η ˆ L t ( f )] ρ t ( d f ) . 3 draw g ∼ ρ t + 1 . Next : we provide two examples that are corollaries of a general result (stated later). Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 1 : dictionary learning X = R K → Z = R k → Y = R � h , Dx � = h T Dx . �→ �→ x Dx Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 1 : dictionary learning X = R K → Z = R k → Y = R � h , Dx � = h T Dx . �→ �→ x Dx within-task algorithm : online gradient descent on h . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 1 : dictionary learning X = R K → Z = R k → Y = R � h , Dx � = h T Dx . �→ �→ x Dx within-task algorithm : online gradient descent on h . EWA-LL, prior : columns of D i.i.d uniform on unit sphere. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 1 : dictionary learning X = R K → Z = R k → Y = R � h , Dx � = h T Dx . �→ �→ x Dx within-task algorithm : online gradient descent on h . EWA-LL, prior : columns of D i.i.d uniform on unit sphere. Theorem (Corollary 4.4) - ℓ is bounded by B & L -Lipschitz � � T n t T n t 1 1 1 1 � � � � ˆ ℓ ( y t , i , h T ℓ t , i ≤ inf inf t Dx t , i ) E T n t T n t D � h t �≤ C t = 1 i = 1 t = 1 i = 1 √ � T + 1 2 k + C Kk T (log( T ) + 7 ) + BL BL � √ √ n t . 4 T T t = 1 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 1 : dictionary learning X = R K → Z = R k → Y = R � h , Dx � = h T Dx . �→ �→ x Dx within-task algorithm : online gradient descent on h . EWA-LL, prior : columns of D i.i.d uniform on unit sphere. Theorem (Corollary 4.4) - ℓ is bounded by B & L -Lipschitz � � T n t T n t 1 1 1 1 � � � � ˆ ℓ ( y t , i , h T ≤ inf E ℓ t , i inf t Dx t , i ) T n t T n t D � h t �≤ C t = 1 t = 1 i = 1 i = 1 √ � + C Kk T (log( T ) + 7 ) + BL + BL 2 k √ √ . 4 ¯ T n Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 1 (dictionary learning) : simulations simulations X = R 5 → Z = R 2 → Y = R with ℓ the quadratic loss, T = 150, each n t = 100. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 1 (dictionary learning) : simulations simulations X = R 5 → Z = R 2 → Y = R with ℓ the quadratic loss, T = 150, each n t = 100. implementation of EWA-LL, at each step, D is updated using N iterations of Metropolis-Hastings. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 1 (dictionary learning) : simulations simulations X = R 5 → Z = R 2 → Y = R with ℓ the quadratic loss, T = 150, each n t = 100. implementation of EWA-LL, at each step, D is updated using N iterations of Metropolis-Hastings. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 2 : finite set of predictors g ∈G h ∈H x �→ g ( x ) �→ h ( g ( x )) . card ( G ) = G < + ∞ , card ( H ) = H < + ∞ Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 2 : finite set of predictors g ∈G h ∈H x �→ g ( x ) �→ h ( g ( x )) . card ( G ) = G < + ∞ , card ( H ) = H < + ∞ within-task algorithm : EWA, uniform prior. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 2 : finite set of predictors g ∈G h ∈H x �→ g ( x ) �→ h ( g ( x )) . card ( G ) = G < + ∞ , card ( H ) = H < + ∞ within-task algorithm : EWA, uniform prior. EWA-LL, uniform prior. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 2 : finite set of predictors g ∈G h ∈H x �→ g ( x ) �→ h ( g ( x )) . card ( G ) = G < + ∞ , card ( H ) = H < + ∞ within-task algorithm : EWA, uniform prior. EWA-LL, uniform prior. Theorem (Corollary 4.2) - ℓ bounded by C & α -exp-concave � � T m T m 1 1 1 1 � � � � ˆ ℓ t , i ≤ inf inf ℓ ( y t , i , h t ◦ g ( x t , i )) E T m T m g ∈G h t ∈H t = 1 i = 1 t = 1 i = 1 � log G + α log H + C . 2 T n ¯ Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 2 : improvement on existing results The “online-to-batch” trick allows to deduce from our online method a statistical estimator with a controled LTL risk in �� � log G + log H O . T n Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Example 2 : improvement on existing results The “online-to-batch” trick allows to deduce from our online method a statistical estimator with a controled LTL risk in �� � log G + log H O . T n In this case, a previous bound by Pentina and Lampert was in �� � � log G log H O + . T n Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions General regret bound Theorem (Theorem 3.1) - ℓ bounded by C If for any g ∈ G , the within-task algorithm has a regret bound R t ( g ) ≤ β ( g , n t ) , then � � T n t 1 1 � � ˆ ℓ t , i E T n t t = 1 i = 1 �� � T n t 1 1 � � � � ≤ inf y t , i , h t ◦ g ( x t , i ) inf ℓ T n t ρ h t ∈H t = 1 i = 1 � � T + 1 ρ ( d g ) + η C 2 + K ( ρ, π ) � β ( g , n t ) . T 8 η T t = 1 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Transfer learning, multitask learning, lifelong learning... 1 A strategy for lifelong learning, with regret analysis 2 Open questions 3 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Efficient algorithms ? Our online analysis allows to avoid explicit probabilistic assumptions on the data, and allows a free choice of the within-task algorithm. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Efficient algorithms ? Our online analysis allows to avoid explicit probabilistic assumptions on the data, and allows a free choice of the within-task algorithm. However, EWA-LL is not “truly online” as its computation requires to store all the data seen so far. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Efficient algorithms ? Our online analysis allows to avoid explicit probabilistic assumptions on the data, and allows a free choice of the within-task algorithm. However, EWA-LL is not “truly online” as its computation requires to store all the data seen so far. Moreover, its computation is not scalable. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Efficient Lifelong Learning Algorithm : ELLA dictionary learning, Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Efficient Lifelong Learning Algorithm : ELLA dictionary learning, fast update of D and β at each step, truly online : no need to store the data, Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Efficient Lifelong Learning Algorithm : ELLA dictionary learning, fast update of D and β at each step, truly online : no need to store the data, very good empirical performances, Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Efficient Lifelong Learning Algorithm : ELLA dictionary learning, fast update of D and β at each step, truly online : no need to store the data, very good empirical performances, no regret bound. Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions More progress on dictionary learning dictionary learning, Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions More progress on dictionary learning dictionary learning, fast update of β at each step, fast update of D at the end of each task, truly online, Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions More progress on dictionary learning dictionary learning, fast update of β at each step, fast update of D at the end of each task, truly online, very good empirical performances, Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions More progress on dictionary learning dictionary learning, fast update of β at each step, fast update of D at the end of each task, truly online, very good empirical performances, LTL bound in �� � � 1 1 O T + . n Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Algorithms : open questions Open question 1 An efficient algorithm with theoretical guarantees (if possible beyond dictionary learning). Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Algorithms : open questions Open question 1 An efficient algorithm with theoretical guarantees (if possible beyond dictionary learning). theoretical analysis of ELLA ? Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Algorithms : open questions Open question 1 An efficient algorithm with theoretical guarantees (if possible beyond dictionary learning). theoretical analysis of ELLA ? can we justify to update D at each step ? this leads to the next big open problem... Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Optimality of the bounds ELLA : updates D at each step. Doing so, after T tasks with n steps in each task, we would expect a bound in �� � � 1 1 O nT + . n Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Optimality of the bounds ELLA : updates D at each step. Doing so, after T tasks with n steps in each task, we would expect a bound in �� � � 1 1 O nT + . n Denevi et al : bound in �� � � 1 1 O T + . n Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Optimality of the bounds ELLA : updates D at each step. Doing so, after T tasks with n steps in each task, we would expect a bound in �� � � 1 1 O nT + . n Denevi et al : bound in �� � � 1 1 O T + . n So, what are the optimal rates in LL & LTL ? Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Insights from a toy model θ 1 fixed once and for all, task t : θ 2 , t fixed for the task for i = 1 , . . . , n , y t , i = ( θ 1 + ε 1 , i , t , θ 2 , t + ε 2 , i , t ) with ε j , i , t ∼ N ( 0 , 1 ) . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Insights from a toy model θ 1 fixed once and for all, task t : θ 2 , t fixed for the task for i = 1 , . . . , n , y t , i = ( θ 1 + ε 1 , i , t , θ 2 , t + ε 2 , i , t ) with ε j , i , t ∼ N ( 0 , 1 ) . � T � n ˆ 1 θ 1 = i = 1 ( y t , i ) 1 can be computed in the online t = 1 nT setting and one has �� � � � 1 | ˆ θ 1 − θ 1 | = O . E nT Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Insights from a toy model θ 1 fixed once and for all, task t : θ 2 , t fixed for the task for i = 1 , . . . , n , y t , i = ( θ 1 + ε 1 , i , t , θ 2 , t + ε 2 , i , t ) with ε j , i , t ∼ N ( 0 , 1 ) . � T � n ˆ 1 θ 1 = i = 1 ( y t , i ) 1 can be computed in the online t = 1 nT setting and one has �� � � � 1 | ˆ θ 1 − θ 1 | = O . E nT Fits our setting with x = ∅ , g θ 1 ( x ) = θ 1 , h θ 2 ( z ) = ( z , θ 2 ) . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Insights from a toy model θ 1 fixed once and for all, task t : θ 2 , t and ε 1 , t ∼ N ( 0 , 1 ) fixed for the task. for i = 1 , . . . , n , y t , i = ( θ 1 + ε 1 , t , θ 2 , t + ε 2 , i , t ) with ε 2 , i , t ∼ N ( 0 , 1 ) . � T ˆ θ 1 = 1 t = 1 ( y t , i ) 1 can be computed in the online setting and T one has �� � � � 1 | ˆ θ 1 − θ 1 | = O . E T Still fits our setting and LTL ! Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Optimal rates : open questions Open question 2 What are the optimal rates in lifelong learning and in LTL ? Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Optimal rates : open questions Open question 2 What are the optimal rates in lifelong learning and in LTL ? requires to define properly class of predictors, Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Optimal rates : open questions Open question 2 What are the optimal rates in lifelong learning and in LTL ? requires to define properly class of predictors, the optimal rate will also depend on the setting. This leads to the next question... Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Are our definitions even right ? Note that the terminology is not exen fixed : for example, Pentina and Lampert call lifelong learning what we call learning to learn (we don’t claim we are right !). Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Are our definitions even right ? Note that the terminology is not exen fixed : for example, Pentina and Lampert call lifelong learning what we call learning to learn (we don’t claim we are right !). We used : LTL : samples from all the tasks presented at once. 1 LL : tasks presented sequentially, within each task, pairs 2 presented sequentially. why not tasks presented sequentially, but within each 3 task, samples presented all at once ? . Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Are our definitions even right ? Note that the terminology is not exen fixed : for example, Pentina and Lampert call lifelong learning what we call learning to learn (we don’t claim we are right !). We used : LTL : “Batch-within-batch” 1 LL : “Online-within-online” 2 “Batch-within-online”, see our paper and Denivi et al . 3 Pierre Alquier Lifelong Learning

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Towards more models ? One can imagine even more settings : observations not ordered by tasks ? Pierre Alquier Lifelong Learning