l1 regularized logistic regression stacking and
play

L1-regularized Logistic Regression Stacking and Transductive CRF - PowerPoint PPT Presentation

Jan 07, 2023 •533 likes •837 views

L1-regularized Logistic Regression Stacking and Transductive CRF Smoothing for Action Recognition in Video Svebor Karaman , Lorenzo Seidenari, Andrew D. Bagdanov, Alberto Del Bimbo Media Integration and Communication Center (MICC) University of

  1. L1-regularized Logistic Regression Stacking and Transductive CRF Smoothing for Action Recognition in Video Svebor Karaman , Lorenzo Seidenari, Andrew D. Bagdanov, Alberto Del Bimbo Media Integration and Communication Center (MICC) University of Florence, Florence, Italy { svebor.karaman, lorenzo.seidenari } @unifi.it, { bagdanov, delbimbo } @dsi.unifi.it http://www.micc.unifi.it/vim/people Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 1 / 18

  2. THUMOS Workshop First International Workshop on Action Recognition with a Large Number of Classes 101 Classes, 5 types: Human-Object Interaction, Human-Human Interaction, Body-Motion Only, Playing Musical Instruments, Sports. 13320 videos (25 groups) Pre-computed and pre-encoded (hard-assigned 4000 BoW) low-level features: STIP, Dense Trajectory Features (MBH, HOG, HOF, TR) 3 splits : 2/3 train, 1/3 test (disjoint groups in train/test) Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 2 / 18

  3. Introduction Our game plan and our goals Priority: establish a working BOW pipeline on given hard assigned coded features (MBH, HOG, HOF, STIP, TR) to establish our baseline Limitations: I Loss due of hard assignment I No contextual features I Lots and lots of classes and features, unclear how to fuse Goal 1: improve the features in our baseline I Use better encoding of provided features (after re-extraction) I Add static contextual features extracted from keyframes Goal 2: experiment with fusion schemes I Regularized stacking of experts I Transductive smoothing of expert outputs Note we did not use any external data or the provided attributes Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 3 / 18

  4. Baseline with provided features (Run-1) Run 1: a respectable baseline Late fusion (sum) of 1-vs-All SVM classifiers (Histogram Intersection Kernel) learned on M = 5 features X E f class( x ) = arg max c ( x ) (1) c f ∈ F org Performance: 74.6% (Split1: 72.85%, Split2: 74.96% , Split3: 75.97%) ��������������� �������� ����������� ������� ������ ������ ����������� ����� ��� � ������������ ������������ ��� � ������������ ��� � ����������� ���������������� ��� ����� � ������������ ��� ������ ������� � ������������ �� � ����� ������������ ������ ���� Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 4 / 18

  5. Better encoding of dense trajectories features Extraction of dense trajectories [Wang:2013] I On a modest cluster of 20CPUs: F 5 nodes F Quad Core 2.7Ghz CPUs F 48GB Total RAM I Total time to extract: 25h I Disk usage: 660GB Extracted features: I Separate x- and y-components (MBHx and MBHy) I Standard concatenation of the two local descriptors (MBH). I Histogram of Gradients (HoG) Fisher encoding of all features independently: I 256 Gaussians with diagonal covariance I Gradients with respect to means and covariances Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 5 / 18

  6. Is context relevant for action recognition? We extract the central frame of each video as keyframe Visualizing the mean keyframe each class is illuminating: Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 6 / 18

  7. Is context relevant for action recognition? We extract the central frame of each video as keyframe Visualizing the mean keyframe each class is illuminating: Basketball Playing Cello Ice Dancing Soccer Penalty Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 6 / 18

  8. Additional contextual features Dense sampled Pyramidal-SIFT [Seidenari:2013] features (P-SIFT and P-OpponentSIFT) on keyframes I Pyramidal-SIFT: three pooling levels, corresponding to 2 × 2 , 4 × 4 , 6 × 6 pooling regions. Each level has its own dictionary: 1500, 2500 and 3000 words respectively. I Spatial pyramid configuration: 1x1, 2x2, 1x3 I Locality-constrained Linear Coding and max pooling [Wang:2010] Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 7 / 18

  9. Late fusion with all features (Run-2) Run-2: more features, better encoding The Fisher encoded MBH, MBHx, MBHy, and the LLC encoded P-SIFT and P-OSIFT are fed to Linear 1-vs-all SVMs Combined with provided feature histograms: total of M = 11 features Performance: 82.46% (Split1: 81.47%, Split2: 83.01%, Split3: 82.88%) Run-1: 74.6% �������� ����������� ������ ������� ��������������� ����������� ����������� � ���� ����������� � ����������� ���� � ����������� ����� ��� ������������� ��� � ������������ ����������� ��� ����� � ����������� ��� ������ ������ � ������������ ��� � ���������������� ��� ������������ ����� �� � ������������ ������ � ���� ������������ � ������������ � ������ ����������� �������� ��������������� ������ ��� ���������� � ������� ������� ����������� Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 8 / 18

  10. Stacking Stacking: learn a classifier on top of the concatenation of expert decisions: [ E j S ( x ) = i ] , for j ∈ { 1 , . . . M } , i ∈ { 1 , . . . N } (2) Having lots of class/feature experts makes THUMOS an excellent playground for this type of fusion approach. Our idea: use L1-regularized LR for class/feature expert selection. Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 9 / 18

  11. Stacking Stacking: learn a classifier on top of the concatenation of expert decisions: [ E j S ( x ) = i ] , for j ∈ { 1 , . . . M } , i ∈ { 1 , . . . N } (2) Having lots of class/feature experts makes THUMOS an excellent playground for this type of fusion approach. Our idea: use L1-regularized LR for class/feature expert selection. Doing it wrong: decisions values on training samples from classifiers trained on those samples (a) Train (b) Test Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 9 / 18

  12. Stacking Stacking: learn a classifier on top of the concatenation of expert decisions: [ E j S ( x ) = i ] , for j ∈ { 1 , . . . M } , i ∈ { 1 , . . . N } (2) Having lots of class/feature experts makes THUMOS an excellent playground for this type of fusion approach. Our idea: use L1-regularized LR for class/feature expert selection. Doing it right: reconstruct the decisions on the training samples by running multiple held out training/test folds (a) Train hold-out (b) Test Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 9 / 18

  13. Logistic regression for stacking (Run-3) Run-3: L1 regularized logistic stacking Motivation: smart weighted/selection scheme Model ( β c , b c ) of class c obtained by minimizing the loss: n ln(1 + e − y i β T S ( x i )+ b ) X ( β c , b c ) = arg min β ,b || β || 1 + C (3) i =1 Performance: 84.44% (Split1: 83.70%, Split2: 85.56%, Split3: 84.07%) Run-2: 82.46% �������� ����������� ������ ������� ��������������� �������� ����������� � ���� ����������� � ���� ����������� � ����� ����������� ��� ������������� ��� � ������������ ����������� ��� ������ � ����������� ��� ������ ����� � ������������ ��� � ���������������� ��� ������������ ����� �� � ������������ ������ � ���� ������������ � ������������ � ������ ����������� �������� ��������������� ������ ��� ���������� � ������� ������� ����������� Svebor Karaman et al. (MICC) THUMOS Submission 40 December 7, 2013 10 / 18

Recommend

Logistic Regression James H. Steiger Department of Psychology and Human Development Vanderbilt

Logistic Regression James H. Steiger Department of Psychology and Human Development Vanderbilt

Introduction The Logistic Regression Model Binary Logistic Regression Binomial Logistic Regression Interpreting Logistic Regression Parameters Examples Logistic Regression and Retrospective Studies Logistic Regression James H. Steiger

777 views • 54 slides
Classifying HIV Vaccination Status with Regularized Logistic Regression Ariful Azad, Arif Khan,

Classifying HIV Vaccination Status with Regularized Logistic Regression Ariful Azad, Arif Khan,

Classifying HIV Vaccination Status with Regularized Logistic Regression Ariful Azad, Arif Khan, Bartek Rajwa, Alex Pothen Purdue University FlowCAP-III, NIH, November 29-30, 2012 This research was supported by grant 1R21EB015707 from the

651 views • 21 slides
Todays lecture Logistic regression How can we use logistic regression for reranking? Shay

Todays lecture Logistic regression How can we use logistic regression for reranking? Shay

Todays lecture Logistic regression How can we use logistic regression for reranking? Shay Cohen How do we set the parameters of a logistic regression model? (based on slides by Sharon Goldwater) How is logistic regression related

652 views • 9 slides
w o o o o o o o x o o o x o o that represents how aligned the o x x x x x x

w o o o o o o o x o o o x o o that represents how aligned the o x x x x x x

Outline Logistic function IAML: Logistic Regression Logistic regression Learning logistic regression Optimization Nigel Goddard and Victor Lavrenko The power of non-linear basis functions School of Informatics

338 views • 6 slides
w o o o o o o o x o o o x o o o that represents how aligned the x x x x x x

w o o o o o o o x o o o x o o o that represents how aligned the x x x x x x

Outline Logistic function IAML: Logistic Regression Logistic regression Learning logistic regression Optimization Nigel Goddard The power of non-linear basis functions School of Informatics Least-squares classification

622 views • 6 slides
w o o o o o o o o x o o o o o x o that represents how aligned the o x x x x

w o o o o o o o o x o o o o o x o that represents how aligned the o x x x x

Outline Logistic function Logistic regression IAML: Logistic Regression Learning logistic regression Optimization Charles Sutton and Victor Lavrenko The power of non-linear basis functions School of Informatics

343 views • 6 slides
Regression 3: Logistic Regression Marco Baroni Practical Statistics in R Outline Logistic

Regression 3: Logistic Regression Marco Baroni Practical Statistics in R Outline Logistic

Regression 3: Logistic Regression Marco Baroni Practical Statistics in R Outline Logistic regression Logistic regression in R Outline Logistic regression Introduction The model Looking at and comparing fitted models Logistic regression in

1.09k views • 47 slides
LEARNING Outline Math Behind Logistic Regression Visualizing Logistic Regression Loss

LEARNING Outline Math Behind Logistic Regression Visualizing Logistic Regression Loss

Logistic Regression CSCI 447/547 MACHINE LEARNING Outline Math Behind Logistic Regression Visualizing Logistic Regression Loss Function Minimizing Log Likelihood Function Batch/Full Logistic Regression Gradient Descent for

498 views • 29 slides
Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335

Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335

Logistic Regression Lecture 6 Logistic Regression Classification Model CS 335 Cost function Gradient descent Linear classifiers and decision boundaries Dan Sheldon Classification Example: Hand-Written Digits Input: 20

541 views • 5 slides
From Logistic Regression to Neural Networks CMSC 470 Marine Carpuat Logistic Regression What

From Logistic Regression to Neural Networks CMSC 470 Marine Carpuat Logistic Regression What

From Logistic Regression to Neural Networks CMSC 470 Marine Carpuat Logistic Regression What you should know How to make a prediction with logistic regression classifier How to train a logistic regression classifier Machine learning

502 views • 33 slides
1 Relation Between Multinomial Logistic Regression Nave Bayes and Logistic Regression

1 Relation Between Multinomial Logistic Regression Nave Bayes and Logistic Regression

Logistic Regression Assumes a parametric form for directly estimating P( Y | X ). For binary concepts, this is: CS 391L: Machine Learning: 1 P Y = X = ( 1 | ) n + w + w i X Bayesian Learning: 1 exp( ) = i 0 i 1

308 views • 5 slides
Workshop 10.5a: Logistic regression Murray Logan August 23, 2016 Table of contents 1 Logistic

Workshop 10.5a: Logistic regression Murray Logan August 23, 2016 Table of contents 1 Logistic

-1- Workshop 10.5a: Logistic regression Murray Logan August 23, 2016 Table of contents 1 Logistic regression 1 2 Worked Examples 5 1. Logistic regression 1.1. Logistic regression 1.1.1. Binary data ( ) Link: log 1 Transform

188 views • 7 slides
Workshop 10.5a: Logistic regression Murray Logan 05 Sep 2016 Section 1 Logistic regression

Workshop 10.5a: Logistic regression Murray Logan 05 Sep 2016 Section 1 Logistic regression

Workshop 10.5a: Logistic regression Murray Logan 05 Sep 2016 Section 1 Logistic regression Logistic regression a d a t r y i n a B ( ) Link: log 1 Transform scale of linear predictor ( , ) into that of the

289 views • 16 slides
Logistic regression and Poisson regression Rasmus Waagepetersen Department of Mathematics

Logistic regression and Poisson regression Rasmus Waagepetersen Department of Mathematics

Logistic regression and Poisson regression Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark October 6, 2017 Outline Logistic regression Poisson regression Binary and count data Linear mixed models very

391 views • 28 slides
Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression

Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression

Learning From Data Lecture 9 Logistic Regression and Gradient Descent Logistic Regression Gradient Descent M. Magdon-Ismail CSCI 4100/6100 recap: Linear Classification and Regression The linear signal: s = w t x Good Features are Important

272 views • 23 slides
Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015

Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015

Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015 http://ce.sharif.edu/courses/93-94/2/ce717-1 / Agenda Agenda Probabilistic Classification Introduction to Logistic regression Binary logistic regression Logistic

362 views • 20 slides
Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

0. Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common property CMU, 2004 fall, Andrew Moore, HW2, pr. 4 2. Linear Regression and Logistic Regression: Definitions Given an input vector X , linear

1.07k views • 22 slides
Topics of the day Logistic regression and generalized linear models Rasmus Waagepetersen

Topics of the day Logistic regression and generalized linear models Rasmus Waagepetersen

Topics of the day Logistic regression and generalized linear models Rasmus Waagepetersen Logistic regression Department of Mathematics Overdispersion Aalborg University Logistic regression with random effects Denmark November 5,

422 views • 7 slides
LOGISTIC REGRESSION, GRADIENT LOGISTIC REGRESSION, GRADIENT DESCENT, NEWTON DESCENT, NEWTON

LOGISTIC REGRESSION, GRADIENT LOGISTIC REGRESSION, GRADIENT DESCENT, NEWTON DESCENT, NEWTON

LOGISTIC REGRESSION, GRADIENT LOGISTIC REGRESSION, GRADIENT DESCENT, NEWTON DESCENT, NEWTON Matthieu R Bloch Thursday, January 30, 2020 1 LOGISTICS LOGISTICS TAs and Office hours Monday: Mehrdad (TSRB 523a) - 2pm-3:15pm Tuesday: TJ (VL

299 views • 14 slides
Binary Logistic Regression + Multinomial Logistic Regression Matt Gormley Lecture 10 Feb. 17,

Binary Logistic Regression + Multinomial Logistic Regression Matt Gormley Lecture 10 Feb. 17,

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Binary Logistic Regression + Multinomial Logistic Regression Matt Gormley Lecture 10 Feb. 17, 2020 1 Reminders

452 views • 43 slides
4. Logistic Regression Simple Logistic Regression Y = 0 or 1 = Pr ( Y = 1 ) ( x )

4. Logistic Regression Simple Logistic Regression Y = 0 or 1 = Pr ( Y = 1 ) ( x )

4. Logistic Regression Simple Logistic Regression Y = 0 or 1 = Pr ( Y = 1 ) ( x ) ( x ) = log = + x logit 1 ( x ) Uses logit link for binomial Y . Equivalently, exp ( + x ) ( x ) = 1 + exp (

896 views • 56 slides
Introduction to Machine Learning Classification: Logistic Regression

Introduction to Machine Learning Classification: Logistic Regression

Introduction to Machine Learning Classification: Logistic Regression compstat-lmu.github.io/lecture_i2ml MOTIVATION A discriminant approach for directly modeling the posterior probabilities ( x | ) of the labels is logistic regression . For

484 views • 9 slides
Logistic regression Shay Cohen (based on slides by Sharon Goldwater) 28 October 2019 Todays

Logistic regression Shay Cohen (based on slides by Sharon Goldwater) 28 October 2019 Todays

Logistic regression Shay Cohen (based on slides by Sharon Goldwater) 28 October 2019 Todays lecture How can we use logistic regression for reranking? How do we set the parameters of a logistic regression model? How is logistic

652 views • 33 slides
Machine Learning - MT 2016 8. Classification: Logistic Regression Varun Kanade University of

Machine Learning - MT 2016 8. Classification: Logistic Regression Varun Kanade University of

Machine Learning - MT 2016 8. Classification: Logistic Regression Varun Kanade University of Oxford November 2, 2016 Logistic Regression Logistic Regression is actually a classification method In its simplest form it is a binary (two classes)

486 views • 14 slides

More recommend