instance learning
play

Instance Learning Boris Babenko, Ming-Hsuan Yang, Serge Belongie - PowerPoint PPT Presentation

Visual Tracking with Online Multiple Instance Learning Boris Babenko, Ming-Hsuan Yang, Serge Belongie Kelsie Zhao Content Goal Background: Tracking by Detection Previous Work New Tracking Solution MILTrack Online MILBoost


  1. Visual Tracking with Online Multiple Instance Learning Boris Babenko, Ming-Hsuan Yang, Serge Belongie Kelsie Zhao

  2. Content • Goal • Background: Tracking by Detection • Previous Work • New Tracking Solution • MILTrack • Online MILBoost • Experiments & Results

  3. Goal Track one arbitrary object in video, given its location in first frame

  4. Background: Tracking by detection • Frame 1 is labeled, tracker location known

  5. Background: Tracking by detection • Crop one positive and some negative patches near tracker x 1 x 3 x 2 Positive Negative

  6. Background: Tracking by detection • Use patches to train the classifier x 1 x 3 x 2 Positive Negative {(x 1 , 1), (x 2 , 0), (x 3 , 0)} Classifier

  7. Background: Tracking by detection • Frame 2 comes Classifier

  8. Background: Tracking by detection • Calculate classifier response within a range of the old tracker location X Old location Classifier

  9. Background: Tracking by detection • Find the maximum response location X X Old location New location Classifier

  10. Background: Tracking by detection • Move tracker Frame 1 Frame 2

  11. Background: Tracking by detection • Repeat Positive Negative Frame 2

  12. Background: Tracking by detection • Problem: If tracker location is not precise, might select bad training examples

  13. Background: Tracking by detection • Problem: If tracker location is not precise, might select bad training examples Model start to degrade!

  14. Background: Tracking by detection • Problem: If tracker location is not precise, might select bad training examples Model start to degrade! • How to select good training examples?

  15. Previous Work • Solution 1: multiple positive examples around tracker location x 2 x 5 x 4 x 3 x 1 {(x 1 , 1), (x 2 , 1), (x 3 , 1), (x 4 , 0), (x 5 , 0)} Classifier

  16. Previous Work • Solution 1 Might confuse classifier! x 2 x 5 x 4 x 3 x 1 {(x 1 , 1), (x 2 , 1), (x 3 , 1), (x 4 , 0), (x 5 , 0)} Classifier

  17. Previous Work • Solution 2: Multiple Instance Learning (MIL) [Keeler ‘90, Dietterich et al. ‘97]

  18. Previous Work: Multiple Instance Learning x 12 x 13 x 11 x 31 x 21 • Multiple examples in one bag [Keeler ‘90, Dietterich et al. ‘97]

  19. Previous Work: Multiple Instance Learning x 12 X 1 x 13 x 11 X 2 X 3 x 31 x 21 • Multiple examples in one bag [Keeler ‘90, Dietterich et al. ‘97]

  20. Previous Work: Multiple Instance Learning x 12 (X 1 , 1) x 13 x 11 (X 3 , 0) (X 2 , 0) x 31 x 21 • Multiple examples in one bag • One bag one label [Keeler ‘90, Dietterich et al. ‘97]

  21. Previous Work: Multiple Instance Learning x 12 (X 1 , 1) x 13 x 11 (X 3 , 0) (X 2 , 0) x 31 x 21 • Multiple examples in one bag • One bag one label • Bag Positive if at least one example is Positive [Keeler ‘90, Dietterich et al. ‘97]

  22. Previous Work: Multiple Instance Learning x 12 (X 1 , 1) x 13 x 11 (X 3 , 0) (X 2 , 0) x 31 x 21 {(X 1 , 1), (X 2 , 0), (X 3 , 0)} Classifier [Keeler ‘90, Dietterich et al. ‘97]

  23. Previous Work: Multiple Instance Learning MIL training input: 𝒀 𝟐 , 𝑧 1 … 𝒀 𝒐 , 𝑧 𝑜 , ( X 1 , 1) x 12 where, x 13 x 11 𝒀 𝒋 = 𝑦 𝑗1 … 𝑦 𝑗𝑛 , ( X 3 , 0) ( X 2 , 0) 𝑧 𝑗 = 𝑛𝑏𝑦 𝑘 𝑧 𝑗𝑘 x 31 x 21 [Keeler ‘90, Dietterich et al. ‘97]

  24. Previous Work: Multiple Instance Learning MIL training input: 𝒀 𝟐 , 𝑧 1 … 𝒀 𝒐 , 𝑧 𝑜 , ( X 1 , 1 ) x 12 where, x 13 x 11 𝒀 𝒋 = 𝑦 𝑗1 … 𝑦 𝑗𝑛 , ( X 3 , 0) ( X 2 , 0) 𝑧 𝑗 = 𝑛𝑏𝑦 𝑘 𝑧 𝑗𝑘 x 31 x 21 Bag babel is 1 if at least one instance is 1 [Keeler ‘90, Dietterich et al. ‘97]

  25. Now we have training examples! How to train the classifier?

  26. Previous Work: MILBoost • MIL + boosting Train a boosting classifier that maximizes log likelihood of bags 𝑚𝑝𝑕𝑀 = log (𝑞 𝑧 𝑗 𝒀 𝒋 )) 𝑗 where, 𝑞 𝑧 𝑗 𝒀 𝒋 ) = 1 − (1 − 𝑞 𝑧 𝑗 𝑦 𝑗𝑘 )) 𝑘 [Viola et al. ‘05]

  27. Previous Work: MILBoost • MIL + boosting Train a boosting classifier that maximizes log likelihood of bags 𝑚𝑝𝑕𝑀 = log (𝑞 𝑧 𝑗 𝒀 𝒋 )) 𝑗 where, 𝑞 𝑧 𝑗 𝒀 𝒋 ) = 1 − (1 − 𝑞 𝑧 𝑗 𝑦 𝑗𝑘 )) 𝑘 ~1 1 [Viola et al. ‘05]

  28. Previous Work: MILBoost • Problem: need all training examples [Viola et al. ‘05]

  29. Previous Work: MILBoost But in tracking, only current frame available [Viola et al. ‘05]

  30. Previous Work: MILBoost But in tracking, only current frame available Need an online training algorithm for MIL [Viola et al. ‘05]

  31. Main Contribution of this paper • Online-MILBoost: Online training for MIL-based classifier • MILTrack New tracking solution using Online-MILBoost

  32. MILTrack workflow X New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  33. MILTrack workflow X New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  34. MILTrack workflow X New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  35. MILTrack workflow X New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  36. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: s = 35 in authors’ experiment Babenko et al., 09

  37. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 2. Use MIL classifier to find new tracker location 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) Babenko et al., 09

  38. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 2. Use MIL classifier to find new tracker location 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) 3. 1) Crop positive examples 𝑌 𝑠 = {𝑦| < 𝑠 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: r = 5 in authors’ experiment Babenko et al., 09

  39. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} ( X 1 , 1) 2. Use MIL classifier to find new tracker location x 12 x 13 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) x 11 3. 1) Crop positive examples 𝑌 𝑠 = {𝑦| < 𝑠 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: r = 5 in authors’ experiment Babenko et al., 09

  40. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 2. Use MIL classifier to find new tracker location 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) 3. 1) Crop positive examples 𝑌 𝑠 = {𝑦| < 𝑠 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 3. 2) Crop Negative examples 𝑌 𝑠, 𝛾 = 𝑦 𝑠 𝑢𝑝 𝛾 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑏𝑥𝑏𝑧 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: 𝛾 = 50 in authors’ experiment Babenko et al., 09

  41. MILTrack workflow New frame comes in: 1. Crop out a set of image patches 𝑌 𝑡 = {𝑦| < 𝑡 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 2. Use MIL classifier to find new tracker location 𝑚 𝑜𝑓𝑥 = 𝑚(argmax 𝑦 ∈ 𝑌 𝑡 𝑞(𝑧 = 1|𝑦)) x 31 x 21 3. 1) Crop positive examples 𝑌 𝑠 = {𝑦| < 𝑠 𝑞𝑗𝑦𝑓𝑚𝑡 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} ( X 3 , 0) ( X 2 , 0) 3. 2) Crop Negative examples 𝑌 𝑠, 𝛾 = 𝑦 𝑠 𝑢𝑝 𝛾 1 𝑞𝑗𝑦𝑓𝑚𝑡 𝑏𝑥𝑏𝑧 𝑔𝑠𝑝𝑛 𝑢𝑠𝑏𝑑𝑙𝑓𝑠 𝑚𝑝𝑑𝑏𝑢𝑗𝑝𝑜} 1: 𝛾 = 50 in authors’ experiment Babenko et al., 09

Recommend


More recommend