Learn earn CN CNNs Ns fr from om Lar arge ge-scale cale We - PowerPoint PPT Presentation

Learn earn CN CNNs Ns fr from om Lar arge ge-scale cale We Web b Im Images ages wi without hout Hu Human an An Anno notations tations Weilin Huang Malong Technologies

Ho How t w to Trai ain a H a High-Perf erfor orma manc nce e CNN June 26, 2017 Data Learni ning ng resour urce ce ImageNet, Webvision How w to do Network How w to do Learni ning ng strategy gy AlexNet, GoogleNet, VggNet, Model el Capabi bility ity ResNet, DenseNet What to do Wh Loss

June 26, 2017 Motivations - Pe Perfor ormance mance on on Im Imag ageNet eNet ha has s been n saturation turation. . From om ~30% 0% (2009) 009) -- -->~2 ~2.2% .2% (2017) 17) - Train ain CNNs Ns withou thout t hu human man labelli elling ng -- --> > weakly kly-superv upervis ised d learn rning ing - Develo lop p ne new appro roaches aches work orking ing on on large ge-scale cale data a in n real-wo world rld scenari narios os - Data ta, mod odel el archi hite tecture cture, los oss, s, train inin ing g strat ategy egy are e all important portant - Train ain CNNs NNs from om web eb image ges s are e most ost com ommon mon tasks sks in n ind ndustri ustries es

Introdu In ducti ction: WebVision Workshop Organizers June 26, 2017 General Chairs J. Berent A. Gupta L. Van Gool R. Sukthankar Program Chairs E. Agustsson Wen Li Limin Wang Wei Li

In Introdu oductio ction: n: Database Construction June 26, 2017 1,000 0 semanti ntic c conce ncept pts from m ILSVRC VRC 2012 WebVi Visi sion on Dataset et: • 2 Sources • 1,000 categories • 2.4M training images • 50K validation Images • 50 K test Image ges Wen Li, Limin in Wang ng, Wei Li, Eirikur kur Agus ustss tsson on, Luc Van Gool ol, "WebV bVisi ision on Database abase: Visual ual Learning rning and d Unde ders rstandin tanding g from om Web Data" .ar arXi Xiv: : 1708.028 8.02862, , 2017. .

Mai ain Ch Chal alleng enge: e: Data Imbalance June 26, 2017

Mai ain Challen enge: e: Label Noise June 26, 2017 Tench Terrapin Caretta

Mai ain Challen enge: e: Label Noise June 26, 2017

June 26, 2017 Related Works - Di Directly ectly le lear arn n from rom no nois isy y la label els s 1. Noise-robu robust st Algorith thms ms 2. Label-cl cleansi eansing ng methods hods --> dif -- ifficul ficult t to id identify ify mis isla labe beled led samples les from m hard train inin ing g samples les - Se Semi-Supervise Supervised d methods thods -- -->N >Need ed a s small ll set of manual ually ly-la labeled beled - Re Recent ent deep ep le learn arning ing ap approaches proaches developed veloped for r both oth groups oups of metho thods ds Im Improve e model l cap apab ability ity of s f stan andard ard neural al networks rks by introdu ducin cing g new trai aining ing strateg ategies. ies.

Methodo dolo logy gy: : Curriculum Learning June 26, 2017 Curriculum iculum learning rning — Train CNNs on tasks with increasing difficulty — Train CNNs using samples with increasing complexity “Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones.” Y. Bengio, J. Louradour, R. Collobert, and J. Weston, Curriculum Learning, ICML, 2009.

Methodo dolo logy gy: : Curriculum Learning Processing June 26, 2017 Steps: ps: - Split a learning problem into a number of subtasks - Order subtasks by difficulty - Decide a task-transform threshold - Find an optimized path that leads to fast convergence and better generalization - Simple principle: proceed harder tasks once easier ones are handled

Methodo dolo logy gy: : Idealistic Curriculum Learning Processing June 26, 2017 T. Matiisen, A. Oliver, T. Cohen, and J. Schulman, Teacher-Student Curriculum Learning, arXiv:170 :1707.0 7.001 0183, 2017.

Methodo dolo logy gy: : Formulate our problem June 26, 2017

Methodo dolo logy gy: : Curriculum Design June 26, 2017 - Split the whole training set into multiple subsets - Rank subsets with increasing complexity - Density-Distance clustering in each category Step One : Similarity Matrix ： Step Two : Sample Density ： Step Three : Sample Distance ： (Rodriguez and Laio, Science , 2014.)

Methodo dolo logy gy: : Curriculum Design Subset1 Subset3 Subset2

Methodo dolo logy gy: : Curriculum Design Subse bset t 1 Subse bset t N Tench Tench Terrapin Terrapin

Methodo dolo logy gy: : Train with Curriculum Learning June 26, 2017 Subset t 1 𝒐 𝒖 𝒏 𝝐𝒈 𝝐𝒈 𝒏 · 𝝐𝑷 𝒍 𝑠 1 =1 𝒏 = · 𝒔 𝒖 Task One 𝒏 𝛜𝒙 𝒋𝒌 𝝐𝑷 𝒍 𝝐𝒙 𝒋𝒌 𝒖 𝒍=𝟐 Subset t 2 𝑢 is number of subtasks, t= 3 𝑠 𝑢 is sample weight, , 𝑠 = {1, 0.5,0.5} 𝑠 2 =0.5 Task Two Subset t 3 𝑠 3 =0.5 Task Three

Methodo dolo logy gy: : Models with Different Training Schemes June 26, 2017 Model- Subset 1 Task sk One Baseli seline ne Model el 1 A Model- Meta Data Task sk Three ee Baseli seline ne Model el 2 B Model- Curriculum Curri rriculu culum m Curricu riculum lum Model el 1 C 2 Subsets Learn earning ing Model- Curriculum Curricu riculum lum Curri rriculu culum m Model el 2 D 3 Subsets Learn earning ing

Methodo dolo logy gy: : Selective Data Balance June 26, 2017 Curriculum Design = 3 subsets Mini-batch = 256 - Samples balance among subsets (three subsets applied) [Subset_1 = 128, Subset_2 = 64, Subset_3 = 64] - Classes balance only on Subset_1 — > Randomly select 128 classes — > Each class only has one sample

Methodo odolo logy gy: : Multi-Scale Convolutional Kernel June 26, 2017 Feature ture 5x5 Map-1 Feature ture Feature ture Concat ncat 7x7 Map-2 Map Input Data Featur ture e 9x9 Map-3 Conv nv.1 Enhan ance ce low-le level vel featu tures res whic ich imp mprov ove e the performa rmance nce (about ut 0.5% 5%).

Result ult: : Testing Loss June 26, 2017 Figure1. Testing ng loss of four ur diff fferent ent mode dels s with Incep eption tion_v2 v2 (also compar paring ing to K-mean an cluste terin ring g in curriculum culum design) gn)

June 26, 2017 Result ults: : Single Model, 10 Crops Table le 1. Dif iffer ferent ent models els Table 3. Model el-D D with h Table 2. Model el-D D with h based d on Incep eption_v2 tion_v2 on various ous networks works various ous amoun unts ts of validati ation on set. hig ighl hly y nois isy y data.

Co Compar ariso isons: ns: Model B & D – Top Positive Categories June 26, 2017 Improve 668 categories, reduce 195 categories, and 137 unchanged

Co Compar ariso isons: ns: Model B & D – Top Negative Categories June 26, 2017 Improve 668 categories, reduce 195 categories, and 137 unchanged

Co Conclusi usions ns June 26, 2017 Summary Su mary: : — > > Train ain high-performa performance nce CNNs Ns from om large ge-scale cale web imag ages es — > > Hand ndle le label bel inc ncon onsi siste stence nce and nd data ta un unbalance alance — > > Be Bette ter r gene neraliza ralization tion capabil pability ity — > > Improv rove e our products ucts where re real-wo world rld data ta was as claw awed d from om Internet rnet with th less s human an labelli lling ng or or labels ls are incon onsis siste tence nce — > > Will ll develop op semi-sup supervised ervised and nd wea eakly kly-supervis upervised ed app ppro roache aches

Team am Members rs June 26, 2017 Our ur Te Team am: Sheng Guo, Weilin Huang, Chenfan Zhuang, Dengke Dong, Haozhi Zhang, Matthew R. Scott, Dinglong Huang Malong Technologies Co., Ltd.

Team am histor ory y Team am members rs ac achievem vements ents on on lar arge-sca scale le chal allen enges: es: - ICCV - 15: ILSVRC2015 (ImageNet): scene classification - 2 nd - CVPR - 15 :Large-scale Scene Understanding Challenge (LSUN): scene classification - 2 nd - CVPR - 15 : ChaLearn Looking at People Challenge 2015: cultural event recognition - 3 rd - CVPR - 16 : Large-scale Scene Understanding Challenge (LSUN): scene classification - 1 st - ECCV - 16: ILSVRC2016 (ImageNet): scene classification - 4 th - CVPR -17: Webvision Image classification – 1 st

About Mal along AI for Product Recognition.