Classifying logistic vehicles in cities using Deep learning Salma - PowerPoint PPT Presentation

Classifying logistic vehicles in cities using Deep learning Salma Benslimane, Simon Tamayo, Arnaud De La Fortelle Mines ParisTech – PSL Research University, Center for Robotics, Paris 75006, France Chair of Urban Logistics – City of Paris, La Poste, ADEME, Groupe Pomona, Renault To cite this work: Salma Benslimane, Simon Tamayo, Arnaud de La Fortelle. Classifying logistic vehicles in cities using Deep learning. World Conference on Transport Research, May 2019, Mumbai, India. ⟨ hal-02144606 ⟩ 15 th World conference on Transport Research - 2019 Mumbai, 26-31 May 2019 1 Schlumberger-Private

• Context • Workflow • Vehicle categorization OUTLINE • Data acquisition • Deep Learning Model • Future Work 2 Schlumberger-Private

Absence of precise information Current use of costly, Increase of small freight about congestion, pollution and inaccurate physical sensors transportation vehicles inside cities freight effect on city for vehicle counting Context 3 Schlumberger-Private

Research question Which policies should Paris city implement as a response to e-commerce effect and structural changes on logistics and transportation? City planning and policies Video based diagnosis Need for quantitative information about: Number of vehicle Classification of freight vehicles’ by type E-commerce last mile delivery inside the city 4 Schlumberger-Private

Demo 5 Schlumberger-Private

Workflow Counting and classifying logistic vehicles Model training and Vehicle classification Data acquisition Data preprocessing evaluation Select and define Query images of the Clean scrapped Train a classifier for categories to classify defined categories images and use a logistic vehicles (balanced) CNN to crop them detection 6 Schlumberger-Private

Vehicle categorization 4 categories for freight transportation in Paris Particularly used in France, not present in existing Datasets Text Text Text Text GVM > 3.5 t GVM ≤ 3.5 t GVM ≤ 3.5 t GVM ≤ 2.7 t H > 3 m 2 m < H ≤ 3m H ≤ 2 m H ≤ 2 m Light-duty Non-Logistic Heavy-duty Medium-duty 7 Schlumberger-Private

Data acquisition Web scraping for balanced and personalized dataset Search Engine (Google) 3D CAD website (hum3D) ○ ○ Define models per category Query categories of vehicles ○ Scrape images per model and ○ Manually classify into caregories organize them in categories Results Results ○ Dataset of 4 categories ○ Dataset of 4 categories Limitations Limitations ○ Synthetic images ○ Corrupt files ○ No background noise ○ No vehicles in images ○ Unbalanced dataset ○ Background noise ○ Same angles of view Balanced , multiple view, diverse dataset with reduced manual annotation 8 Schlumberger-Private

Dataset cleaning ● Cleaning the dataset from Google: ○ Deleting corrupt files ○ Correcting the extensions ○ Deleting images not containing vehicles using existing object classification neural netwoks ● Cleaning the entire dataset: ○ Delete noise of the background by applying convolutional neural network YOLO to the images and cropping only the vehicle Illustration of image processing by using DarkNet classifier to classify and crop the vehicle from the image 9 Schlumberger-Private

Model training Transfer learning 2 methods: ○ Train last layer (feature extraction) ○ Initialize weights of network with pretrained model on ImageNet (weight initialization) 10 Schlumberger-Private

Transfer learning Results Training CNN Model 90% 72,000 images CNN Model 10% Accuracy Inception CNN 86.2 % MobileNet CNN 90.2 % Testing 11 Schlumberger-Private

Transfer learning: Results Result of training Inception V3 and MobileNet V2 on the 72 000 images dataset Feature extraction Weights initialization Inception V3 MobileNetV2 Inception V3 MobileNetV2 86.2 % 90.1 % 91.8 % 90.47 % Accuracy 194s 65s 60 minutes 44 minutes Running time Passenger vehicle Light-duty vehicle MobileNet feature extraction normalized confusion matrix 12 Schlumberger-Private

Demo Summary Detect and classify logistic vehicles from a video feed -> count vehicle by type Video demo of the pipeline output 13 Schlumberger-Private

Future Work • Higher category refinement in non-logistic categories (adding SUVs, PickUps, etc.) • Increase Database size & vehicle images’ angle of view • Coordinating counting vehicles between intersections 14 14 Schlumberger-Private

Questions? Thank you 15 Schlumberger-Private

Context Impact of e-commerce on rapid increase of freight transportation inside the cities Use of smaller capacity vehicles (SUVs, Vans) for last miles delivery Evolving size and structure of freight transportation fleet Value of B2C e-commerce sales in France from 2012 to 2018 (in billion U.S. dollars) 16 Schlumberger-Private

Classifier Architecture MODEL PROCESSING & NETWORK WEB SCRAPING CATEGORIZATION CROPPING TRAINING Select and define Query images of the Clean scrapped Train a classifier for categories to classify defined categories in images and use a logistic vehicles balanced numbers CNN to crop them detection Category 1 Category 1 Category 1 Processed Models.txt Raw Images Images Category 2 Category 2 Category 2 Processed Models.txt Raw Images Images Category n Category n Category n Processed Models.txt Raw Images Images 17 Schlumberger-Private

Initial vehicle categorization Considering 6 categories of vehicles (5 logistics): Large trucks Heavy-duty Medium-duty Careful: High overlap between classes Remorques Camions Fourgons Intermediate Light-duty Non-logistic duty Note: this categorization is not used in existing dataset => existing datasets cannot be directly used for training Fourgonnettes VUL Passenger 18 Schlumberger-Private

Vehicle categorization Considering 4 categories of vehicles (3 logistics) with less overlap: Heavy-duty Medium-duty Light-duty Non logistic GVM > 3.5 t GVM ≤ 3.5 t GVM ≤ 3.5 t GVM ≤ 2.7 t H > 3 m 2 m < H ≤ 3m H ≤ 2 m H ≤ 2 m 19 Schlumberger-Private

Data acquisition: Web scraping for balanced and personalized dataset Aim ● Construct a balanced dataset with minimum manual labellisation by querying images of vehicles by subcategory from 3D CAD website by model from a Search Engine (Google) (hum3D) ○ Define models per category ○ Scrape images per model and ○ Query categories of vehicles organize them in categories ○ Manually classify into subtypes Results Results ○ Dataset of X categories ○ Dataset of X categories Limitations Limitations ○ Corrupt files ○ Synthetic images ○ Mismatch of name extension and ○ No background noise type ○ Unbalanced dataset ○ No vehicles in images ○ Background noise ○ Same angles of view 20 Schlumberger-Private

Transfer learning ● 2 methods explored: ○ Train last layer (feature extraction) Figure 18: Representation of transfer learning architecture ○ Initialize weights of network with pretrained model on ImageNet (weight initialization) 21 Schlumberger-Private

Transfer learning: Results 6 CATEGORY DS 4 CATEGORY DS 4 CATEGORY DS 6000 images 60 000 images 72 000 images 4 CATEGORY 4 CATEGORY DS UNBALANCED DS 4000 images 150 000 images 74 % 93 % 82 % 86 % 86 % --- 96 % 88 % --- 90 % Figure 19: Results of iterations for training convolutional networks on generated datasets 22 Schlumberger-Private

Extended video 23 Schlumberger-Private