Hardware & Software Platform for Next Generation Industrial - PowerPoint PPT Presentation

Hardware & Software Platform for Next Generation Industrial Drones Chetak Kandaswamy Kai Yan Helmut Prendinger

What’s next in industry drones? Market: Technical topics: • Infrastructure inspection • Advanced controller (No GPS, complexed obstacles) • Agriculture • Long endurance and Self-Diagnosis • Disaster Observation • Vision based sensing • Search & Rescue

Infrastructure inspection Under-bridge inspection, No GPS. • Using external camera to • automatically detect and maintain the position of copter. Small LiDARs on-board for • secondary collision avoidance. Wired power supply • Developed by enRoute Co., Ltd. • In-use (Feb., 2016)

Long range observation Achieved 67 minutes hovering, • with a single 450Wh battery Aero-efficient frame made by • TORAYCA T-800S (The same on Boeing-787 Dreamliner) Self-Diagnosis battery pack, • warning ahead of failure. developed by Hitach Maxell, Ltd. Developed by enRoute Co., Ltd. • In-production (May., 2016)

Vision based sensing Depth sensing with a single Camera • Surrounding sensing with four cameras for 360 degree collision avoidance. • Enabled by Jetson TX1 (Implemented in CUDA) • Developed by LabRomanec Inc., Developer’s kit available soon (www.labromance.com) •

Drones as service Hobbyist and scientific Surveillance research - Aerial reconnaissance - Aerial video capturing - Track endangered - Journalism event capture animals/poachers - Cricket/Football/wedding - Track solar panels - Track property - Agricultural farms - Dangerous place - Railway lines - Extreme sports - Defense against other Lightweight Drones drones Deep Transfer Learning Rescue Missions Delivery Drones - High range, good - Vaccines to remote cameras locations - Good samaritans taking - Courier in crowded area care of elderly - Pizzas/dry cleaning - Dengue epidemic - Delivery in dangerous places Security

Object Recognition: ImageNet

Pixel-wise label: PASCAL VOC

FCN-8 ● Transform Fully connected layers into Convolutional layers ● Instead of classes, get a heatmap at the output ● Learnable upsampling to bring output to initial size ● Refine the output using different layer’s predictions ○ Shallow layers : fine scale ○ Deep layers : coarse scale

An example : FCN-8s

Deep Transfer Learning (DTL) • DTL emerged as a new paradigm in machine learning in which, a machine is trained using deep models on a source problem, and then transfer learning to solve a target problem. • DTL is an alternative to transfer learning with shallow architectures, in which one specifies a model to several hidden levels of non-linear operations and then estimates the parameters via the likelihood principle. Why DTL? Utilizes the high-level features using Deep Models. Utilizes Transfer Learning method for limited labeled data problems. Overcomes traditional Transfer Learning methods negative feature transfer causing optimization to fall into bad solution space.

DTL method 1: Layerwise Transfer Learning Application: Drug-discovery Task : Classification of chemical mechanisms of action (MOA) by identifying substances that alter the phenotype of a cell which prevent tumor growth and metastasis. Classify : Host cell or Tumor cell Challenge : Every day thousands of drugs are tested on millions of samples. Each sample has ~5000 cells leading to billion of cells to check. Capturing the images for analysis takes 6 months at a time. Costing 10,000 Euros for each trail. Cancerous cells of Breast Examples of different MOA captured after compound incubation of Breast Cancer cells. Result of DTL: Transference of weights of the source model obtained positive transference and we observe around 30% computation speed up and improvement in overall efficiency.

DTL Method 1 for Drones Crowd Unlabeled sourcing Images Validating the model Drone Data with the drone data Deep transfer learning Labeled Images Fine-tuning Layerwise Other Data Source-Target-Source Ensemble Multi source Unlabeled Implementing existing Images Feature search space deep transfer learning methods on Caffee Aerial images: Google map Satellite Non-aerial images: Imagenet

DTL method 2: Multi-Source STS: Cross-sensor Biometrics Recognition Example: In case of periocular images captured rom multiple devices may have different resolution, size, Illumination setting, etc. Practical problems of cross-sensor biometrics is that these data is collected from various devices and often we need to train machine separately for different machines. (Intro-compatibility issues)

Result of Multi-Source STS Result : DTL performed ~10 % better than the Deep Learning model.

DTL Method 1 and 2 for Drones Input data variations: (Multiple sources) Angles - 45 degree or 90 degree Altitude - High or Low Resolution - High or Low Crowd Unlabeled sourcing Images Validating the model Drone Data with the drone data Deep transfer learning Labeled Images Fine-tuning Layerwise Other Data Source-Target-Source Ensemble Multi source Unlabeled Implementing existing Images Feature search space deep transfer learning methods on Caffee Aerial images: Google map Satellite Non-aerial images: Imagenet

Training methods Object detection Segmentation Training VGGNet GoogLeNet ResNet Inception-v4 FCN-8 SegNet (New model) methods Deep Learning ImageNet + Fine- tuning on drone dataset Source-Target- Source Multi-source Ensemble (Multi-source ensemble) (New method)

Initial members: Prendinger Lab. Check out for updates on http://www.deepdrone.net/ ● Deep learning for Drones ● Jetson TX1 based flight controller ● Silver/Bronze cloud based mission controller ● Deep Drone Dataset (D3) ○ Online Annotation Tool