Obstacle Avoidance for Monocular Drones Table of Contents I. - PowerPoint PPT Presentation

Obstacle Avoidance for Monocular Drones

Table of Contents I. Technological context II. Neural Networks for obstacle avoidance i. Deep Reinforcement Learning ii. The two axis of development III. . Depth Net i. FOE Extraction pretraining ii. From FlowNet to DepthNet iii.Multiple Shift DepthNet iv.BatLib + DepthNet v. Auto supervision for finetuning IV. . Future work V. Conclusion Obstacle Avoidance for Monocular Drones 2

I. Technological context Obstacle Avoidance for Monocular Drones 3

Technological context ● Today's trend for consumer drones, – Follow Me – Flight Plan ● Parrot is working on it but it needs some safety features Obstacle Avoidance for Monocular Drones 4

Technological context ● We need to make sure the drone is going to avoid obstacles when in these modes. ● We aim to avoid « jerk » so that the footage remains stable and the customer is still happy ● We cannot use expensive sensors, we only have 1 camera and IMU. Obstacle Avoidance for Monocular Drones 5

Technological context ● Parrot already have a strong piloting library, that accepts high level commands ● They also have a good video stabilization algorithm, so that we can assume input videos is just translated. ?? Video Video stab ? Piloting Motors Obstacle Avoidance for Monocular Drones 6

Analytical solutions ● Structure from motion – SLAM – Obstacle detection – PathFinding ● SLAM is either very expensive or very sparse, which is a problem for consumer products ● These solutions also imply to render a whole 3D environment which is a waste of computation ● We want an end-to-end system that directly applies a policy to get a smooth obstacle detection Obstacle Avoidance for Monocular Drones 7

II. Neural Networks for obstacle avoidance Obstacle Avoidance for Monocular Drones 8

Deep Learning and Autonomous Vehicules ● Very fashionable for autonomous cars since 2005, not so much for drones ● DAVE , 2005, was already trying to avoid obstacles – It learned to drive by mimicking human piloting : it was not reinforcement learning ● LAGR: Learning Applied to Ground Robotics – Made classification to incoming road Muller, U., Ben, J., Cosatto, E., Flepp, B., & Cun, Y. L. (2006). Off-road obstacle avoidance through end-to-end learning . In Advances in neural information processing systems (pp. 739-746). Sermanet, R. H. P., Erkan, J. B. A., Scoffier, M., & LeCun, K. K. U. M. Y. Learning Long-Range Vision for Autonomous Off-Road Driving. Obstacle Avoidance for Monocular Drones 9

Deep Learning and Autonomous Vehicules ● So what's the problem with drones ? – It's impossible to relate depth of the pixels according to the horizon, in cluttered environment, every pixel is an obstacle, what we don't know is its distance ● The learning task is more difficult because it needs a context, and it may be impossible to have a ground truth to work with. – Human piloting ressources are expensive, and not necessarily perfect – Still got some interesting results with imitation training Ross, S., Melik-Barkhudarov, N., Shankar, K. S., Wendel, A., Dey, D., Bagnell, J. A., & Hebert, M. (2013, May). Learning monocular reactive uav control in cluttered natural environments. In Robotics and Automation (ICRA), 2013 IEEE International Conference on (pp. 1765-1772). IEEE.

Reinforcement learning for robots ● Presumably the best way to have a well condensed system, as it only deals with inputs and outputs ● As it is, it's not reasonnable to apply it on a real drone, too many crashes ! Real world is very complex and we need an efficient way of learning Obstacle Avoidance for Monocular Drones 11

Introducing Deep Reinforcement Learning ● Deep Q Learning, solves complexe Atari games only by looking at the picture output and simple rewards. ● Combined with DQN, Deep DPG (DDPG) can solve continuous states problems such as driving games ● Why not drones ? Let's try out Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing atari with deep reinforcement learning. In Nature Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., ... & Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971. Obstacle Avoidance for Monocular Drones 12

Deep Q learning State, Reward Policy Environment s t , r t-1 Action a t T ∑ t' − t r t' = γ R t t ' = t We approximate Q' with a neural net Q(s,a,W) and change our policy to maximize it, ● which is possible for a limited set of possible actions a t = max a Q ( s t ,a ,W ) How to learn Q ? We need a loss function which is differentiable, we try to approach ● Bellman equation y = r t + γ max a' Q ( s t + 1 ,a' ,W t − 1 ) 2 L ( W )=[ y − Q ( s t ,a t ,W t )] ∇ W L ( W )= [ ( y − Q ( s t ,a t ,W ) ) ∇ W Q ( s ,a,W ) ] 13

Deep Dpg learning a t = max a Q ( s t ,a ,W ) Easy to find if the set of action is finite. What about continuous action ? We need to ● find a better method to find the best action possible. We design with Q a function that maps directly to actions, maximizing Q regarding the action a t s t s t Actor (µ) Environment s t ,r t-1 Critic (Q) L ( W ' )=| max a Q ( s t ,a )− Q ( s t , μ( s t ,W ' ) ) | During Inference, only the Actor will be called. We now have a mapping directly from ∇ W ' L =∇ a Q ( s ,a )∇ W ' μ( s,W ' ) previous states to any continuous action Obstacle Avoidance for Monocular Drones 14

Deep Reinforcement Learning applied to autonomous drone flying ● Parrot already has an implemented simulator, based on Gazebo, which behave (hopefully) exactly like the real drone. ● We can use that for testing our algorithms, it's much faster and safer than real training ● Gazebo simulator although realistic for physical interactions is not very similar to real video footage. Our system won't work if it's only trained on simulator Obstacle Avoidance for Monocular Drones 15

The domain adaptation problem ● Suppose we have something working great on simulator, how do we make it work in real life ? ● How can we use already trained networks as a pretrained net ? ● Can we train the network in simulator differently ? Obstacle Avoidance for Monocular Drones 16

The pretraining problem ● The necessary dataset is going to be huge if we want not to be context-sensitive, even for a simulation ● We can try to condense information in higher level such as depth map in a supervised training, and then train a simpler network with reinforcement learning Obstacle Avoidance for Monocular Drones 17

The two axis of development Environment Complexity Depth Map Goal for real Real footage Conditions Obstacle Simulator Depth Map Avoidance in Extraction simulator Network Complexity Supervised Reinforcement Obstacle Avoidance for Monocular Drones 18

III. Depth Net Obstacle Avoidance for Monocular Drones 19

Definitions ● Focus Of Expansion (FOE, Φ) Rigid scene and rotation-less movement only ● Disparity (δ) Obstacle Avoidance for Monocular Drones 20

Introducing Flow Net Flow + Focus of Expansion = Depth Map multiscale reconstruction ● Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., ... & Brox, T. (2015). Flownet: Learning optical flow with convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2758-2766). Obstacle Avoidance for Monocular Drones 21

Introducing Flow Net ● Simple architecture, with only convolutions, LeakyReLU and concatenation ● Earlier feature maps are used for flow construction to retain spatial information ● However, poor results compared to more complicated architectures such as FlowNetC, FlowNet2 and GC-Net Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2016). Flownet 2.0: Evolution of optical flow estimation with deep networks. arXiv preprint arXiv:1612.01925. Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., & Bry, A. (2017). End-to-End Learning of Geometry and Context for Deep Stereo Regression. arXiv preprint arXiv:1703.04309. Obstacle Avoidance for Monocular Drones 22

Flow Net → Depth Net ● Similar problem to solve, we want the network to output directly depth for rigid scenes and rotation less cameras. ● FlowNet is trained on all possible flow and thus can be considered overkill for rigid scenes problem, which allows us to consider smaller variations of FlowNet ● Problem : FlowNet is fully convolutional. Makes sense for optical flow, but not instinctively for depth Obstacle Avoidance for Monocular Drones 23

Why not using depth from FlowNet ? ● For rotation less camera we have this equation : ● Implies an indetermined form when approaching FOE, which will make errors diverge dramatically Obstacle Avoidance for Monocular Drones 24

The Fully Convolutional problem ● Depending on FOE and also optical center (P 0 ) : needed operation for depth is depending on its location. For each area, you have to locate FOE and relative position to P 0 ● Instinctively you would need a FC layer at some point to solve the problem : connection from feature maps to FC vector gives away FOE and P 0 Obstacle Avoidance for Monocular Drones 25