Team 2007 ECE Senior Design 2019/2020 Final Oral Presentation - PowerPoint PPT Presentation

Team 2007 ECE Senior Design 2019/2020 Final Oral Presentation Package Delivery 4-Dec-2019 UAV Sponsor UConn ECE Department Faculty Advisor Professor Shalabh Gupta Team Members Alex Maric (EE/CSE) Brandon D’Agostino (EE/CE) Brian Fomenko (EE)

Outline ● Background ● Project Description ● Design Budget ● ● Timeline ● RACI Chart ● Q & A

Background ● Unmanned Aerial Vehicle (UAV) & drone applications Military ■ Disaster Relief ■ Recreation ■ Package Delivery ■ ● Shipping giants investing in drone technology Amazon ■ General Atomics MQ-9 Reaper DHL Hunter-killer UAV ■ Google Wing ■ ● Why? Faster shipping ■ Service to remote locations ■ Potential cost savings ■ Eco-friendly Amazon Prime Air DJI Mavic Pro ■ Delivery Drone Consumer Drone

Project Definition Design Focus Package Delivery Drone ● No problem or specific statement-of-need provided ● Define our own objective [Reference] Last year’s UConn Senior Design Team 1829 (2018/2019) ● Bluetooth-controlled physical manipulator ■ Optical Flow Sensor (stability) ■ Basic object recognition ■ No object position relative to drone or ■ UConn Senior Design Team 1829 autonomous navigation capabilities

Project Objective ● Build fully-functional remote controlled drone ● Modify drone to Identify packages ■ Calculate package position/distance relative to drone ■ Navigate to package position ■ Descend to pick-up position ■

Project Objective Navigate to pick-up 1 Identify package area (manual) 2 Calculate package 3 Navigate to package position vector Lower into pick-up 4 position Y 5 position X Z [Xp, Yp, Zp]

Constraints & Considerations ● Indoor operation No GPS available ■ ● Complexity Any one feature (object recognition, autonomous flight, etc.) could be a ■ senior design project unto itself! Try to leverage as many existing technologies/products as possible ■ Time ● Two academic terms (Fall 2019 & Spring 2020) ■ ● Budget $1,000 USD, provided by UConn ECE department ■

Design Overview Drone Development Platform Phase 1 - Build remote-controlled drone - Open-source, extensible platform - Economical Phase 2 Computer Vision & Object Recognition - Recognize packages - Output position, size, index, etc. Phase 3 Package Position Calculation - Determine package position/size relative to drone - Communicate with flight controller Phase 4 Autonomous Navigation - Hook into autopilot stack - Use calculated data as reference for autonomous navigation

Phase 1 - Drone Development Platform ● No pre-built drones , have to build one Should be extensible ■ Cost ■ Requires extensive research ● ● Multiple components Holybro S500V2 Kit Airframe ■ PM07 Power Module DC Motors ■ Electronic speed controllers ■ Power distribution and supply modules ■ Pixhawk 4 Flight Flight controller ■ Controller Radio receiver FrSky Radio ■ Receiver Remote transmitter ■ LiPo Battery ■ FrSky Radio Etc. Transmitter ■

Phase 1 - Drone Development Platform ● Pixhawk 4 Flight Controller Autopilot hardware (open-hardware) ■ Hiqh-quality & low-cost ■ Built-in sensors (IMU, barometer, GPS, ...) ■ Runs PX4 Flight Control Software ■ ● PX4 Flight Control Software Open-source flight control software for drones (and more) ■ Modular and extensible architecture ■ Built for autonomy ■ Pixhawk 4 ● Purpose Flight Controller Read sensor data ■ Drone position/orientation/velocity ■ Drone control & navigation ■ Radio communication ■

Phase 2 - Computer Vision & Object Recognition ● Pixy2 Camera with Onboard Image Processor ■ Object-recognition ■ ● Configured with PixyMon application Object training ■ Camera monitoring ■ Debugging programs ■ Pixy2 Camera Signature number ■ ● Outputs object data Position, size, X/Y screen coordinates ■ Dimensions, index ■ ● Communication interfaces I2C, SPI, UART, USB ■ Pixy2 connected to embedded system

Phase 2 - Computer Vision & Object Recognition PixyMon Data Output seen from microcontroller program PixyMon with Signature Label Numbers Example Pixy barcodes

Phase 2 - Computer Vision & Object Recognition Raspberry PI USB port Pixy2 Interface Connections

Phase 3 - Package Position Calculation ● Pixy2 Provides screen-space object position, orientation, and size ■ Accessible over UART/SPI/I2C ■ ● Pixhawk 4 Flight Controller Provides drone position, orientation, altitude, velocity, etc. ■ Uses built-in Inertial Measurement Unit (IMU) ■ Accessible over UART/SPI/I2C ■ ● Known quantities Recognized object size ■ Camera orientation relative to drone ■ Image size, FOV, etc. ■ ● Linear Algebra Go from camera-space coordinates… ■ To drone-space coordinates ■ Reverse of 3D graphics transforms ■

Phase 3 - Package Position Calculation

Phase 4 -Autonomous Navigation ● PX4 Flight Control Software Storage Open-source ■ Database Parameters Logger (Missions/FLASH) (EEPROM/SD/...) (MAVlink/SD) Highly-modular, extensible ■ Well documented ■ External Drivers Additions ● MAVLink Camera Gimbal/Mount (UART/UDP) Control Message Raspberry Pi (or equivalent) ■ Bus Pixy 2 Airspeed/Optical ■ FastRTPS GPS Flow/… (UART/UDP) Package data in drone-space Sensors ■ coordinates Object RC Input IMU Drivers Recognition Computer (PPM/SBUS/...) (SPI/I2C/CAN) (UART/SPI/I2C) ● Modifications Flight Control Integrate with proven PX4 ■ autopilot Position Position/Altitude Altitude/Rate Autonomous Controller Estimator Controller Flight Use package position as reference ■ Output Sensors Electronic Speed Drivers Hub Control (ESC/Servo)

Block Diagram Embedded System PX4 ESC & Motors Flight Controller Drone position/velocity/orientation Drone-space package position/orientation Camera-space package size/position/orientation Pixycam Pixy2 Sensors - IMU - Compass - Barometer - Etc. Remote Control

Project Management - Budget

Project Management - RACI Chart

Project Management - Timeline

Thank You! Questions?