WE INNOV OVAT ATE. E. WE DELIVER. ER. YOU SUCCEED. CCEED. Defense Systems and AI-Enabled Visio ion Computing Kevin Moran (kevin.moran@abaco.com)
Abaco Systems advances the capabilities of the warfighter by providing game changing mission ready embedded systems, components and technologies to defense contractors. Our products reduce program risk, allow technology insertion with affordable readiness, and ultimately help platforms reach deployment sooner with lower cost 2
Highly Experienced Team of 800+ Professionals with Global Reach 3
Our Portfolio 4
ABACO SYSTEMS SMALL FORM FACTOR RUGGED BOX GVC1000 -40°C to +71°C Rugged TX2 SoM Digital Protocols MilCAN / CAN High Speed 10 Gig Ethernet Integrated SATA Storage Expandable Future IO Military Connectors Designed for Rugged applications for use in Harsh environments including Military Vehicles, UAVs, Robotics, Avionics and Industrial. Aligned to military environmental specifications 5
ABACO SYSTEMS SMALL FORM FACTOR RUGGED BOX GVC2000 -40°C to +71°C Rugged GM107 Maxwell ADC DAC ARINC429 (October) 1553 28V I/O Digital I/O Audio Designed for Rugged applications for use in Harsh environments including Military Vehicles, UAVs, Robotics, Avionics and Industrial. Aligned to military environmental specifications 6
Vision functions supported by TX2-integrated GVC1000/2000 • Deep Learning, inference at the edge. • Advanced image processing ISP and compression • Data parallelism using CUDA and OpenCL 7
What is Autonomous? From the Oxford Dictionary… • Freedom to govern itself or control its own affairs • Freedom to act independently • Device capable of operating without direct human control It is not simply… • Automated, • Remotely operated (unmanned), • Guided. For our purposes: Able to conduct complex, cooperative, extended missions within broad objectives, principles and guidelines. 8
Modern Origin: ALVINN 1985-88 9
ALVINN Road Test - 1988 10
Vehicle AI: abridged chronology – personal / commercial ALVINN 1988 Chevy Tahoe: DARPA Urban Challenge 2007 Tesla S Autopilot 2014 Robo-taxi Robo-tram Robo-pizza Robo-beer 11
Vehicle AI: abridged chronology – military ALVINN 1988 Oshkosh TerraMax: DARPA Urban Challenge 2007 Lockheed Martin F-35A 2014 Tremendous variety of applications, missions, environments! Autonomous Platform Autonomous Underwater Vehicle Boeing QF-16 Robot Pack Mule 12
AI will serve many purposes War gaming/planning Logistics Electronic warfare Cyber security, resiliency Collaborative missions AR displays, wearables ISR Missiles Big data, cloud computing 13
DoD invests in Autonomous and AI DoD Unmanned Systems Roadmap Excerpted from roadmap: • Autonomous system is self-directed to reach a human-directed goal. • Machine learning…autonomous systems can develop modified strategies. • In unforeseen situations the autonomous system finds the optimal solution. • Smart teams of unmanned systems operating autonomously…conduct operations in contested environments. • Industry and academic partnerships will be critical. 14
All Branches active with Autonomous R&D Land Sea Air Boeing QF-16 Swarm Patrol Cooperative Task Trucks Sea Hunter S-76 Helicopter - SARA Robotic Sentries Cessna 208 – DARPA ALIAS Legged Squad Support UUV 15
Swarm capabilities across all branches Air Force: Predator swarms Army: Robotic Sentries Navy: Swarm Patrol Boats Common core: Swarm is given broad cooperative mission. • Control is decentralized. • Vehicles extend communications beyond line of sight. • SWARM On-guard and patrolling 24x7. • MDARS Decision ability to move from defensive to offensive posture. • 16
Autonomous vehicles and AR require vision Target classification, tracking, …add registration and fusion, perception stabilization Bot oth demand fid fidelity, , ac accuracy an and low low la late tency! 17
Demanding visual environments and challenges “Off - road”: spherical field On-road: planar field 2D localization • Lack of intentionally active threats • Stable terrain 3D localization • • Blue skies Contested space: omni-directional threats • • Explosions: altered terrain, EMPs • Degraded visual environments • 18
Software - AXIS Enabled Middleware for High Performance Image processing , visualization & graphics AXIS ImageFlex 19
AXIS ImageFlex Image processing and visualization toolkit 20
Vision solutions: General
Machine Vision examples Neural Network Classification Neural Network Object Detection CUDA Interoperability Adaptive Image Fusion GPU Stabilization Image Morphing Situational Awareness 22
Neural Network Classification • Demonstrates integration of AI classifier into an ImageFlex application. • Uses a pre-trained CAFFE-based Googlenet neural network. • Converted to an optimized inference engine via ImageFlex conversion utility, this, in turn, uses NVIDIA’s TensorRT. • ImageFlex also provides a AI annotator tool to assist labelling images to prepare for neural net training for frameworks such as CAFFE using NVIDIA’s DIGITs. (CAFFE (Convolutional Architecture for Fast Feature Embedding) is a deep learning framework, originally developed at UC Berkeley.) 23
Neural Network Classification 0:17 24
Neural Network Object Detection • Demonstrates integration of AI object detection into an ImageFlex application • Uses NVIDIA’s DetectNet object detection neural network. • As per the simpler classifier, this is converted to an optimized inference engine via ImageFlex conversion utility, that uses NVIDIA’s TensorRT. • ImageFlex is used to draw the bounding boxes based on the detections and certainty threshold. 25
Neural Network Object Detection 0:20 26
CUDA Interoperability • Utilizes ImageFlex simple API to facilitate CUDA and ImageFlex interoperability. • Allows user to integrate their CUDA processing into an ImageFlex application. • Eliminates need for complex and confusing OpenGL and CUDA interop code. • API also provides OpenCL interoperability support. 27
CUDA Interoperability 0:20 28
Adaptive Image Fusion • Two types of fusion mode available: o Alpha blend – simple weighted combination of two input images. o Adaptive fusion – designed to maintain high-resolution detail of both images. • Provides per-pixel frequency cut & other fusion weightings. Enables tailoring to best meet fusion demands of application and sensor attributes. • Leverages GPU via OpenGL ‘ shader ’ language - OpenGLSL. GPU agnostic. High performance, minimal CPU overhead. 29
Adaptive Image Fusion 0:32 30
Vision solutions: Augmented Reality / Displays
GPU Stabilization • Use a CUDA-optimized algorithm to stabilize video, correcting for shake in horizontal and vertical planes, plus rotations zoom. • Algorithm is based on the Lucas-Kanade Method. 32
GPU Stabilization 0:20 33
Distortion Correction and Image Morphing Provides simple barrel / pin cushion distortion correction. Complex adaptive correction via “morphing” per grid. More complex dense grid can morph to arbitrary shape. E.g. to compensate for HMD eye pieces or complex optics. Each sub-rectangle in grid can morph in three different ways to suite requirements. 34
Distortion Correction and Image Morphing 0:17 35
Situational Awareness • ImageFlex provides API to create and draw cube maps. • Provides the capability to generate a full or partial ‘skybox’ from live camera array or equirectangular videos files. • Facilities real- time 360 or full “spherical” situational awareness applications via camera stitching. • Configuration utility enable set- up of stitched “skybox” panoramic from camera array and capture configuration parameters for skybox application. 36
Situational Awareness 0:27 37
Conclusion and final messages • Vital Defense is key to security. • Autonomous will transform Defense - Artificial Intelligence as a linchpin. • GPGPU-enabled deep-learning graphics/vision/compute solutions are critical. • Opportunities for developers are incredible and incredibly important! 38
Thank you! Questions? WE INNOVATE. WE DELIVER. YOU SUCCEED. Kevin.Moran@abaco.com
Recommend
More recommend
