Testing & Validation Evaluating Mobility Performance of - PowerPoint PPT Presentation

Modeling & Simulation, Testing & Validation Evaluating Mobility Performance of Unmanned Ground Vehicles Michael P. Cole 1 Cory M. Crean 1 David J. Gorsich, PhD 1 Paramsothy Jayakumar, PhD 1 Abhinandan Jain, PhD 2 Tulga Ersal, PhD 3 1. US Army TARDEC 2. NASA Jet Propulsion Lab 3. University of Michigan 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Motivation Testing & Validation • Military is highly interested in autonomy-enabled systems • Army currently uses NATO Reference Mobility Model (NRMM) to evaluate vehicle mobility both on & off-road [1] – Shortcoming: cannot evaluate UGV technologies such as autonomy, remote control with latency, ABS, traction control, etc. • Task: How to evaluate unmanned ground vehicle (UGV) mobility performance in simulation? 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Notional Relationship for Teleoperation Testing & Validation 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Background Testing & Validation • Years of research in field of teleoperation – Undersea robots [2], ground robots [3-6], manipulators [7-10], vehicles [12-13] • Literature review highlighted lack of research at high speeds (>25 mph) over range of latencies [12-14] • Developed testbed for UGV simulations • Performed baseline experiment using testbed: teleoperation of Polaris MRZR 4 military vehicle 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Simulation Environment Testing & Validation • JPL Rover Analysis Modeling & Simulation (ROAMS) [15] – High fidelity dynamics engine – Latency injection – User input via steering wheel & pedals User Time Delay x(t – 𝜐 (t)) Teleoperation Schematic Driver Visual Feedback UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited. 8/9/2018

Modeling & Simulation, UGV Modes of Teleoperation Testing & Validation • Pure teleoperation – No driver aids – Latency: 0 to 1000ms by 200 ms • Enhanced teleoperation [13] – Model free predictor – First order time delay system – Latency: 0 to 1000ms by 200 ms Teleoperation Predictor Schematic [13] 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Participants and Test Procedure Testing & Validation Participants • Internal research team, 7 users • Little to no prior experience using vehicle simulators Test Procedure Course Layout in ROAMS • Task: drive along centerline of curvy roadway • Guidance: maximize speed while minimizing path deviations • Training phase prior to testing phase • Run failure criteria – Two wheel lift off – Drive off roadway for >5 seconds 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Simulation Results Testing & Validation • As latency increases, average speed decreases and RMS error increases. 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Performance Effects of Training Phase Testing & Validation Post training, drivers achieved higher average speeds and higher RMS error (path deviation). Drove more aggressively. 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Enhanced Teleoperation Performance Testing & Validation Enabling predictor increased performance. Statistically significant increase in average speed. Statically significant decrease in RMS error. 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Mobility-Latency Relationship Testing & Validation • Overlaid TARDEC teleoperation simulation results – 3 different vehicles – 3 different path following scenarios • Exponential regression used to develop functional relationship 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Conclusions Testing & Validation • Mobility performance worsens with increased latency. • Functional relationship provides capability to predict teleoperation mobility performance for path following scenarios. 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Future Work Testing & Validation • Investigating autonomy-enabled systems • Semi & Full autonomy – Human/Machine collaboration – Waypoint following – MPC-based autonomy • Soft soil terrains – Sand, grass, mud • V&V using test data 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, Acknowledgements Testing & Validation This research was carried out in part at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, References Testing & Validation 1. D. Ahlvin and P. Haley, "NATO Reference Mobility Model Edition II, NRMM II User’s Guide," U.S. Army Waterways Experiment Station, Corps of Engineers, Vicksburg, MS, Technical Report Number GL-92-19, 1992. 2. C. Bulich, A. Klein, R. Watson, and C. Kitts, "Characterization of delay-induced piloting instability for the triton undersea robot," 2004 IEEE Aerospace Conference, Big Sky, MT, United States, vol. 1, pp. 409-423, 2004. 3. J. P. Luck, P. L. McDermott, L. Allender, and D. C. Russell, "An investigation of real world control of robotic assets under communication latency," 2006 ACM Conference on Human-Robot Interaction, Salt Lake City, Utah, United States, vol. 2006, pp. 202-209, 2006. 4. F. Penizzotto, S. Garcia, E. Slawinski, and V. Mut, "Delayed Bilateral Teleoperation of Wheeled Robots including a Command Metric," Mathematical Problems in Engineering, pp. 460476 (13 pp.), 2015. 5. E. Slawinski, V. A. Mut, P. Fiorini, and L. R. Salinas, "Quantitative Absolute Transparency for Bilateral Teleoperation of Mobile Robots," IEEE Transactions on Systems, Man and Cybernetics, Part A (Systems and Humans), vol. 42, no. 2, pp. 430-42, 2012. 6. J. Storms, K. Chen, and D. Tilbury, "A shared control method for obstacle avoidance with mobile robots and its interaction with communication delay " The International Journal of Robotics Research, vol. 36, no. 5-7, pp. 820-839 2017. 7. Z. B. Tamas Heidegger, "Extreme Telesurgery," INTECH, Croatia, 2010. 8. A. K. Bejczy, W. S. Kim, and S. C. Venema, "The phantom robot: predictive displays for teleoperation with time delay," IEEE International Conference on Robotics and Automation, Los Alamitos, CA, USA, pp. 546-51, 1990. 9. J. C. Lane, C. R. Carignan, B. R. Sullivan, D. L. Akin, T. Hunt, and R. Cohen, "Effects of time delay on telerobotic control of neutral buoyancy vehicles," 2002 IEEE International Conference on Robotics and Automation, Piscataway, NJ, USA, vol. 3, pp. 2874-9, 2002. 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.

Modeling & Simulation, References Testing & Validation 10. W. R. Ferrell, "Remote manipulation with transmission delay," IEEE Transactions on Human Factors in Electronics, vol. HFE-6, no. 1, pp. 24-32, 1965. 11. Avatar Teleopeation [Online], available: /www.nrec.ri.cmu.edu/projects/long_distance_teleoperation_avatar/. 12. J. Davis, C. Smyth, and K. McDowell, "The effects of time lag on driving performance and a possible mitigation," IEEE Transactions on Robotics, vol. 26, no. 3, pp. 590-593, 2010. 13. Y. Zheng, M. J. Brudnak, P. Jayakumar, J. L. Stein, and T. Ersal, "An Experimental Evaluation of a Model-Free Predictor Framework in Teleoperated Vehicles," 13th IFAC Workshop on Time Delay Systems, 2016. 14. T. T. Vong, G. A. Haas, and C. L. Henry, "NATO Reference Mobility Model (NRMM) Modeling of the DEMO III Experimental Unmanned, Ground Vehicle (XUV)," Army Research Laboratory, ARL-MR-435, 1999. 15. A. Jain, J. Guineau, C. Lim, W. Lincoln, M. Pomerantz, G. Sohl, and R. Steele, "ROAMS: Planetary surface rover simulation environment," International Symposium on Artificial Intelligence, Robotics and Automation in Space, Nara, Japan, 2003. 8/9/2018 UNCLASSIFIED: Distribution Statement A. Approved for public release; distribution is unlimited.