Next Steps to Creating Personalized and Adaptive Virtual Reality/Augmented Reality Applications Wendy Nilsen Computer and Information Science & Engineering Directorate National Science Foundation
National Science Foundation’s Mission “To promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense...” Image: NSF's current headquarters, 4201 Wilson Boulevard, Arlington, VA. Credit: National Science Foundation
NSF Supports All of Science & Engineering NATIONAL SCIENCE FOUNDATION OFFICE OF THE DIRECTOR NATIONAL SCIENCE BOARD (NSB) 703.292.8000 703.292.7000 France A. C ó rdova Maria T. Zuber Director Chair Vacant Diane L. Souvaine Deputy Director Vice Chair Joan Ferrini-Mundy OFFICE OF NATIONAL SCIENCE INSPECTOR GENERAL BOARD OFFICE Chief Operating Offic r (OIG) Allison C. Lerner, Michael Van Woert, Inspector General Executive Offic r 703.292.7100 703.292.7000 DIRECTORATE FOR DIRECTORATE FOR DIRECTORATE FOR DIRECTORATE FOR DIRECTORATE FOR DIRECTORATE FOR DIRECTORATE FOR SOCIAL, BEHAVIORAL, & BIOLOGICAL COMPUTER & EDUCATION & HUMAN GEOSCIENCES MATHEMATICAL & ENGINEERING ECONOMIC SCIENCES SCIENCES INFORMATION SCIENCE & RESOURCES (GEO) PHYSICAL SCIENCES (ENG) (BIO) ENGINEERING (CISE) (EHR) (MPS) (SBE) Fay L. Cook, James F. Kurose, William (Jim) Lewis, Barry W. Johnson, Roger Wakimoto, James L. Olds, Fleming Crim, Acting Assistant Acting Assistant Assistant Director Assistant Director Assistant Director Assistant Director Assistant Director Director Director Erwin Gianchandani, Kellina M. Craig- Margaret Cavanaugh, Deborah Lockhart, Jane Silverthorne, Deputy AD Sylivia M. James, Clifford Gabriel , Henderson Deputy AD Deputy AD Deputy AD Acting Deputy AD Acting Deputy AD Deputy AD 703.292.8500 703.292.8800 703.292.8400 703.292.8900 703.292.8600 703.292.8300 703.292.8700 OFFICE OF OFFICE OF LEGISLATIVE OFFICE OF BUDGET, OFFICE OF DIVERSITY & OFFICE OF THE OFFICE OF INTEGRATIVE OFFICE OF INFORMATION INCLUSION GENERAL COUNSEL ACTIVITIES & RESOURCE INTERNATIONAL SCIENCE & PUBLIC AFFAIRS FINANCE, & AWARD MANAGEMENT (ODI) (OGC) (OIA) MANAGEMENT & ENGINEERING (OLPA) (OISE) (BFA) (OIRM) Joanne S. Tornow, Rebecca Keiser, Martha A. Rubenstein, Rhonda Davis, Lawrence Rudolph, Suzanne Iacono, Amanda Greenwell, Head / Chief Human e e Head Head / Chief Financial e Head General Counsel Head Capital Offic r Head Offic r Peggy Hoyle , Samuel B. Howerton , e 703.292.8020 703.292.8040 Donna Butler, e 703.292.8070 Deputy Offic H e ad Teresa Grancorvitz , Deputy GC e Deputy Offic H ead Deputy Offic H e ad 703.292.8060 703.292.8710 703.292.8100 703.292.8200 National Science Foundation 4201 Wilson Boulevard Arlington, Virginia 22230 TEL: 703.292.5111 | FIRS: 800.877.8339 | TDD: 800.281.8749 February 2017 e
CISE Divisions Office of Advanced Cyberinfrastructure supports and coordinates the development, acquisition, and provision of state-of-the-art cyberinfrastructure resources, tools and services essential to the advancement and transformation of science and engineering. Computing and Communication Foundations advances computing and communication theory, algorithms for computer and computational sciences and architecture and design of computers and software. Computer and Network Systems invent new computing and networking technologies and finds new ways to make use of current technologies. Information and Intelligence Systems studies the interrelated roles of people, computers, and information to increase our ability to understand data, as well as to mimic the hallmarks of intelligence in computational systems
CISE by the Numbers: FY 2016 success 23% rate 7,228 senior researchers FY 2016 $934 M research budget 1,908 1,238 awards other professionals 8,301 490 proposals postdoctoral associates 18,241 6,565 people supported graduate students 372 panels 2,660 undergraduate students
Data Current as of 2014
CISE programs address national priorities For a comprehensive list of CISE funding opportunities, visit: http://www.nsf.gov/funding/pgm_list.jsp?org=CISE Image Credit: ThinkStock Cybersecurity Image Credit: Texas Advanced Computing Center Image Credit: CCC and SIGACT CATCS National Strategic Big Data R&D Computing Initiative Image Credit: Eliza Grinnell/Harvard SEAS Image Credit: Calvin Lin, University of Texas, Austin Image Credit: US Ignite Manufacturing, Image Credit: ThinkStock Computer Science Robotics, & Smart Education Smart Cities Understanding the Systems Brain
CISE’s Economic and Societal Context • CISE is at the center of an ongoing societal transformation and will be for decades to come. • Advances in computing, communications and information technologies, and cyberinfrastructure: - accelerate the pace of discovery and innovation; and - are crucial to achieving national and societal priorities. Image Credit: Nicolle Fuller, NSF Image Credit: ThinkStock Image Credit: Texas A&M University Image Credit: Wikicommons Image Credit: Public domain Food, Energy, & Broadband & Secure and Risk & Health & Wellbeing Water Systems Universal Trustworthy Resilience Connectivity Cyberspace
Economic impact of CISE: From Federally- funded research to billion dollar industries 2010 Advances in computing, communications and information technologies, 2000 and cyberinfrastructure: • drive U.S. competiveness, (e.g., IT accounts for 25% of economic 1990 growth since 1995), and Product • have profound impacts on our ($B,$10B) daily lives. 1980 Industry 1970 University From Continuing Innovation in Information Technology , NRC, 2012.
Virtual Reality/Augmented Reality Visioning Workshop – July 2017 • Workshop with 40+ academic, industry and government experts to identify the scientific gaps in creating personalized and adaptive VR/AR systems
Workshop Goals • To identify the research needed to achieve individual personalization and adaptation in the areas of visual, auditory and tactile perception, as well as the social, behavioral and cognitive patterns key to adaptation – Perceptual Systems – Social, behavioral and cognitive patterns
USE CASE: Training for First Responders Jessie manages training and professional development for first responders. Thanks to new VR technologies, it has become relatively easy for Jessie to port volumetric models of real world settings into VR to build new training scenarios. Also, she can tweak the scenario settings for different training objectives. With earlier training systems, Jessie used to be frustrated because she couldn’t customize training scenarios for the trainees. This doesn’t result in effective training because her students enter the program with a wide variety of skill sets and experience. Specifically, she sees recruits employ different strategies - both effective and ineffective - when navigating complex, rapidly changing environments and would like a training solution that supports such nuances.
User-Centric Hierarchical Benchmarks • Hardware and software agnostic benchmarks should be developed for assessing the end-to-end user experience to determine how well systems support a user to complete a given task. Categories to include: – Detection – Navigation (locomotion) – Selection – Manipulation • Sample tasks to be included: – Visual acuity – Contrast sensitivity – Disparity acuity – Localization in space (visual and auditory) – 3D motion acuity – Hand tracking – Distance estimation – Pointing – Interception
Optimizing the human-machine interface Optimization should have the goal of a natural interaction - interaction that extracts a low cognitive load and a high level of mastery, which is adapted and personalized for users and tasks. Optimization should also achieve all this and fade into the human’s background. • Optimization includes: – Different types of user interfaces – Different kinds of interaction tasks – Match between task and setting • Operation in the real world requires: – Intelligent sensing/modeling of the environment – Algorithms for virtual objects in the physical space; – Methods to specify flexible environments that adapt to the physical world; – Studies of how the physical environments result in different user experiences and acceptance. • Task-specific optimization – Identify aspects that foster learning • What are the universal aspects and what requires personalization?
Identifying User States & Traits Understanding the emotional and physiological states of users real-time is critical to developing and assessing adaptation and ensuring VR/AR usability. • Potential states and traits include: – Engagement, which can range from mild interest to the full concentration “flow” – Presence “the sense of “being there” – Frustration – Boredom – Confidence or self-efficacy – Tenacity • Internal states to detect/predict motion sickness
Recommend
More recommend
