Lindemans Lectures: Virtual Reality & Serious Games (Part 1) - PowerPoint PPT Presentation

Lindeman’s Lectures: Virtual Reality & Serious Games (Part 1) Robert W. Lindeman Assistant Professor Interactive Media & Game Development Human Interaction in Virtual Environments (HIVE) Lab Department of Computer Science Worcester Polytechnic Institute gogo@wpi.edu

Five-Lecture Structure  July 15  Introduction to Game Development  July 16  Game Design (part 1)  July 23  Game Design (part 2)  July 29  Virtual Reality / Serious Games  July 30  Future Gaming (Natural Interaction, MMOs, Mobile) R.W. Lindeman - WPI Dept. of Computer Science 2 Interactive Media & Game Development

Today’s Outline  What is Virtual Reality?  Why is it important?  What are Serious Games? R.W. Lindeman - WPI Dept. of Computer Science 3 Interactive Media & Game Development

Motivation  Much excitement (and hype) about how VR was going to change things  VR has not made inroads into everyday life  Lagging technology  Lack of understanding of usability issues  Lack of "killer app"  Still remains mainly in research labs  Video games show great promise  Training scenarios - surgery, military, therapy  Long-Term Goal  Make VR more usable R.W. Lindeman - WPI Dept. of Computer Science 4 Interactive Media & Game Development

What is Virtual Reality?  You tell me! R.W. Lindeman - WPI Dept. of Computer Science 5 Interactive Media & Game Development

Virtual Reality Systems 1929 – Link Flight Simulator  1946 – First computer (ENIAC)  1956 – Sensorama  1960 – Heileg’s HMD  1965-68 – The Ultimate Display  1972 – Pong  1973 – Evans & Sutherland Computer Corp.  1976 – Videoplace  1977 – Apple, Commodore, and Radio Shack PCs  1979 – First Data Glove [Sayre] (powerglove -89)  1981 – SGI founded  1985 – NASA AMES  1986-89 – Super Cockpit Program  1990s – Boom Displays  1992 – CAVE (at SIGGRAPH)  1995 – Workbench  1998 – Walking Experiment  R.W. Lindeman - WPI Dept. of Computer Science 6 Interactive Media & Game Development

Link Flight Simulator  1929 - Edward Link develops a mechanical flight simulator  Train in a synthetic environment  Used mechanical linkages  Instrument (blind) flying  http://www.wpafb.af.mil/ museum/early_years/ey1 9a.htm R.W. Lindeman - WPI Dept. of Computer Science 7 Interactive Media & Game Development

Sensorama Morton Heilig, 1956 Motorcycle simulator - all senses • visual (city scenes) • sound (engine, city sounds) • vibration (engine) • smell (exhaust, food) Extend the notion of a ‘movie’ R.W. Lindeman - WPI Dept. of Computer Science 8 Interactive Media & Game Development

Sensorama by Morton Heilig (1960) R.W. Lindeman - WPI Dept. of Computer Science 9 Interactive Media & Game Development

Heilig's HMD (1960) Simulation Mask from Heilig’s 1960 patent  3D photographic slides  WFOV optics with focus control  Stereo sound  Smell R.W. Lindeman - WPI Dept. of Computer Science 10 Interactive Media & Game Development

Ivan Sutherland  The Ultimate Display (FIPS 1965)  Data Visualization: “A display connected to a digital computer…is a looking glass into a mathematical wonderland.”  Body Tracking: “The computer can easily sense the positions of almost any of our body muscles.” R.W. Lindeman - WPI Dept. of Computer Science 11 Interactive Media & Game Development

Ultimate Display (cont.)  Virtual Environments that mimic real environments: “A chair display in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal.”  VEs that go beyond reality: “There is no reason why the objects displayed by a computer have to follow ordinary rules of physical reality with which we are familiar.” R.W. Lindeman - WPI Dept. of Computer Science 12 Interactive Media & Game Development

First HMD-Based VR 1965 - The Ultimate Display paper by Sutherland 1968 - Ian Sutherland’s HMD R.W. Lindeman - WPI Dept. of Computer Science 13 Interactive Media & Game Development

Molecular Docking Simulator  Incorporated force feedback  Visualize an abstract simulation R.W. Lindeman - WPI Dept. of Computer Science 14 Interactive Media & Game Development

Data Gloves  Light, electrical or metal detectors compute “bend”  Electrical sensors detect pinches.  Force feedback mechanical linkages R.W. Lindeman - WPI Dept. of Computer Science 15 Interactive Media & Game Development

1985 - NASA Ames HMD  McGreevy and and Humphries  Wearable immersive HMDs  LCD “Watchman” displays  LEEP Optics  Led to VIVID, led by Scott Fisher R.W. Lindeman - WPI Dept. of Computer Science 16 Interactive Media & Game Development

FakeSpace Boom Display: Early 1990s R.W. Lindeman - WPI Dept. of Computer Science 17 Interactive Media & Game Development

CAVE - 1992 R.W. Lindeman - WPI Dept. of Computer Science 18 Interactive Media & Game Development

Virtual Workbench-1995 (Responsive Workbench, Immersidesk, etc.) R.W. Lindeman - WPI Dept. of Computer Science 19 Interactive Media & Game Development

Current Best VE  UNC Pit Experiment  Fear of Heights a Strong Response  Thousands of visitors  Compelling Experience  Haptics  Low Latency  High Visual Quality R.W. Lindeman - WPI Dept. of Computer Science 20 Interactive Media & Game Development

VPL Founded - 1985  First VR Company  VPL Research by Jaron Lanier and Thomas Zimmerman  Data Glove  Term: Virtual Reality R.W. Lindeman - WPI Dept. of Computer Science 21 Interactive Media & Game Development

1995 - Effectiveness of computer-generated (VR) graded exposure in the treatment of acrophobia in American Journal of Psychiatry R.W. Lindeman - WPI Dept. of Computer Science 22 Interactive Media & Game Development

Major Reinvigoration: Hardware Evolution  High expense  PC performance surpasses Graphics supercomputers  SGI RealityEngine (300k tris – 1993)  XBox (150 mil tri/sec - 2001)  XBox360 (500 mil tri/sec - 2005)  Wii-mote  Large Volume Displays  VR Estimated $3.4 billion industry in 2005 R.W. Lindeman - WPI Dept. of Computer Science 23 Interactive Media & Game Development

Background  VR defined:  Fooling the senses into believing they are experiencing something they are not actually experiencing  Virtual reality systems consist of:  Graphical/audio/haptic/... rendering  Content  Tracking of people and objects  Collision detection  Interaction techniques  Optional, but common:  Networking  Autonomous agents R.W. Lindeman - WPI Dept. of Computer Science 24 Interactive Media & Game Development

Keys to Success  High fidelity (or realism)  Graphics, audio, haptics, behaviors, etc.  Low latency  Tracking  Collision detection  Rendering  Networking  Ease of use  Low cumber for users  Easy integration for programmers  Compelling Content R.W. Lindeman - WPI Dept. of Computer Science 25 Interactive Media & Game Development

The Senses  See (Visual Sense):  Visuals are excellent!  Hear (Aural Sense):  Spatialized audio is very good!  Smell (Olfactory Sense):  Very hard! Too many types of receptors.  Touch (Haptic Sense):  Application specific and cumbersome  Taste (Gustatory Sense):  We know the base tastes, but that is it! R.W. Lindeman - WPI Dept. of Computer Science 26 Interactive Media & Game Development

See: Head-Mounted Displays R.W. Lindeman - WPI Dept. of Computer Science 27 Interactive Media & Game Development

See: Projection-Based Environments R.W. Lindeman - WPI Dept. of Computer Science 28 Interactive Media & Game Development

See: Projection-Based Environments (cont.) R.W. Lindeman - WPI Dept. of Computer Science 29 Interactive Media & Game Development

Hear: Sound in VR  Display techniques  Multi-speaker output (sound cube)  Headphones  Bone-Conduction  Waveform filtering  Simple balance & volume control  Head-Related Transfer Functions  Software "Standards"  OpenAL  A3D from Aureal (RIP!)  VRSonic.com R.W. Lindeman - WPI Dept. of Computer Science 30 Interactive Media & Game Development

Smell: Olfactory Sense  Two main problems  Scent generation  Tens of thousands of receptor types  Scent delivery  Easier problem R.W. Lindeman - WPI Dept. of Computer Science 31 Interactive Media & Game Development

Smell: Air Cannon (Yanagida, 2004) CLIP R.W. Lindeman - WPI Dept. of Computer Science 32 Interactive Media & Game Development

Touch: Haptic Feedback in VR  Tactile: Surface properties  Most densely populated area is the fingertip (okay, it's the tongue)  Kinesthetic: Muscles, Tendons, etc.  Also known as proprioception R.W. Lindeman - WPI Dept. of Computer Science 33 Interactive Media & Game Development

Vibrotactile Feedback Projects CLIP TactaBoard and TactaVest R.W. Lindeman - WPI Dept. of Computer Science 34 Interactive Media & Game Development