Immersive Telepresence in in Immersive Telepresence Immersive - PowerPoint PPT Presentation

Immersive Telepresence in in Immersive Telepresence Immersive Telepresence in Responsive Virtual Environments Environments Responsive Virtual Responsive Virtual Environments Martin Göbel GMD - German National Research Center for Information Technolgy IMK - Institute for MediaCommunication goebel@gmd.de http://imk.gmd.de/docs/ww/ve/ DFN Jahrestagung, Berlin 10.3.99 1

GMD is is .... VMSD .... VMSD is is ... ... GMD GMD is .... VMSD is ... ❏ Geman National Research ❏ GMD´s high end Center for Information graphics divison Technology ❏ ~25 scientist & ❏ 1200 employees ~15 students ❏ ~100M$ budget ❏ runnig Onyx´s, cave-like CyberStage ❏ 1/3 project funding 2-sided Workbenches ❏ 8 institutes: (near) BONN, wall projections BERLIN, DARMSTADT BOOM, virtual studio audio lab ❏ Together with TAN - HoloBench - cavelike CUBE - Tanorama DFN Jahrestagung, Berlin 10.3.99 2

brief VR ( VR (device device) ) History History brief brief VR (device) History ❏ 88-90 NASA AMES ❏ 90-92 pioneers in Europe: UK, Germany ❏ 92/93 Responsive Workbench (GMD), CAVE (EVL) ❏ 93-97 demo centers in Germany ❏ 94- EUROPEAN project funding ❏ 95- industrial interest (automotive) ❏ 96- many more CAVEs & Benches ❏ 97- industry caves ❏ 98 2-sided Responsive Workbench DFN Jahrestagung, Berlin 10.3.99 3

Outline Outline Outline ❏ Projective Virtual Environments ❏ TeleConferencing - TelePresence ❏ Immersive TelePresence Trials ❏ Conclusions DFN Jahrestagung, Berlin 10.3.99 4

The Responsive Workbench 1993 The Responsive Workbench 1993 The Responsive Workbench TM 1993 TM TM ❏ by Wolfgang Krüger ❏ table top display: desk - table - workbench ❏ horizontal plane: 1.8*1.3m ❏ GMD meanwhile version 4: flat, 10°, dual sided ❏ NOW: product by many companies eyes screen mirror projector DFN Jahrestagung, Berlin 10.3.99 5

The CyberStage (1996) (1996) The CyberStage The CyberStage (1996) ❏ CAVE UIC: Cruz Neira et al 92/93 Operation Theatre Prototyping ❏ cave-like room size display ❏ 8 channel spatial audio ❏ acoustic floor side view side view Tracker VW Sharan Interior Design mirrors screens data VW acoustic floor MDC DFN Jahrestagung, Berlin 10.3.99 6

The 2- 2-sided sided Resp Resp. . Workbench Workbench (1998) (1998) The The 2-sided Resp. Workbench (1998) ❏ THE VE display for interactive applications ❏ extended viewing frustrum ❏ head tracked (as others) ❏ 2-8 channel audio screens eyes large interaction space (even for 2 viewers) collision free 3D interaction within frustrum mirror mirror dual hand interaction DFN Jahrestagung, Berlin 10.3.99 7

Compatibility by Compatibility by AVOCADO System AVOCADO System Compatibility by AVOCADO System ❏ since 4/96; beta 1/99 Application Programming ❏ C++ VE framework AVOCADO ❏ OGL, Performer Performer Scheme ❏ scripting in Scheme IRIX 6.5 ❏ modular, extendible ❏ scene graph API ❏ distribution by active object replication and IP multi casting ❏ configurable to sensores and services (IO) DFN Jahrestagung, Berlin 10.3.99 8

Application Structure Application Structure Application Structure Sensor Node Field Field Node Reference Field Field Field Field Connection Sensor Node Node Field Field Field Field Field Field Field Field Service Service Service Service DFN Jahrestagung, Berlin 10.3.99 9

Applications under Investigation Applications under Investigation Applications under Investigation ❏ Engineering Visualization ❏ Medical Visualization ❏ GeoScience Visualization ❏ Architecture, interior Design ❏ Mathematics ❏ Molecular Design ❏ Education, Entertainment, Story Telling ❏ ... DFN Jahrestagung, Berlin 10.3.99 10

VR Application development Application development VR VR Application development ❏ Presentation visual/ audible / haptic rendering ❏ Navigation point&fly / head tracking / physical probs ❏ Interaction gloves / pointing devices / task specific tools ❏ Networking single site / two users / teams & communities ❏ Communication shared images / audio / video / guestures ❏ Towards Collaboration in virtual teams DFN Jahrestagung, Berlin 10.3.99 11

TeleConferencing ... ... TeleConferencing TeleConferencing ... ❏ Most real-world applications require communi- cation and - if possible - collaboration between geographically distributed teams ❏ good practice: --> pictureTel, shared X applications, … ❏ what is urgently needed: large screen interactive stereo visualization + video-conferencing for synchronous, symmetric and asymmetric collaboration DFN Jahrestagung, Berlin 10.3.99 12

Teleport Teleport (1995) (1995) Display rooms with wall size Video wall Merging of real and virtual environments Chroma keying or image segmentation Head tracking Stereo video textures and surround audio Video texture via ATM (D1: 270mbs to 32mbs) DFN Jahrestagung, Berlin 10.3.99 13

Teleteaching Teleteaching Teleteaching ❏ operating between GMD and University Geneva ❏ bidirectional video + audio ❏ documents as video but also via internet available U Geneva Session GMD Session DFN Jahrestagung, Berlin 10.3.99 14

connecting connecting CAVE to RWB (1997) CAVE to RWB (1997) connecting CAVE to RWB (1997) ❏ EUROPEAN ACTS Project for distributed Video Production (DVP) ❏ task with NCSA Urbana, Caterpillar ❏ 97 - 98; on 34mbs ATM (also SC98, Orlando) ❏ aim: - connect NCSA CAVE to GMD RWB via transatlantic ATM - heterogenous VE systems; usability study - shared industrial application by Caterpillar - audiovisual communication DFN Jahrestagung, Berlin 10.3.99 15

NCSA-GMD Caterpillar scenario Caterpillar scenario NCSA-GMD NCSA-GMD Caterpillar scenario Responsive Workbench CAVE Driver at NCSA Supervisor Truck Driver Supervisor at GMD Cat Driver Videotextures Visible image at Workbench DFN Jahrestagung, Berlin 10.3.99 16

Immersive TelePresence trial (´97) (´97) Immersive TelePresence trial Immersive TelePresence trial (´97) Blue Room CyberStage virtually present remote Audio &Video simulate the communication and collaboration between two projective VR systems DFN Jahrestagung, Berlin 10.3.99 17

Immersive Telepresence Installation Installation Immersive Telepresence Immersive Telepresence Installation SP SP Blue Room Cavelike CyberStage Monitor SP Stereo SP SP Camera L R Audio Infra-red Mixer Audio Mixer Camera Sirius 1 2 Framestore Sirius 1 Onyx 2-pipe RE Onyx 2-pipe IR Ultimatte 2 DFN Jahrestagung, Berlin 10.3.99 18

Immersive Telepresence Immersive Telepresence Immersive Telepresence Teaching and training within an immersive virtual environment: ❏ Expert/student roles ❏ Stereo-video integration into virtual world ❏ Interaction and communication in a Virtual Environment ❏ Asymmetric blue room studio - cavelike scenario DFN Jahrestagung, Berlin 10.3.99 19

Immersive Telepresence Immersive Telepresence Immersive Telepresence ❏ symmetric solutions in caves??? Many open problems … camera position, light situation, unichrome for keying, ... ❏ start with 2-sided RWB - fixed camera ... field of cameras - IR stereo camera - blue wall ❏ at least 155mbs ATM for bidirectional av- stereo; additional video sources cost extra! DFN Jahrestagung, Berlin 10.3.99 20

Cooperative Responsive Workbench Cooperative Responsive Workbench Cooperative Responsive Workbench ATM OC12 : ISDN 8 So 2-sided RWB at Site A 2-sided RWB at Site B Evtl. More Sites DFN Jahrestagung, Berlin 10.3.99 21

Immersive Telepresence Telepresence Immersive Immersive Telepresence Collaboration in distributed immersive virtual environment ❏ Equal roles ❏ Stereo-video integration into virtual world ❏ Interaction and communication in a Virtual Environment ❏ Symmetric Scenario DFN Jahrestagung, Berlin 10.3.99 22

Video Video Video DFN Jahrestagung, Berlin 10.3.99 23

Conclusions & & further Research further Research Conclusions Conclusions & further Research ❏ 1:1 cRWB with reasonable depth recognition ❏ moving team members (out of camera field)?? ❏ preservation of parallax and reduction of distortion at any? Time?? ❏ Collaborative interaction ❏ N:m extension is application dependent ❏ Re-extension to CAVE … we would like, BUT less industrial interest and much more difficult. DFN Jahrestagung, Berlin 10.3.99 24

Acknowledging …. …. Acknowledging Acknowledging …. ❏ Dr Stephen Barras ❏ Jonny Plate ❏ Jeremy Eccles ❏ Michael Tirtasana ❏ Dr Gerhard Eckel ❏ Henrik Tramberend ❏ Dr Bernd Fröhlich ❏ Gerold Wesche ❏ Dr Simon Gibbs ❏ Jürgen Wind ❏ Gernot Goebbels ❏ Björn Zehner ❏ Frank Hasenbrink ❏ Berthold Kirsch ❏ Dr Vali Lalioti ❏ Uli Lechner http://imk.gmd.de/docs/ww/ve/ DFN Jahrestagung, Berlin 10.3.99 25