Virtual Environments: System Architectures Anthony Steed Simon Julier Department of Computer Science University College London http://www.cs.ucl.ac.uk/teaching/VE
Outline • Problem Statement • Representing the Environment • User dynamics • Execution Models
Problem Statement • Problem Statement • Representing the Environment • User dynamics • Execution Models
Reminder - VE is an Immersive, Mediated Communication Medium User Synthetic User Environment Environment Interface Devices Mediated Medium Real Environment
Key Requirements of Systems • Speed of update – Especially in rendering and haptics • Latency – Time from tracker update to display change should be as short as possible (ideally <75ms) • Consistency – Environment state should be consistent with input • Expressivenees – Environment should respond to a range of user input
Modules and Responsibilities Graphics Rendering External Graphics Databases Scene-Graph Audio Master Audio Rendering Environment Scene-Graph Interaction Processing Haptic Input Devices Scene-Graph Haptic Rendering Network
Different Display Modes Have Different Requirements • Video (N copies – for stereo and multiple screens) – Maintain copy of visual state – Render as fast as possible (~60Hz) – Synchronise with other renders • Audio – Maintain copy of audio state – Render without glitches (requires fast interrupt) • Haptics – Maintain copy of haptic data Render as fast as possible (~1000Hz)
Representing the Environment • Problem Statement • Representing the Environment • Dynamics • Execution Models
Environment • Environment is a broad term, but what is it we are actually modelling? – Something that can be rendered and interacted with such that • We utilize capabilities of display system • Maximize the opportunity for interaction • Ellis states that VEs have 3 main components: – Content – Geometry – Dynamics
Contents • Environment is made up of discrete items known as objects and actors • Objects – Discrete and identifiable – Described by property vectors • Actors are objects that initiate interactions • The self is a special kind of actor with a point-of-view
Representing the Contents • Unfortunately little agreement about conventions, schema, specifications for describing environments – Relates to issues about defining and using ontologies – Standards where they exist usually focus on visual representation – Possibility that some standards will emerge • Well-known example is the Distributed Interactive Simulation (DIS) Entity Model
Motivation for DIS (SIMNET) • Born out of needs for large- scale military simulations: – Hundreds of different types of entities – Dozens of servers scattered throughout the world – Real-time – Man-in-the-loop • Complicated environments • Complicated interactions
DIS Environmental Model • World modelled as a set of entities – All entity locations available to all entities – All entities can serve as actors – All interactions between entities via events – Networking achieved using a mix of approaches: • Ground truth information } To be • State change information discussed in • Dead reckoning Week ~11 • Entities and events are described by their Protocol Data Units (PDUs)
IEEE Standards 1278.1-1995 & 1278.1a-1998 Representing Entities with DIS
Representing the Environment with DIS • Environments are considered to be object states which aren’t associated with a specific entity • Environmental states can come in several flavours: – Gridded data (e.g., terrain) – Point objects (e.g., trees) – Linear objects (e.g., roads) – Area objects (e.g., bogs)
Geometry • Description of the environmental field of action • Contains: – Dimensionality: The degree of freedom of the position vector – Metric: The basic mathematical rules for defining order, distance, etc. – Extent: The range of possible values of the position vector • Defines the “space” where the environment is described
Describing Environment Geometry • Typically Euclidean – simple (x, y, z); suitable for many applications • However not that straight forward – Euclidean not that useful for describing geometry on spheroids (e.g. planets) • Use a locally linear model (i.e. on a tangent plane) – Not terribly useful in large-scale collaborative environments (everyone wants to be at (0,0,0)) • Use a differential geometry model (i.e. everyone sets their own coordinates and connections between models have relative transforms)
Describing Object Geometry • Objects need to have a description in physical space • Implicit or assumed to be 3D Cartesian coordinates with a 1m unit scale usually • Described in two steps: – Describe the basic form of the environment • 3D models, usually polygonal, there are standards for this (VRML, DXF, OBJ) – Add properties to objects • Visual properties: colour, texture, shading, … • Sound properties: sources, reflectivity, … • Material properties: weight, elasticity, … • Semantic properties: (name, role, age, …) • No standards for this • Often implemented using a scene-graph
Graphs • A graph consists of vertices and edges • Vertices define the “state” information • Edges define “relationships” • Scene-graphs are directed and acyclic Arbitrary graph
Graphs • A graph consists of vertices and edges • Vertices define the “state” information • Edges define “relationships” • Scene-graphs are directed and acyclic Directed graph
Graphs • A graph consists of vertices and edges • Vertices define the “state” information • Edges define “relationships” • Scene-graphs are directed and acyclic Directed acyclic graph
Graphs • A graph consists of vertices and edges • Vertices define the “state” information • Edges define “relationships” • Scene-graphs are directed and acyclic Directed acyclic graph Arbitrary graph Directed graph
Scene-graphs Root node • In a scene-graph, vertices are often called nodes Group nodes – Store state information – Can include arbitrary property information • All graphs have a root node which defines the base of the Leaf nodes tree • All other nodes divided into two types: – Group nodes – Leaf Nodes
Group Nodes • Group nodes have multiple nodes as children – Child nodes can be other group nodes or leaf nodes • Applies common state information to multiple objects – State information propagates down the graph • Examples include: – Transformations – Switch nodes – Effects • Bump mapping, scribing, specular highlights
Bumpmapping Scribing Examples (OpenSceneGraph) Anisotropic Lighting Cartoon
Leaf Nodes • Leaf nodes cannot have children • State information relates to the appearance of specific objects • Examples include: – Geometry – Image based rendering • Billboards • Impostors
Billboards Examples (OpenSceneGraph) Impostors
Dynamics • These are the rules of interaction between the contents • These can be: – Differential equations of Newtonian dynamics to describe kinematic and dynamic relationships – Grammatical rules for pattern-matched triggered actions • Many different ways of doing this from imposing numerical approximations to Newtonian physics, through to plain old C++ / Java / XVR coding
Implementing Dynamics as Standalone Processes • Dynamics implemented as separate processes / threads • Can change state of the Dynamics graph in arbitrary ways – Change values of nodes – Add / remove nodes
Implementing Dynamics Within the Scene-Graph • Fairly “autonomous” dynamics can be achieved by embedding dynamics within the scenegraph Animation Node • Animations are group nodes which apply state changes to their children • Examples include: – Animation paths – Particle systems Animated nodes
Example Animation and Particle System
Generalised Dynamics: Application Nodes • Pre-defined behaviours allow lots of effects but are autonomous – Script nodes (e.g., in VRML) can be used to generalise behaviour • Most extreme example are application nodes : – Entire VR application is written as a group node in the scenegraph – Application contains certain resources (e.g., viewport to display graphics) – Application owns and manages all of the nodes beneath it – Unifies application and environment state • Capabilities include: – Multiple applications in same environment – Load balancing – Dynamic workgroup management
User Dynamics • Problem Statement • Representing the Environment • User dynamics • Execution Models
Managing Data from Input Devices • So far we’ve talked about objects and actors • However, the user actively participates in the environment as a type of actor • The way the user interfaces with the system is through the input devices
Recommend
More recommend
