An open source framework for graphical related tasks Clment Forest - PowerPoint PPT Presentation

An open source framework for graphical related tasks Clément Forest Digital-Trainers www.digital-trainers.com

VgSDK in short

External dependencies

Supported hardware

Base principles The user describes the objects in a scene graph The engine manages and optimises all the low level GPU related tasks

Base principles ● Architecture similar to aspect-based engine architecture in 'GPU Pro 4'. The world is described as an The engine exposes a state oriented direct acyclic graph and update it during the (DAG) graph traversal Root Specialised components ( aspects ) use that state to do Setup Scene their respective tasks (OpenGL rendering, scene Camera Object 1 Object 2 Drawstyle manipulation, bounding boxes computation, ...) Material Material Geometry Geometry

Abstraction of the 3D API ● OpenGL is one aspect among others (vgeGL) vgeDirectX, vgeOpenGLES, vgeMantle could be implemented in ● the same way. It is not necessary to implement the totality of the nodes in a given ● aspect.

Modularity ● Most of the external dependencies are wrapped and optional. core vgBase, vgd, vgDebug, vgm images io engines ui algorithms vgCairo, vgFFmpeg vgio, vgPhysfs, vge, vgBullet, vgUI, vgGTK, vgQt, vgAlg vgITK vgOpenAssetImport, vgeGL vgSDL, vgWx vgTrian

Modularity 2014 2010 2008 2004 2003

Functionalities Usually provided by a node in the scene graph. ● Displacement ● Offscreen rendering

Functionalities ● Tesselation Adaptative and uniform Phong subdivision ● ● Stereoscopy (prototype)

Functionalities ● Image processing (basic filters) Arithmetics, resize, blur, edge detection, ... ● ● Complex filters antialiasing FXAA, depth of field, noise, ... ●

Functionalities ● Fluid simulation

Shaders ● Can be defined in a specialised node vertex, geometry, tesselation control, tessellation evaluation, ● fragment shaders. Rapid prototyping, ● But hard to maintain (new shader for every combination of light ● and of materials, ...). Shp< Program > program = Program::create( "myShader" ); program->setVertex( "....." ); program->setFragment( "....." );

Shaders ● Can be automatically generated by the engine The shaders are only created on demand (lazy evaluation) ● The combinatorial explosion is manage by the engine ● It is easy to add new functionalities ● A cache system limits unnecessary recompilations ●

Example of a custom shader

Binding Python ● The nodes and the graphical canvas are exposed in Python via Swig ● Swig can also be used to enable other programming languages (C#, java, Lua, javaScript) Eg: loading of a scene and modification of the scene graph using ● a Python file.

I/O 3D data: dae, Blender, 3DSMax or Wavefront files ● Images: png, openexr, jpeg, psd, targa, tiff. ● Image and sound extraction from videos with FFmpeg. ● Loading and display of polices using cairo. ● Data encryption (blowfish). ●

Examples of uses (Ircad) Patient reconstruction Negatoscope 2D and 3D

Examples of uses (Ircad) Ultrasound simulation Surgical planning

Viewer Qt ● Scene graph editor ● Inspection and modification of the automatics shaders. ● Image and video capture ● Camera information capture

Viewer Sofa ● Simulation and rendering in two different threads ● Synchronisation of the visual scene on the simulation ● Visual mappings done in the rendering thread

Laparoscopic simulator

Contact ● Clement.Forest@digital-trainers.com ● https://code.google.com/p/vgsdk/ ● Liste de diffusion ● Source, Binairies, SDK, dependancies, ....

Scene description ● Data are stored in nodes ● Nodes are organised hierarchically in a scene graph (DAG) ● Scene nodes are attached to each other and follow each others movements

Node types Groups to construct a scene ● graph (Switch, Separator, GeoMorph ...) Shapes to define 3d objects ● (VertexShape, Box, Quad, Grid, ...) Attributes to set materials, ● lights, post-processing, rigid transformations and so on Kits to build higher level nodes ● by hiding a sub-scene graph (Dragger, ...)

Creating Nodes ● Creating node is done using static factories vgd::Shp< vgd::node::Material> material = vgd::node::Material::create("redMaterial"); vgd::Shp<> is an alias of boost::shared_ptr<> ●

Scene graph ● Creating a group node vgd::Shp< Group > group = Group::create( "rootOfScene" ); ● Adding a child group->addChild( material ) Or group->insertChild( material ) ● Access to the first child vgd::Shp< Material > m = group->getChild<Material>(0);

Canvas ● Search node result = findFirstByType<VertexShape>(); result = findFirstByRegex("tumor.*"); ● Canvas refresh refresh( REFRESH_IF_NEEDED, ASYNCHRONOUS ); refresh( REFRESH_FORCED, SYNCHRONOUS ); ● Ray casting hitNode = castRay(xMousePosition, yMousePosition); HotNode = castRay(raySource, rayDirection, outputHitTriangle)

Lights ● Create a directional light light1 = DirectionalLight::create("light1"); ● Switch on/off the light light1->setOn( false ); ● Set the direction of the light light1->setDirection( vgm::Vec3f(0,0, -1) );

Lights (continuation) ● Create a second light light2 = SpotLight::create("light2", 1/*light index*/); ● Change properties of the second light light2->setOn(true); light2->setPosition( vgm::Vec3f(0,0, 5) ); light2->setDirection( vgm::Vec3f(0,0, -1) ) light2->setCastShadow( true );

Fields ● Nodes store their data in fields. ● Fields are all dynamic node->addField( newField ); ● Introspection Retrieve all field names node->getFieldNames() ● Access to a field node->getField( FieldName ); ●

Fields (continuation) ● Access control Read only access to a field ● EditorRO<FieldType > editorRO = getFieldRO(name) Read write access to a field ● EditorRW<FieldType > editorRW = getFieldRW(name) Design pattern observer applied to field access. ●

Loading and modifying meshes ● Loading a mesh vgOpenAssetImport::Loader loader; Shp< VertexShape > shape = loader.load( filename ); ● Accessing its positions and primitives EditorRW< TMultiField< Vec3f > > editor_pos = shape->getVertexRW(); EditorRW< TMultiField< unsigned long > > editor_ind = shape->getVertexIndexRW(); EditorRW< TMultiField< Primitive > > editor_pri = shape->getPrimitiveRW();

Adaptive Phong Tessellation // Enables phong tessellation tessProp->setTessellation( TessellationProperties::PHONG ); // Specifies the minimum (1) and maximum (32) tessellation level used by the tessellation control shaders to tessellate the incoming primitive. tessProp->setRange( vgm::Vec2f(1.f, 32.f) ); // When tessellation method is set to PHONG and TessellationLevel.method==PIXELS_PER_EDGE, then this field specifies the desired number of pixels per edge tessProp->setPixelsPerEdge( 15.f ); // Chooses adaptive tessellation using the number of pixels per edge desired as criterion tessLevel->setMethod( TessellationLevel::PIXELS_PER_EDGE );