Do you wish you could hear the audio and read the transcription of - PowerPoint PPT Presentation

Do you wish you could hear the audio and read the transcription of this session? Then come to JavaOne SM Online where this session is available in a multimedia tool with full audio and transcription synced with the slide presentation. JavaOne Online offers much more than just multimedia sessions. Here are just a few benefits: • 2003 and 2002 Multimedia JavaOne conference sessions • Monthly webinars with industry luminaries • Exclusive web-only multimedia sessions on Java technology • Birds-of-a-Feather sessions online • Classified Ads: Find a new job, view upcoming events, buy or sell cool stuff and much more! • Feature articles on industry leaders, Q&A with speakers, etc. For only $99.95, you can become a member of JavaOne Online for one year. Join today! Visit http://java.sun.com/javaone/online for more details!

Compelling Graphics for Mobile Applications using the Mobile 3D Graphics API (JSR-184) Jyri Huopaniemi Nokia Research Center Mark Patel Motorola Kari Pulli Nokia Mobile Phones | JavaOne 2003 | Session 2472

Overall Presentation Goal Learn about the Mobile 3D Graphics API design concepts, technical solutions, and related standards 2 | JavaOne 2003 | Session 2472

Speaker’s Qualifications • Dr. Jyri Huopaniemi is Senior Research Manager at Nokia Research Center, and specification lead of JSR-135 (Mobile Media API) and JSR-184 (Mobile 3D Graphics API) JCP standards • Mark Patel is a Principal Staff Engineer responsible for J2ME™ graphics architecture at Motorola • Dr. Kari Pulli heads Nokia Mobile Phone’s graphics research and represents Nokia both at JSR-184 and Khronos OpenGL ES 3 | JavaOne 2003 | Session 2472

Why 3D Graphics? Phone screens may be flat… but the world certainly isn’t! 4 | JavaOne 2003 | Session 2472

Applications for Mobile 3-D • Games! • User interfaces • Screen savers • Character animation • Maps • Visualization 5 | JavaOne 2003 | Session 2472

Presentation Agenda • Introduction and Background • Mobile 3D API overview • File Format • Integration with MIDP • Summary • Q&A 6 | JavaOne 2003 | Session 2472

Introduction and Overview | JavaOne 2003 | Session 2472

What Is JSR-184? • JSR-184 = Mobile 3D Graphics API for J2ME™ • The first Mobile Java™ 3D graphics standard • Optional package for J2ME™, to be used with MIDP and CLDC 1.1 • Defines a high-level and low-level interface for bringing 3-D graphics to MIDP 8 | JavaOne 2003 | Session 2472

JSR-184 Expert Group Nokia (specification lead) Intergrafx • • Aplix MathEngine • • ARM Motorola • • Bandai Research In Motion • • Cellon France Siemens • • Cingular Wireless Sony Ericsson • • Fathammer Sumea • • France Telecom Sun Microsystems • • Fuetrek Superscape • • HI Corporation Symbian • • Hybrid Graphics Texas Instruments • • In-Fusio 3d4W • • Insignia Solutions Vodafone • • Intel Zucotto Wireless • • 9 | JavaOne 2003 | Session 2472

JSR-184 API Requirements • Must haves: – Retained mode access (scene graph) – Immediate mode access (OpenGL subset or similar) – Possible to use both modes in a unified way – No optional parts in this JSR – Importers for various data types (e.g., meshes, scene graphs) – Minimal garbage collection – Interoperable with other related Java APIs • Should haves: – Implementable within 150 KB on a real mobile terminal 10 | JavaOne 2003 | Session 2472

JSR-184 Schedule • Spring 2002—First discussions • April 2002—Filing of JSR • June 2002—1st Expert Group meeting • Feb 2003—Community review • May 2003—Public review • July 2003—Proposed final draft 11 | JavaOne 2003 | Session 2472

The API Structure | JavaOne 2003 | Session 2472

Immediate vs. Retained Mode • Retained mode – Based on a scene graph • Rooted by a World • With a Camera, Lights, objects, etc. – First: update animation tracks – Then: tell World to render itself • Immediate mode – Set current camera, lights, blending modes, … – Then draw geometry – Similar to OpenGL state machine 13 | JavaOne 2003 | Session 2472

Scene Graphs Are Made of Nodes • Camera – Perspective/orthographic • Group – For organizing things – World, the root of the world, with an active camera • Light – Point, directional, spot, ambient • Mesh – Geometry + appearance • Sprite – Fast-to-draw 2D images 14 | JavaOne 2003 | Session 2472

Inherited Node Properties • Relative transformation (w.r.t parent, cumulative) – TxRxSxM • First a generic 4x4 projective matrix M • Then scale, rotate, and translate (easier to animate) • Additional optional transform by aligning to – Another node; current camera; pick ray – Replaces R: TxAxSxM • Inherited alpha factor – For per-node transparency effects 15 | JavaOne 2003 | Session 2472

Mesh • Mesh is 3D geometry (with associated Appearances) – Polygonal surfaces, made of triangle strips – VertexBuffer (location, normal, color, …) • Composed of submeshes, made of – IndexBuffer (triangle strip indices) – Appearance (material, texture, …) • Submeshes are rendered in the order of ascending Appearance layers – Opaque submeshes rendered before transparent ones on a layer 16 | JavaOne 2003 | Session 2472

MorphingMesh • Taking a sum of differences from base mesh allows arbitrary combination of partial animations, including exaggerations 17 | JavaOne 2003 | Session 2472

MorphingMesh • Several VertexBuffers – Base • Base attributes and non-morphed attributes (attributes: location, color, texcoord, …) – Several morph targets • Outcome – A weighted sum of base and differences ( ) ‡” = + R B w T - B i i i 18 | JavaOne 2003 | Session 2472

SkinnedMesh • A skeletally animated Mesh • Without skinning, you get cracks; you can also smoothly interpolate the joints 19 | JavaOne 2003 | Session 2472

SkinnedMesh • Details… – Nodes (bones) attach to groups of vertices – Vertices can have multiple controlling Nodes • The weights can vary – A simple skeleton can be used to animate a complex mesh • Requires less storage than mesh morphing (and much less than vertex animation) • The skin – Comes from connecting the vertices 20 | JavaOne 2003 | Session 2472

Appearance • Appearance has five component classes – GeometryMode – CompositingMode – Material – Texture2D – Fog 21 | JavaOne 2003 | Session 2472

Appearance Modes • GeometryMode – Culling and shading (1–2 sided, smooth/flat, …) – Hints for local viewer (light), persp. correction • CompositingMode – Alpha blend/test, depth test/write/offset, color write • Material – Vertex color tracking (ambient and diffuse) – Ambient, diffuse, specular, emissive, shininess 22 | JavaOne 2003 | Session 2472

Rest of Appearance • Texture2D – Input the actual image – Filtering (mipmap, point/linear) – Wrapping (clamp/repeat) – Modes (add, blend, decal, modulate, replace) • Fog 23 | JavaOne 2003 | Session 2472

Animation: Keyframes • Traditional cartoon process a’la Disney – Master animators draw the main characters and their poses in keyframes – Junior animators draw the frames between the keyframes (in-betweens) • KeyframeSequence – Each keyframe is a value at a time instance • Scalar, vector – Various interpolation methods • Step, linear, spline for scalars, vectors • Slerp, squad for orientations (quaternions) 24 | JavaOne 2003 | Session 2472

AnimationController • Controls the sequence – Do-it-once (open) or looping (closed) – Translates world time into sequence time • With several animations for the same target – A weighted (normalized) average is taken • Can bundle up several sequences, control them together – Hands, feet, body swings -> walk 25 | JavaOne 2003 | Session 2472

AnimationTrack • AnimationTrack associates – KeyframeSequence and – AnimationController with – Animation target (animatable property) • Visibility, shininess, color, scale, … • Each Object3D can have several AnimationTracks, even for the same target 26 | JavaOne 2003 | Session 2472

Searching in a Scene Graph • Picking – Group and Camera have pick methods – Returns a RayIntersection • The first Mesh and data about intersection point – Nodes can have a scopeID to set pick scope • BTW, scopeID can be used also for Lighting • Finding – Every Object3D has a property called userID – You can search for reachable objects with a matching userID 27 | JavaOne 2003 | Session 2472

Graphics3D • The way to do immediate mode rendering • Setup the rendering target – A MIDP Graphics object – Image2D • Draw to intersection of – Rendering target – Graphics3D object – Viewport • Set state, draw geometry, repeat 28 | JavaOne 2003 | Session 2472