Developing Mobile Multimedia Apps Meta-Media Mobile Applications - PDF document

Outline Developing Mobile Multimedia Apps • Meta-Media • Mobile Applications Part 1 � Botanic: A Case Study • Methodology � • Tools & Techniques Part 2 Luiz Velho IMPA • Botanic: A Case Study 1 2 A Revolution in Process • Informatics + Telecommunications New Media 
 • Digital Content <Meta-Media> • Interactive Interfaces • Virtual Communities • Big Data • Internet Services 3 4 Trends in Hardware Software Perspectives • Processing • Interoperability – Parallelism – Standards • Memory • Distributed Computing – Unlimited – Agents • Network • Smart Interfaces – Pervasive – Avatars • Data (I/O) • Augmented Reality – High Fidelity – Immersion 5 6

Product Directions Meta-Media • Ubiquity Mem. E/S – Various Kinds Proc. • Portability – Miniaturization Mem. E/S Proc. • Connectivity – Multimodal Mem. E/S Proc. • Design … Network … – Form and Function 7 8 Challenges • Integration – Technologies / Data / Areas • New Paradigms Mobile Applications – Computation / Languages / Interface • Applications – Personal Satisfaction – Social and Cultural Insertion – Business Models 9 10 What's Different Now? Specifics of Mobile Apps • Portable Phone / Tablet / Watch / … - • Networked • Desktop Cellular / Wifi / Bluetooth - • Web • Sensors • Mobile - GPS / Accelerometer / Compass Ecosystem • Media - Cameras / Audio / Display 11 12

A New Platform Interactive Media Devices • Gesture / Sketch Interaction • Ubiquitous - Outdoors / Indoors / Vehicle • Real-Time 3D Graphics • Natural Interface • Pictures / Movies - Multitouch / Proximity • Music / Podcast • High Quality A/V - Stereo Sound / HD Video / Megapixel Photo • Rich Text 13 14 Graphics & Vision App Types • Many… • Games • Augmented / Virtual Reality • Computational Photography • Media Authoring / Performance • Social Networks 15 16 Our Focus The Botanic App Themes • Tom Jobim and Botanical Garden • Art / Entertainment • Navigation / Travel • Music / Photo • Social 17 18

Methodology The Real Picture App Lifecycle • Feedback Loop • Design Make • Development • Testing Idea • Deployment Learn Show • Maintenance case study… 19 20 System Platform • Hardware / Software - Apple / iOS Tools and Techniques • Form-Factor - iPhone / iPad Mini • Capabilities - Navigation / Media / Interaction 21 22 Development Environment Xcode • IDE - Xcode / Interface Builder • Language - Objective-C / C++ • Revision System - Git 23 24

Interface Builder Objective-C storyboard widgets • object-oriented language (Smalltalk + C) 25 26 Design Patterns Git • revision control system • Model-View-Controller • Target-Action • Delegation 27 28 Frameworks Cocoa Touch Layer • Layers of iOS Technologies • UIKit • MapKit • TextKit 29 30

Media Layer Core Services • AV Foundation • Core Foundation • Media Player • Core Data • Core Location • OpenGL ES • OpenAL • Core Motion 31 32 iOS SDK Production Tools • Xcode + iOS Simulator • Image Photoshop - • Video Final Cut X - • Animation After Effects - • Etc…. 33 34 Genesis • The Book Botanic 35 36

Motivation The Book • Meu Querido Jardim Jardim Botânico • Research new media - • Experimentation mobile platforms - • Practice real apps - (* innovation cycle) poetry photographs 37 38 Why What • Macro / Micro Navigation • Information - Jardim Botânico - Visitor’s Guide • Rich Media Content • Navigation Tool Espaço Tom Jobim - - Map / Tours • Data Collection - Photos / Notes 39 40 How • Design Choices - iOS / iPhone Case Study • Functionality Information / Exploration / Sharing - • Multimedia Content Poetry / Music / Photos - 41 42

The Creation Process 2013 2014 • Design • Research • Development • Testing • Deployment • Maintenance 43 44 Design Concept • Inspire / Explore / Share • Concept • Background Research • UI Design • Visual Style 45 46 Background.. User Interface - I • Related Apps (circa Feb. 2013) • Basic Structure Kew Gardens - London • Flow Garden Guide of Chicago tab controller navigation controller Memphis Botanic Garden … progressively less interface mostly just information! 47 48

User Interface - II User Interface - III • Complete Structure • Element Design Evolution (ex: Map View) 49 50 Research Visual Style • iOS 6 • Maps • Augmented Reality • iOS 7 • 360° Panoramas • Media & Animation 51 52 Maps - I Maps - II • No available GIS data! • Construction Steps main alleys zones buildings and sites Hardcopy from JB Google Maps 53 54

Augmented Reality - I Augmented Reality - II • Software Architecture • Labels on Screen search 
 UIKit parameters Site Data (camera) labels drawing Sensor location OpenGL callbacks orientation buckets 55 56 360° Panoramas - I 360° Panoramas - II • Representation and Rendering • Interactive Augmented Reality active labels Central Fountain Tom Monument texture sphere virtual camera (equiretangular image) (polygonal mesh) contents (OpenGL) enhanced hyperlink scene 3D sound OpenAL 57 58 Animation - I Animation - II (example) audio video • “Uber Media” Engine A1 keyframe 1 Sequence of Image Based Animations 
 - (Ken Burn’s Effect) T1,2 Transition Between Animations - - Synchronized Audio A2 • Simple Authoring Language T2,3 - Key-Frame based keyframe 2 A3 59 60

Development Content Production • Curatorship Selection from the Book • Planning / Integration • Content Production • Implementation poems photos 61 62 Testing Implementation • Apple Frameworks • Other Frameworks UIKit Open Flow - - MapKit F-Gallery - - • Internal Evaluation - Core Location Core Motion - • Ad Hoc Tests Core Data - - OpenGL • Validation OpenAL - - AV Foundation 63 64 Deployment Ad-Hoc Test • In-App Feedback + Google Web Form • Distribution • Marketing • Advertising • Press 65 66

App Store Distribution Marketing • Apple Developer + iOS University Program • App Store + Website and Video 67 68 Advertising Press • Poster / Flyer downloads Sept / Oct 2013 web print 69 70 Maintenance Mediatic Tours Inspired by Tom Jobim • Revisions • Tom’s Morning Stroll version 2.6 - Multimedia - • Follow-up • Touring the Garden with Tom Jobim New Content - Video - 71 72

Demo Video 73 74 Future Directions Acknowledgments • Institutional Content • Data Management • Content Authoring • Social Network 75 76 Questions ? 77