Artimate : an articulatory animation framework for audiovisual - PowerPoint PPT Presentation

Introduction Modelling Architecture Conclusion Artimate : an articulatory animation framework for audiovisual speech synthesis Ingmar Steiner and Slim Ouni Speech Group Dublin, March 9, 2012

Introduction Modelling Architecture Conclusion Motion Capture Face/Body Performance

Introduction Modelling Architecture Conclusion Motion Capture Face/Body Performance source: photobucket.com

Introduction Modelling Architecture Conclusion Motion Capture Face/Body Performance source: photobucket.com source: tumblr.com

Introduction Modelling Architecture Conclusion Motion Capture Speech

Introduction Modelling Architecture Conclusion Motion Capture Speech source: turbosquid.com

Introduction Modelling Architecture Conclusion Motion Capture Speech? source: turbosquid.com

Introduction Modelling Architecture Conclusion Model Assembly Static model & EMA

Introduction Modelling Architecture Conclusion Model Assembly Rigging

Introduction Modelling Architecture Conclusion Model Animation Skeleton

Introduction Modelling Architecture Conclusion Model Animation Surface

ven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . . A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . m a ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction Modelling Architecture Conclusion Architecture

ven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . . A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . m a ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler

ven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . . A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . m a ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler Artimate Core

ven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . . A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . m a ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler Artimate Core Artimate Demo App

ven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . . A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . m a ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler Artimate Core Artimate Demo App

ven m a Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . A . . . . . . . Artimate Core . . . . . . . . . . . ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . Artimate Demo App

ven m a Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . A . . . . . . . Artimate Core . . . . . . . . . . . ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . Artimate Demo App

ven m a Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . A . . . . . . . Artimate Core . . . . . . . . . . . ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . Artimate Demo App

ven m a Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . A . . . . . . . Artimate Core . . . . . . . . . . . ext o peech . R . Y . . . . . . . . . . . . . . . . . . . . Artimate Demo App

Introduction Modelling Architecture Conclusion Architecture Artimate Model Compiler ven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M . A . . . . . . . Artimate Core . . . . . . . . . . . m a ext o peech R . . Y . . . . . . . . . . . . . . . . . . . . Artimate Demo App

Introduction Modelling Architecture Conclusion Summary • Adapted skeletal animation for articulatory animation

Introduction Modelling Architecture Conclusion Summary • Adapted skeletal animation for articulatory animation • Motion capture data (EMA)

Introduction Modelling Architecture Conclusion Summary • Adapted skeletal animation for articulatory animation • Motion capture data (EMA) • Modular, cross-platform implementation with open-source dependencies

Introduction Modelling Architecture Conclusion Summary • Adapted skeletal animation for articulatory animation • Motion capture data (EMA) • Modular, cross-platform implementation with open-source dependencies • Self-contained, portable resource

Introduction Modelling Architecture Conclusion Further work • Handle noisy EMA data

Introduction Modelling Architecture Conclusion Further work • Handle noisy EMA data • Testing (validation, portability, integration, . . . )

Introduction Modelling Architecture Conclusion Further work • Handle noisy EMA data • Testing (validation, portability, integration, . . . ) • Documentation

Introduction Modelling Architecture Conclusion Further work • Handle noisy EMA data • Testing (validation, portability, integration, . . . ) • Documentation • Public release