LectureSight Automatic Camera Control in Lecture Recordings - PowerPoint PPT Presentation

LectureSight Automatic Camera Control in Lecture Recordings Benjamin Wulff Alexander Fecke bwulff@uos.de afecke@uos.de University of Osnabrück, Germany 03.04.2013

Introduction Technology Deployment Table of Contents 1 Introduction Project Overview Motivation 2 Technology System Architecture Camera Setup Video Analysis Camera Control 3 Deployment Scene Regions Scripting Advanced scenarios B. Wulff, A. Fecke LectureSight 2 / 20

Introduction Project Overview Technology Motivation Deployment Project Overview Context of the Project B. Wulff, A. Fecke LectureSight 3 / 20

Introduction Project Overview Technology Motivation Deployment Project Overview Context of the Project Opencast Community Project B. Wulff, A. Fecke LectureSight 3 / 20

Introduction Project Overview Technology Motivation Deployment Project Overview Context of the Project Opencast Community Project − → close integration with Opencast Matterhorn B. Wulff, A. Fecke LectureSight 3 / 20

Introduction Project Overview Technology Motivation Deployment Project Overview Context of the Project Opencast Community Project − → close integration with Opencast Matterhorn runs in same OSGI container (Apache Felix) camera control activated by recording schedule B. Wulff, A. Fecke LectureSight 3 / 20

Introduction Project Overview Technology Motivation Deployment Project Overview Context of the Project Opencast Community Project − → close integration with Opencast Matterhorn runs in same OSGI container (Apache Felix) camera control activated by recording schedule initiated by ETH Zürich ID Multimedia Services B. Wulff, A. Fecke LectureSight 3 / 20

Introduction Project Overview Technology Motivation Deployment Project Overview Context of the Project Opencast Community Project − → close integration with Opencast Matterhorn runs in same OSGI container (Apache Felix) camera control activated by recording schedule initiated by ETH Zürich ID Multimedia Services partner in development of Opencast Matterhorn replacing closed-source tracking solution B. Wulff, A. Fecke LectureSight 3 / 20

Introduction Project Overview Technology Motivation Deployment Motivation B. Wulff, A. Fecke LectureSight 4 / 20

Introduction Project Overview Technology Motivation Deployment Motivation to cut costs, most lectures are recorded automatically − → no camera operator B. Wulff, A. Fecke LectureSight 4 / 20

Introduction Project Overview Technology Motivation Deployment Motivation to cut costs, most lectures are recorded automatically − → no camera operator problem especially in big lecture halls B. Wulff, A. Fecke LectureSight 4 / 20

Introduction Project Overview Technology Motivation Deployment Motivation Static camera perspectives B. Wulff, A. Fecke LectureSight 5 / 20

Introduction Project Overview Technology Motivation Deployment Motivation Static camera perspectives 1 (cleverly chosen) wide shot B. Wulff, A. Fecke LectureSight 5 / 20

Introduction Project Overview Technology Motivation Deployment Motivation Static camera perspectives 1 (cleverly chosen) wide shot 2 close shot on the lectern B. Wulff, A. Fecke LectureSight 5 / 20

Introduction Project Overview Technology Motivation Deployment Motivation In a perfect world... Image source: Prof. Dr. Vornberger 2012, Algorithms, Session 1: Introduction B. Wulff, A. Fecke LectureSight 6 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Camera Setup B. Wulff, A. Fecke LectureSight 7 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Video Analysis Continuously maintained foreground mask B. Wulff, A. Fecke LectureSight 8 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Video Analysis Movie Time! B. Wulff, A. Fecke LectureSight 9 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Video Analysis Finding a person’s head B. Wulff, A. Fecke LectureSight 10 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Video Analysis Finding a person’s head P head : min ( | x centroid − x | ) ∧ max ( y ) B. Wulff, A. Fecke LectureSight 10 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Video Analysis Finding a person’s head P head : min ( | x centroid − x | ) ∧ max ( y ) K-means Clustering on person silhouette uneven number of prototypes cluster positions are updated with every frame B. Wulff, A. Fecke LectureSight 10 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Camera Control B. Wulff, A. Fecke LectureSight 11 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Camera Control PTZ camera is operated by a steering worker that decides about the camera movement: B. Wulff, A. Fecke LectureSight 11 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Camera Control PTZ camera is operated by a steering worker that decides about the camera movement: if correction moves are necessary B. Wulff, A. Fecke LectureSight 11 / 20

Introduction Camera Setup Technology Video Analysis Deployment Camera Control Camera Control PTZ camera is operated by a steering worker that decides about the camera movement: if correction moves are necessary which speeds for pan and tilt should be used B. Wulff, A. Fecke LectureSight 11 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios Customizing LectureSight Adjusting the system to the scene B. Wulff, A. Fecke LectureSight 12 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios Customizing LectureSight Adjusting the system to the scene B. Wulff, A. Fecke LectureSight 13 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios Customizing LectureSight A simple camera operator script 1 var target = false; 2 3 function init () { 4 home_pos = new Position( 5 Configuration .get("director.home.x"), 6 Configuration .get("director.home.y")); 7 } 8 9 function step () { 10 if (! target) { 11 Camera.setTarget(home_pos ); 12 target = findClosest(Tracker.trackedObjects , home_pos ); 13 } else { 14 if (target.isTracked ()) 15 Camera.setTarget(target.get("head.center")); 16 else if (Tracker. currentTime - target.lastSeen > 2500) 17 target = false; 18 } 19 } 20 21 function findClosest (objects , pos) { 22 ... 23 } B. Wulff, A. Fecke LectureSight 14 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios LectureSight in Action Movie Time! B. Wulff, A. Fecke LectureSight 15 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios More advanced scenarios B. Wulff, A. Fecke LectureSight 16 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios More advanced scenarios B. Wulff, A. Fecke LectureSight 17 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios More advanced scenarios B. Wulff, A. Fecke LectureSight 18 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios More advanced scenarios B. Wulff, A. Fecke LectureSight 19 / 20

Introduction Scene Regions Technology Scripting Deployment Advanced scenarios Thank you! Benjamin Wulff University of Osnabrück bwulff@uos.de www.lecturesight.org B. Wulff, A. Fecke LectureSight 20 / 20