Catching Events in Video Streams Mohan M. Trivedi Computer Vision and Robotics Research Laboratory Electrical and Computer Engineering Department Jacobs School of Engineering Research Review February 28, 2003
Event Catching in Video Streams Presentation Outline Research Scope Intelligent Environments with Ubiquitous Vision Vision Systems for Intelligent Environments Event Catching in Intelligent Roads and Outdoor Spaces Intelligent Rooms and Indoor Spaces
Intelligent Environments Environmental Awareness Space Activity Awareness Awareness (Static) (Dynamic) Intelligent Environments can: • Develop and maintain awareness of events • Adapt to the dynamic changes in their surroundings • Interact in a natural, efficient and flexible manner with the users Summarization Televiewing and Recall
Event Catching in Video Streams Video Streams � Data or signals in spatio-temporal domain Events � “Patterns” (semantically meaningful) in spatio-temporal domain Research Highlights: Decisions � Event based Actions • Multiple Cameras • Distributed Video Arrays • Integrated Vision Systems • Multiple level Abstractions • Semantic Databases
DIVA: Traffic and Incident Management Incident Detection and Management DIVA: Distributed Interactive Video Arrays KMET KMET : Automatic discovery of available services. Advanced Remote Agent: Outdoor Robotar for communication links, may also carry CMS (Changeable Message Sign) Tele-existence. Distributed architecture, Roadside Active Network Adaptor : A controller for adaptive control multiple sensors, 2-way wireless multimedia streaming Main unit for on-scene incident for ramp metering and intersection control verification (confirmed detection). Replaces Locally Active Little Agent: A team of small, flexible robots which current practice of waiting for CHP vehicle. work under the supervision of RA. Lower bit rate links, selected sensors, task-specific design. Team capability is very attractive
UCSD Research Testbeds and Infrastructures Research Testbeds and Infrastructures I-5 Video/ HDR Base Televiewing Antenna Video Base Base Node_1 Command Node Mobile Node_1 Wireless Test Zone – Multiple pan/tilt/zoom rectilinear cameras and Omnidirectional Cameras – Real-time 360° view of the area surrounding the pole – Wired to the lab using fibers – Sixteen high-bandwidth bidirectional Video steams accessible over internet – Televiewing for digital pan/tilt/zoom
Real-Time Shadow Segmentation IEEE CVPR 2001; IEEE Trans. PAMI, 2003 ( ) ( ) ( ) ( ) = − α CD x I x x E x i i i i ( ) ( ( ) ( ) ) 2 α = − α x arg min I x E x i i i α
Motion Based Event Capture UCSD Campus Interstate 5 DIVA for Bridge Coronado Bridge San Diego Bay
DIVA at Super Bowl Riverbed-Qualcomm Stadium: Night Surveillance Gas Lamp District: Crowd Monitoring Sea Port Command Center: Perimeter Sentry Night Surveillance Friars Road: Live Party area Traffic Flow Notification
Perimeter Sentry Real time histogram of occupancy of MZ is continuously updated. Log of the presence and of the movements inside the MZ is stored.
Semantic Queries: using Environmental Structure Spatial Structure of A Highway Segment
Distributed Video Networks and Event Based Servoing Camera cluster Monitor In a camera cluster, the normal flow can be better handled by switching among cameras Camera cluster Monitor In a camera cluster, when an incident occurs, the monitor can choose the “best” camera view and control its PTZ .. and even choose to “follow” the car responsible of the incident!!
DIVA System Architecture from primary cameras ... EVENT DETECTION INTERFACE E-A DATABASE event-action tuple 1 event-action tuple 2 event-action tuple 3 .... event-action tuple N FOCUS-OF-ATTENTION to secondary PTZ ACTION DECISION cameras ... MAKER DRIVING DIRECTIONS to robots ...
“DIVA”--Event Driven Servoing EVENT ACTION The DIAMOND stopped car in a given area Zoom in architecture (like emercency lane) (Secondary camera 1) exhibits great stopped car in a given area Zoom on the license plate Primary Secondary (like emercency lane) (Secondary camera 2) flexibility. Using incident detected Zoom on the incident the interface the Primary (Secondary camera 1) user can create incident detected take a close of the injuries (injuries) (Secondary camera 2) new event-action flat tire car Look if the driver needs help tuples (Secondary camera 1) Secondary Primary incident detected Take video from all the Secondary perspectives possible (Robot omnidirectional 1) – drive toward the incident site .... .... Primary Secondary
Camera Handover and Event Based Servoing M OV I NG Camera 3 FI X E D Camera 1 FI X E D Camera 2
Multiple Abstractions Simultaneous 3D tracking of Face recognition Face orientation estimation multiple blobs Capture of “interesting” events Kohsia Huang Mohan Trivedi comment Kohsia Huang time Mohan Trivedi answer question Ivana Mikic presentation whiteboard area MVA, 2003
Video Array for Ubiquitous Coverage Thermal Infrared Rectilinear camera network
Body Modeling and Movement Analysis System 3D Voxel Reconstruction Key Features: Original data Segmentation Voxel data 1. Completely automated system for motion capture 2. Multi-resolution 3D voxel generation 3. Heuristic model initialization, refinement, and tracking procedure 4. Robust voxel labeling procedure that handles large frame to frame displacements Potential Applications: Model initialization Tracking 1. Advanced user interfaces Initial estimate Model refinement EKF prediction Voxel labeling EKF update using the Bayesian 2. Video games and computer animation network 3. Motion analysis for medical and sports purposes Human Body Model . . . Torso position: 3 parameters z ( ) λ d 2 S 0 Torso orientation: 4 parameters torso . x y coordinate Joint angles: 16 parameters system . . = 23 parameters world = + joint coordinate d 0 . 8 S system angle = d 0 . 7 H limits = valid postures ( ) λ 1 d H 0 IEEE CVPR 2001; IJCV 2003
Body Modeling, Movement, Posture and Gait
Body and Movements neck 2 2 neck 0 0 -2 -2 20 40 60 80 100 120 140 20 40 60 80 100 120 140 2 2 shoulder (L) 0 0 -2 -2 shoulder (R) 20 40 60 80 100 120 140 20 40 60 80 100 120 140 2 2 0 0 -2 shoulder (L) -2 shoulder (R) 20 40 60 80 100 120 140 20 40 60 80 100 120 140 2 2 hip (L) 0 0 hip (R) -2 -2 20 40 60 80 100 120 140 20 40 60 80 100 120 140 2 2 hip (L) hip (R) 0 0 -2 -2 20 40 60 80 100 120 140 20 40 60 80 100 120 140 2 2 0 0 elbow (L) elbow (R) -2 -2 20 40 60 80 100 120 140 20 40 60 80 100 120 140 2 2 0 0 knee(L) knee(R) -2 -2 20 40 60 80 100 120 140 20 40 60 80 100 120 140
Event Catching in Video Streams Thanks !! Website: cvrr.ucsd.edu Distributed Control Centers
Recommend
More recommend
