Shot Boundary Experiments at The University of Iowa David - PowerPoint PPT Presentation

Shot Boundary Experiments at The University of Iowa David Eichmann1,2 & Dong - Jun Park2 1School of Library and Information Science 2Computer Science Department

Basic Assumptions • A relatively small number of ‘ basic ’ metrics can be composed into a metric that can perform well • Observed with ASR ( e.g., Rover ) • For this year, focus on localized video measures • i.e., contiguous pairs of frames

Basic Metrics • Color Histogram Similarity • pixels compressed to a 9 - bit color scheme, yielding a 512 - bin histogram • Frame Color Distance • scale frames to 60 x 60 thumbnails and then average the color space distance of all pixel pairs • Frame Edge Distance • generate an edge representation of frames and then the percentage of entry and exit edges

A Sample Image

A Sample Image

Composite Metrics • Boolean Predicate of Basic Metrics • Composite - 1: h < 0.95 & ( d < 0.80 | e < 0.85 ) • Composite - 2: ( h < 0.82 & d < 0.82 ) | ( h < 0.79 & e < 0.79 ) • Product of Basic Metrics • d * e * h < 0.60

Tuning / Visualization (<<'"("%=>?@#-AB !( !"#' !"#& +,-,./0,12 !"#% !"#$ :,61/974 4:B4 C,618B0/- 78-A86,14D(!E8F9:/0,46 A08:F71!E8F9:/0,46 1461!641!E8F9:/0,46 !" !)"" !)$" !)%" !)&" !)'" !%"" !%$" !%%" !%&" !%'" !*"" 3,-4!564789:6;

Tuning / Visualization (::'"$(";<==#+>? !( !"#' !"#& )*+*,-.*/0 !"#% !"#$ 8*4/-752 28?2 @*4/6?.-+ 56+>64*/2A(!B6C78-.*24 >.68C5/!B6C78-.*24 /24/!42/!B6C78-.*24 !" !" !$" !%" !&" !'" !("" !($" !(%" !(&" !('" !$"" 1*+2!342567849

O ffi cial Runs All Cuts Gradual Run Metric F - F - Rec Prec Rec Prec Rec Prec Rec Prec UIowaSB0301 histo. 0.445 0.804 0.554 0.937 0.178 0.389 0.234 0.960 UIowaSB0302 dist. 0.607 0.855 0.835 0.963 0.051 0.158 0.178 0.826 UIowaSB0303 comp - 1 0.657 0.785 0.810 0.948 0.285 0.360 0.274 0.907 UIowaSB0304 prod. 0.722 0.785 0.893 0.976 0.306 0.330 0.300 0.938 UIowaSB0305 comp - 2 0.665 0.432 0.772 0.957 0.406 0.123 0.286 0.777

Shot Boundaries, Overall Results !( !"#' !"#& )*+,-.-/0 !"#% !"#$ 4-.5/6*27 8-.520,+ ,/79/.-5+:( 9*/8;,5 ,/79/.-5+:$ !" !" !"#$ !"#% !"#& !"#' !( 1+,233

Shot Boundaries, Cut T ransitions !( !"#' !"#& )*+,-.-/0 !"#% !"#$ 4-.5/6*27 8-.520,+ ,/79/.-5+:( 9*/8;,5 ,/79/.-5+:$ !" !" !"#$ !"#% !"#& !"#' !( 1+,233

Shot Boundaries, Gradual T ransitions !( !"#' !"#& )*+,-.-/0 !"#% !"#$ 4-.5/6*27 8-.520,+ ,/79/.-5+:( 9*/8;,5 ,/79/.-5+:$ !" !" !"#$ !"#% !"#& !"#' !( 1+,233

Shot Boundaries, By T ransition Type & Source 456!7*20.-6-/0.8!92.-,!:+6;/<. 456!7*20.-6-/0.8!4/9:/.-6+!;+6</=. !( !( !"#' !"#' !"#& !"#& )*+,-.-/0 )*+,-.-/0 !"#% !"#% >?4!,/9:/.-6+@( >?4!:*/=5,6 !"#$ !"#$ =94!;-.6/>*2? >?4!,/9:/.-6+@$ =94!<-.620,+ 4AA!,/9:/.-6+@( 4@@!;-.6/>*2? 4AA!:*/=5,6 4@@!<-.620,+ 4AA!,/9:/.-6+@$ !" !" !" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( 1+,233 1+,233

Shot Boundaries, By T ransition Type & Source 4*25623!7*20.-8-/0.9!:2.-,!;+8</5. 4*25623!7*20.-8-/0.9!:/;</.-8+!=+8>/5. !( !( =:>!<-.8/?*2@ ?@:!,/;</.-8+A( =:>!5-.820,+ ?@:!<*/56,8 >AA!<-.8/?*2@ ?@:!,/;</.-8+A$ >AA!5-.820,+ :BB!,/;</.-8+A( !"#' !"#' :BB!<*/56,8 :BB!,/;</.-8+A$ !"#& !"#& )*+,-.-/0 )*+,-.-/0 !"#% !"#% !"#$ !"#$ !" !" !" !"#$ !"#% !"#& !"#' !( !" !"#$ !"#% !"#& !"#' !( 1+,233 1+,233

Conclusions • Basic metrics can perform surprisingly well on cuts • Composite metrics can damp out peculiarities of component metrics, just as in ASR • Product metrics appear to be the way to go • No arcania of boolean exploration

Future W ork • The obvious... • Frame sequence metrics • Follow the approach presented here • Specialized event detectors • camera fl ash • video e ff ects ( e.g., wipes, dissolves, ... )