ACM Multimedia ’12 31.10.2012 Semi-Automated Magazine Layout Using Content-Based Image Features Mikko Kuhna Aalto University Ida-Maria Kivelä School of Science Pirkko Oittinen Department of Media Technology
Contents 1. Introduction 2. Visual concept examples 3. Content-based image features 4. System comparison and results 5. Conclusion Mikko Kuhna 2 mikko.kuhna@aalto.fi http://mikkokuhna.com
Introduction Mikko Kuhna 3 mikko.kuhna@aalto.fi http://mikkokuhna.com
Introduction T A T A T = A Mikko Kuhna 4 mikko.kuhna@aalto.fi http://mikkokuhna.com
Introduction T T A T = A A T T T A A A Mikko Kuhna 5 mikko.kuhna@aalto.fi http://mikkokuhna.com
Introduction T T A T = A A T X X T T A A A Mikko Kuhna 6 mikko.kuhna@aalto.fi http://mikkokuhna.com
Introduction T T A T = A A T T X X T T T T A A A A A A Mikko Kuhna 7 mikko.kuhna@aalto.fi http://mikkokuhna.com
Introduction T T A T = A A T T X X X X T T T T A A A A A A Mikko Kuhna 8 mikko.kuhna@aalto.fi http://mikkokuhna.com
Our solution CSS3 media queries Content-based image features + (responsive web design) Combined Image Image Image image feature feature feature feature maps maps values algorithms color saliency article color map color palette skin color text salient color map overlaying map edge image map cropping importance map textureness contents map page center of importance face image face map alignment orientation perspective Mikko Kuhna 9 mikko.kuhna@aalto.fi http://mikkokuhna.com
Article color examples article text color overlaying Mikko Kuhna 10 mikko.kuhna@aalto.fi http://mikkokuhna.com
Article color examples Mikko Kuhna 11 mikko.kuhna@aalto.fi http://mikkokuhna.com
Salient color map saliency = face detection + skin color + color saliency Mikko Kuhna 12 mikko.kuhna@aalto.fi http://mikkokuhna.com
Thresholded salient color maps Mikko Kuhna 13 mikko.kuhna@aalto.fi http://mikkokuhna.com
Color palette Mikko Kuhna 14 mikko.kuhna@aalto.fi http://mikkokuhna.com
Color palette examples Mikko Kuhna 15 mikko.kuhna@aalto.fi http://mikkokuhna.com
Text overlay examples text image contents overlaying cropping page Mikko Kuhna 16 mikko.kuhna@aalto.fi http://mikkokuhna.com
Image importance Mikko Kuhna 17 mikko.kuhna@aalto.fi http://mikkokuhna.com
Image importance face detection Viola & Jones (2001) Mikko Kuhna 18 mikko.kuhna@aalto.fi http://mikkokuhna.com
Image importance face detection textureness Viola & Jones (2001) Bae et al. (2006) Mikko Kuhna 19 mikko.kuhna@aalto.fi http://mikkokuhna.com
Image importance face detection textureness color saliency Viola & Jones (2001) Bae et al. (2006) Achanta et al. (2009) Mikko Kuhna 20 mikko.kuhna@aalto.fi http://mikkokuhna.com
Image importance importance importance = face detection + skin color + textureness + edges + color saliency Mikko Kuhna 21 mikko.kuhna@aalto.fi http://mikkokuhna.com
Image features in action Combined Image Image Image image feature feature feature feature maps maps values algorithms color saliency article color map color palette skin color text salient color map overlaying map edge image map cropping importance map textureness contents map page center of importance face image face map alignment orientation perspective Mikko Kuhna 22 mikko.kuhna@aalto.fi http://mikkokuhna.com
Test magazine Combined Image Image Image image feature feature feature feature maps maps values algorithms color saliency article color map color palette skin color text salient color map overlaying map edge image map cropping importance map textureness contents map page center of importance face image face map alignment orientation perspective lakercompendium.com tietokone.fi/lehti Mikko Kuhna 23 mikko.kuhna@aalto.fi http://mikkokuhna.com
Color palette article color Mikko Kuhna 24 mikko.kuhna@aalto.fi http://mikkokuhna.com
Importance map Importance map Article cover page, landscape 1024 x 768 px Article cover page, portrait Contents page 768 x 1024 px 400 x 200 px Navigation bar 100 x 100 px text overlaying contents image page cropping Mikko Kuhna 25 mikko.kuhna@aalto.fi http://mikkokuhna.com
Column alignment + mass_center( importance ) image alignment Fernandes & Oliveira (2008) Mikko Kuhna 26 mikko.kuhna@aalto.fi http://mikkokuhna.com
System overview Document Document model Document template (HAML) (XML) (HTML5) Stylesheet Stylesheet Magazine App Authoring tool template (LESS) (CSS3) (e.g. Baker or web app) Image feature Images Processed images algorithms (JPEG, PNG) (JPEG, PNG) Mikko Kuhna 27 mikko.kuhna@aalto.fi http://mikkokuhna.com
System overview Document Document model Document template (HAML) (XML) (HTML5) Manual work lakercompendium.com (InDesign-files) Stylesheet Stylesheet Magazine App Authoring tool template (LESS) (CSS3) (e.g. Baker or web app) bakerframework.com Image feature Images Processed images algorithms (JPEG, PNG) (JPEG, PNG) Mikko Kuhna 28 mikko.kuhna@aalto.fi http://mikkokuhna.com
System comparison Manual Automatic Responsive •Paginated • Paginated •Continuous •InDesign-layout • Automated layout •Semi-automated layout •Woodwing reader • 2-levels •Web-based Mikko Kuhna 29 mikko.kuhna@aalto.fi http://mikkokuhna.com
System comparison, navigation Manual Automatic Responsive Manual Automatic Responsive A.1 A.2 S.1 S.1 S.2 C CP ... CP ... CP ... P .1 P .1 P .1 P .2 P .1 A.2 A.1 A.2 A.1.1 A.1.2 A.2.1 A.2.2 A.1 P .2 P .2 P .1 P .1 P .1 P .1 A.1 A.1.1 A.1.2 A.2.1 P .3 P .2 P .2 P .2 A.1.1 A.2.1 P .3 P .3 C = Cover, CP = Contents Page, S = Section, A = Article, P = Page Mikko Kuhna 30 mikko.kuhna@aalto.fi http://mikkokuhna.com
Quantitative results Ma Statements Manual Automatic Responsive Aut ”fully disagree” ( − 3 ) or ”fully agree” ( +3 ) I like the current section -0.10 (0.57) 0.45 (0.44) 0.43 (0.40) Appearance is interesting 0.58 (0.55) 0.68 (0.53) 0.98 (0.39) Appearance is clear 0.00 (0.65) 0.73 (0.62) 1.30 (0.40) Colors are pleasing 0.53 (0.53) 0.43 (0.47) 1.30 (0.36) Readability is good 0.20 (0.54) 0.30 (0.54) 1.05 (0.41) Font type is good 0.05 (0.50) 0.03 (0.57) 0.65 (0.41) ”too little” ( − 3 ), ”neutral” ( ± 0 ) or ”too much” ( +3 ) Number of images -0.02 (0.28) 0.05 (0.31) 0.05 (0.19) Amount of text 0.33 (0.29) -0.30 (0.33) 0.08 (0.19) Size of images 0.05 (0.31) 0.23 (0.23) -0.06 (0.23) Amount of whitespace -0.55 (0.29) 0.33 (0.29) -0.05 (0.21) Mikko Kuhna 31 mikko.kuhna@aalto.fi http://mikkokuhna.com
Qualitative results positive user comment negative user comments Manual Automatic Responsive Manual Automatic Responsive U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other 9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 % 125 277 TOTAL 148 62 150 75 Mikko Kuhna 32 mikko.kuhna@aalto.fi http://mikkokuhna.com
Qualitative results positive user comment negative user comments Manual Automatic Responsive Manual Automatic Responsive U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other 9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 % Mikko Kuhna 33 mikko.kuhna@aalto.fi http://mikkokuhna.com
Qualitative results positive user comment negative user comments Manual Automatic Responsive Manual Automatic Responsive U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other 9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 % Mikko Kuhna 34 mikko.kuhna@aalto.fi http://mikkokuhna.com
Qualitative results positive user comment negative user comments Manual Automatic Responsive Manual Automatic Responsive U Clarity U Usability V Layout R Pagination U Navigation U Interaction R Previews R Readability V Mag. likeness U Use logic V Contents page V Overall picture V Browsability V Sections R Image gallery R Columns U Nav. bar V Visual cues V Image quality V Traditionality Other 9 % 0 % 9 % 12 % 0 % 12 % 15 % 0 % 15 % Mikko Kuhna 35 mikko.kuhna@aalto.fi http://mikkokuhna.com
Recommend
More recommend
