Overview/Questions What are Computer Graphics? How do video games - PDF document

CS101 Lecture 32 Advanced Topics: Computer Graphics & Computer Vision John Magee 6 August 2013 1 Overview/Questions – What are Computer Graphics? – How do video games work? – How are computer animation movies made? – What is Computer Vision? – Video-based Human-Computer Interaction 2 1

What are Computer Graphics? Many uses: – User interfaces, Applications – Design, Art – Video games – Animation – Visualization 3 Recall: Vector Graphics Example A vector-graphics example in Python. Recall Computing: Input  processing  output Computer graphics in general: Description of scene  Visual representation (Image) 4 2

2D Graphics Imagine a flat world Vector Graphics assignment by CS108 student, Spring 2008 5 2D Games  Arrangement of graphic elements on a plane.  Can be vector graphics or raster graphics.  Sprite – A (small) raster graphic that can be moved around a display. Sprites and masks from Pac-man 6 3

What happens in real world? Pinhole camera 3D world is projected through a pinhole or a lens on to a 2D plane. 7 3D Graphics Description of 3D scene and viewpoint  simulated projected image How do you describe 3D world? • Math equations – spheres, planes, etc. • Polygon mesh – represent a complex 3D surface with many triangles. 8 4

Real-Time vs. Long Rendering Rendering – The process of creating the images from the description. Usually involves Rasterization.  Interactive graphics (GUI, Video Games) must render the scene quickly.  Art, Animation, etc. can take much longer. 9 Ray Tracing  Traces a path of light from the camera through the image pixels into the scene description.  Long rendering time. Good for creating photorealistic images. 10 5

Ray Tracing – My CS680 homework assignment #4 from 2001 11 Ray Tracing  Some better examples:  POV-Ray Hall of Fame – http://hof.povray.org/ Persistence of Vision Ray Tracer http://www.povray.org/ 12 6

Other Techniques Texture Mapping Shading 13 Bump Mapping Sphere Bump Map Result 14 7

Graphics Accelerator – Specialized hardware for real-time rendering (usually gaming!) 15 Graphics in the news recently: WSJ: In Pixar's 'Brave,' New Software Adds Rich Detail http://online.wsj.com/article/SB10001424052702303836404577472942036042750.html http://www.msnbc.msn.com/id/47914137/ns/technology_and_science-science/t/brave-brings-hair-raising-animation- life/#.T-flfrVrMYQ http://phys.org/news/2012-06-brave-features-hair-raising-animations.html 16 8

Graphics in the news recently:  Throwing shade: how Pixar changed the way light works for 'Monsters University' – Inside the studio's new Global Illumination system  http://www.theverge.com/2013/6/21/4446606/how-pixar- changed-the-way-light-works-for-monsters-university 17 Animation How does Pixar make their movies? – They have custom software and a large “rendering farm” http://www.elephantsdream.org/ Open Movie Project: – 2 Films made with free/open source software. – Made with Blender http://www.blender.org/ and other software. – Code for the movie is available. http://www.bigbuckbunny.org/ 18 9

Computer Vision How are Computer Graphics and Computer Vision Related? Recall: Computer graphics in general Description of scene  Visual representation (Image) Computer Vision in general: Image(s)  Some description of the scene Example - Input: Image Output: Face locations Fujifilm camera demo 19 Computer Vision Many Subfields 20 10

Some Easy Techniques  Color Analysis  Motion Analysis  Template matching (Ignore the extra detail on the next few slides) 21 Color Analysis Skin color analyzed by lookup of 2D histogram: Histogram can be updated during operation  22 11

Motion Analysis Motion analysis by frame differencing: Recall: Video compression uses frame differencing. 23 Template Matching Sum of Absolute Differences  Normalized correlation coefficient matching over multi-resolution search space.  12 x 16 Template matching over all resolutions  24 12

Face Tracking 25 Face Detection vs. Face Recognition Face Detection exploits the similarities between human faces. Face Recognition exploits the differences between human faces. 26 13

Computer Vision What can go wrong? – You might not know anything about a scene! – Lighting could change! – People could do weird things! 27 Human-Computer Interaction We’re all used to mouse and keyboard… But you could use a camera to track motion… Camera Mouse http://www.cameramouse.org/ (Free Download) A user with severe paralysis using the Camera Mouse Articles and Videos: http://www.bu.edu/today/2009/04/10/seeing-eye-mouse http://www.bu.edu/today/2011/big-meaning-in-the- smallest-movements/ 28 14

Google Similar Images http://www.youtube.com/watch?v=6fD2t4d2Ln4 http://similar-images.googlelabs.com/ 29 Bat Tracking http://www.cs.bu.edu/faculty/betke/research/bats/ 30 15

Computer Vision in Games  Playstation Eye – http://en.wikipedia.org/wiki/PlayStation_Eye – http://www.youtube.com/watch?v=bgLN7uOj8Jg  Xbox Kinect – http://www.youtube.com/watch?v=oACt9R9z37U – http://www.youtube.com/watch?v=GH_gDreIdcM&feature=related 31 Take-Away Points – 2D and 3D Computer Graphics – Real-time rendering – Ray Tracing – Computer Vision – Human-Computer Interaction 32 16