Overview Implementation Segmentation History Frame Buffering Results DEMO Real-Time Note Digitizer! – ECE532 Team A \ / \ / 350 / \ / \ 3 ! – Sean Bell, Sanae Rosen, Konstantine Tsotsos University of Toronto, Engineering Science
Overview Implementation Segmentation History Frame Buffering Results DEMO We’ve all been in class... Projectors and tablets used for delivering notes Hard to record content, lots of wasted data or no effective way of replaying!
Overview Implementation Segmentation History Frame Buffering Results DEMO Goal Easy to record notes Efficient storage Without loss of written content
Overview Implementation Segmentation History Frame Buffering Results DEMO Goal Easy to record notes Efficient storage Without loss of written content
Overview Implementation Segmentation History Frame Buffering Results DEMO Goal Easy to record notes Efficient storage Without loss of written content
VideoToRam 8-bit Buf V0 Decode & 1-bit YCbCr 8-bit Buf V1 Threshold Thresh WriteHistory 32-bit Buf T0 New 32-bit Bus Bus 32-bit Delay 32-bit Buf T1 History Master Master History Relative DDR Delayed Delay Mem Addr Addr WriteFrame MPMC DDR Memory Write Enable Bus Port A pixel histories Logic Master Port B frame for display Delayed Port C Addr Video Out Port D Port E TFT Controller VGA Microprocessor Bus Master Overview Implementation Segmentation History Frame Buffering Results DEMO Initial system for real-time performance High-performance, real-time video processing Entirely in hardware
Overview Implementation Segmentation History Frame Buffering Results DEMO Initial system for real-time performance High-performance, real-time video processing Entirely in hardware VideoToRam 8-bit Buf V0 Decode & 1-bit YCbCr 8-bit Buf V1 Threshold Thresh WriteHistory 32-bit Buf T0 New 32-bit Bus Bus 32-bit Delay 32-bit Buf T1 History Master Master History Relative DDR Delayed Delay Mem Addr Addr WriteFrame MPMC DDR Memory Write Enable Bus Port A pixel histories Logic Master Port B frame for display Delayed Port C Addr Video Out Port D Port E TFT Controller VGA Microprocessor Bus Master
Overview Implementation Segmentation History Frame Buffering Results DEMO Revised System '*%"+,+-./& !*$"&012"3& 8"6+%*$#& 9:;:3& !+#*6& !*$"&012"3& ,>,& '()& 4"/.$56& :+$73+??"3& 01<&=.<7"3& 4"#/"$7.5+$& !"#"$%& =@=:& :1<7+/&E@& FG*<5$#&E@& C0& -4ABA& D)-,& I3*5$#& FG7H&@+37& '"3*J6.5+$& @!0& K17L17& ("$"3.5+$& D<"3&E$7"3M.6"& =.$.#"/"$7&
Overview Implementation Segmentation History Frame Buffering Results DEMO Revised System 9,+:".# 3'&"4'%# 0121%# )'*+,-&# !"# $;<':# )'.*',/+-",# $%"&'(("%# =;(/"%># 5','%+/'# ?59# 67/87/#
Overview Implementation Segmentation History Frame Buffering Results DEMO Qualitative Approach Given an input image, would like to classify each pixel: Ink Not Ink
Overview Implementation Segmentation History Frame Buffering Results DEMO Decision Tree Classifier Cr > 140? Paper and ink alone is 1 Yes No easy not ink Y > 115? YCbCr is decorelated 2 Human skin profile 3 Yes No Build decision tree to classify! not ink ink
Overview Implementation Segmentation History Frame Buffering Results DEMO Decision Tree Classifier Cr > 140? Paper and ink alone is 1 Yes No easy not ink Y > 115? YCbCr is decorelated 2 Human skin profile 3 Yes No Build decision tree to classify! not ink ink
Overview Implementation Segmentation History Frame Buffering Results DEMO Calibration Base Image Cr segmentation
Overview Implementation Segmentation History Frame Buffering Results DEMO Calibration Base Image Y Segmentation
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Pixel History Classification: … … … … … History Buffer … … … … Output: … Time
Overview Implementation Segmentation History Frame Buffering Results DEMO Double Buffering '$()*&+& !"#$%& -$%#$%& '$()*&,&
Overview Implementation Segmentation History Frame Buffering Results DEMO Circular Buffering '$()*&+& '$()*&,& -$%#$%& !"#$%& '$()*&./0"%)*& '$()*&1& '$()*&2&
Overview Implementation Segmentation History Frame Buffering Results DEMO Successes Near real-time recording of handwriting Rejection of user’s hand and writing utensil from image stream Rich writing interface: erasing, multiple colours Playback of previous writing
Overview Implementation Segmentation History Frame Buffering Results DEMO Future Improvements Recording Save to PC
Overview Implementation Segmentation History Frame Buffering Results DEMO Future Improvements Recording Save to PC
Overview Implementation Segmentation History Frame Buffering Results DEMO Design Philosophy Our philosophy : No integration stage: rotate tasks 1 Explore alternatives concurrently and choose best 2 Adapt design quickly 3
Overview Implementation Segmentation History Frame Buffering Results DEMO Lessons Learned Design complexity and tough performance criteria don’t always lead to most effective system Solve as much as possible in software
Overview Implementation Segmentation History Frame Buffering Results DEMO DEMO
Recommend
More recommend
