Attendee Control Final talk for the Bachelors Thesis by Leon Nissen - PowerPoint PPT Presentation

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Attendee Control Final talk for the Bachelor’s Thesis by Leon Nissen Monday 9 th December, 2019 Chair of Network Architectures and Services Department of Informatics Technical University of Munich

Problem definition TUMexam is based on the principle that handwritten exams are digitized by scanning. To identify the exam and the corresponding student, TUMexam uses stickers with information to be attached to the first page of the exam. The student has to sign his attendance on the tutors sheet. • The paper on which the information is printed is expensive. • Printing all stickers takes a long time. • The sticking process is time-consuming. Figure 1: TUMexam sticker on exam sheet Leon Nissen — Attendee Controll 2

Visionary scenario Leon Nissen — Attendee Controll 3

Related work Tool Advantage Disadvantage ML model accurate needs data Tesseract integration accuracy CRAFT accurate slow Apple Vision FW on-device iOS only Cloud services accurate privacy Table 1: Related work advantages & disadvantages (a) Apple Vision framework [1] (b) CRAFT – Character-Region Awareness For Text detection Leon Nissen — Attendee Controll 4

Workflow • Recognize ID card • Find registration number • Cut registration number • Recognize digit Leon Nissen — Attendee Controll 5

Workflow – Recognize ID card Figure 2: Recognize ID card – Google Cloud Vision API [2] D complete Blue White Sum Correct 55 36 91 Wrong 1 0 1 Table 2: Student ID card type identification. Leon Nissen — Attendee Controll 6

Workflow – Find registration number (a) morphologyEx function applied on student ID. (b) Highlight all text. D complete Blue White Sum Correct 47 36 83 Wrong 9 0 9 Table 3: Find registration number Leon Nissen — Attendee Controll 7

Workflow – Cut registration number Figure 3: Naive Cutting (a) Frog Window Cutting (b) Sliding Window Cutting Figure 4: Other cutting approaches D number Blue White Sum Correct 376 288 664 Wrong 0 0 0 Table 4: Cut registration number into digits. Leon Nissen — Attendee Controll 8

Workflow – Recognize digit Generated-Data (total: 620,000) Real-Data (total: 310,000) Blue White Sum D digit Correct 374 287 661 Wrong 2 1 3 Table 5: Machine learning model evaluation for single digits. Leon Nissen — Attendee Controll 9

Statistics – Overall result • Recognize ID card = 98.91% • Find registration number = 90.21% • Cut registration number = 100% • Recognize digit = 99.54% For calculation we use the accuracy of all systems. 0.9891 · 0.9021 · 1 · 0.9954 8 = 0.8599 ≈ 86 % Leon Nissen — Attendee Controll 10

Limitations & problems Figure 5: Misrecognized ID card. (a) Shows 1 (b) Shows 3 (c) Shows 5 Figure 6: Misrecognized digits Leon Nissen — Attendee Controll 11

Future work • Evaluate the current frameworks to find and cut out the student ID on an iPhone (Felix Schrimper) • Develop a Minimum Viable Product (MVP) • Test the MVP in production • Based on an iOS app, create an Android application Leon Nissen — Attendee Controll 12

References Apple. Documentation Vision Framework. https://developer.apple.com/documentation/vision . Accessed: Oct. 30, 2019. Google. Vision AI. https://cloud.google.com/vision/ . Accessed: Dec. 9, 2019. sckit learn. A demo of K-Means clustering on the handwritten digits data. https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_digits.html . Accessed: Dec. 9, 2019. OpenCV. Morphological Transformations. https://docs.opencv.org/trunk/d9/d61/tutorial_py_morphological_ops.html . Accessed: Nov. 10, 2019. OpenCV. Image Processing - Sobel Derivatives. https://docs.opencv.org/2.4/doc/ tutorials/imgproc/imgtrans/sobel_derivatives/sobel_derivatives.html . Accessed: Nov. 4, 2019. Technical University of Munich. Schrift und Satz. https://portal.mytum.de/corporatedesign/index_html/vorlagen/index_schrift . Accessed: Dec. 9, 2019. Leon Nissen — Attendee Controll 13

Backup – Recognize ID card clt = KMeans(n_clusters=1) Figure 7: K-Means clustering [3] Leon Nissen — Attendee Controll 14

Backup – Find registration number – morphologyEx blackhat = cv.morphologyEx(img, cv.MORPH_BLACKHAT, kernel) It is the difference between the closing of the input image and input image. [4] Figure 8: MorphologyEx function [4] Leon Nissen — Attendee Controll 15

Backup – Find registration number – Sobel sobel = cv2.Sobel(blackhat, ddepth=cv2.CV_32F, dx=1, dy=0, ksize=-1) The Sobel Operator is a discrete differentiation operator. It computes an approximation of the gradient of an image intensity function. [5] Figure 9: Sobel function [5] Leon Nissen — Attendee Controll 16

Backup – Recognize digit – Font The font of the student card comes closest to the font of the TUM’s own font. This can be found on the TUM website under corporate design. [6] Figure 10: TUM Neue Helvetica [6] Leon Nissen — Attendee Controll 17