Computational modeling of reading Danie niela Rotelli lli Computer Science PhD Student Università di Pisa
Outline • How do we read? • The dyslexic reader • The project 7
How do we read?
Visual cortex The human lobes 9
Fovea The visual cortex 10
Written word processing 11
Objects contours: erased junctions • spared junctions • three-dimensional objects • Neuronal hierarchy Objects recognition system 12
The neuronal hierarchy 13
Semantic lexicon [knight] [ˈnaɪt] Lexical conversion Ortographic lexicon Phonologicallexicon Access to pronunciation Lexical access [k] [n] [i] Ortographic units Phonological units [' n] [a] [ɪ] [t] Spelling to sound (letters, graphemes) [g h t] (phonemes, syllables) conversion Speech Visual production normalisation knight Written input Spoken output Word reading process 14
The dyslexic reader
Normal readers Dyslexic readers The dyslexic brain 16
Reading and symmetry 17
The project
BETTER WAITING LISTS AND QUALITY OF MEDICAL TEACHING PRACTICES SCREENING OVERHEADS SERVICES Goals 19
Literacyskil ills ls: "understanding, evaluating, using In Italy, about 21% of fifteen-year-old and engaging with written text to students have poor literacy skills. participate in society, achieve one’s goals and develop one’s knowledge and potential" Level 1: In Italy, the proportion of adults "read a brief text on familiar topics performing at or below level 1 (out of 5) and locate a single piece of specific is 28%, much higher than the EU-17 information" average (16.4%). Background 20
1) Assessing ing readin ding ski kills lls • laborious and time-consuming • separate monitoring interlocked abilities: ✓ word rendering ✓ linguistic comprehension / kləʊs / CLOSE / kləʊz / 2) Data elic licit itatio ion, storage and nd ana naly lysis is • lack of adequatetechnologicalsupport for natural reading in ecological context: ✓ limited ecological validity ✓ artificially-restrained settings Motivation 21
READING TEXT NARRATIVES Reading ding assessment: NATURAL READING CONDITIONS ✓ efficient ✓ replicable and reliable ✓ customisable and flexible ✓ scalable ✓ ecological TEXT ✓ multi-dimensional UNDERSTANDING ✓ non invasive MINIMALLY CONSTRAINING DEVICES Key requirements 22
ReadLet Re Reading Ef Efficiency Model "The ability to fully understand connected texts by minimising reading time" The proposed solution 23
Required information about the reader and the task being administered The protocol - Step 1 25
Read (either silently or aloud) while sliding the finger across the written text The protocol - Step 2 26
✓ retrieving the general content ✓ retrieving word meaning from context Multiple-answer questions ontext ✓ identifying specific information in the text content ✓ retrieving information from syntactic structure ✓ forming mental representations The protocol - Step 3 27
Time-alig ligne ned d recorded d data: ✓ voice recording ✓ finger sliding time ✓ time of reading ✓ time of question answering ✓ correct answers Data acquisition 28
Data acquisition 29
Data acquisition 30
400 children 200 Tuscany, Italy (Italian) 200 Ticino canton, Switzerland (Italian) 15 Fez, Morocco (French and Arabic) 50% F/M 3rd -5th grade level 1300 trials 1/3 listening (3 pages) + questionnaire (15 questions) 1/3 reading aloud 1/3 silent reading (3 pages) + questionnaire (15 questions) ✓ touchscreen time-stamped data ✓ microphone audio data 15 stories and questionnaires ✓ time-stamped data of questionnaire filling 9 Italian 3 French 3 Arabic Preliminary testing 31
Finger touch processing 32
≠ CHILDIT2 Speech processing 33
The architecture 34
People
People 36
Thank you daniela.rotelli@phd.unipi.it
References • Dehaene, S., (2009). Les neuronesde la lecture. • Dehaene, S., et al., (2010). How Learning to Read Changesthe Cortical Networks for Vision and Language . • Ferro, M., Pirrelli, V., et al. , (2010). Reading as active sensing: a computational model of gaze planning in word recognition . • Ferro, M., Marzi, C., Pirrelli, V., et al., (2018). ReadLet: Reading for Understanding. • Gage, N., Baars, B. (2018). Fundamentals of cognitive neuroscience. • Comphys Lab - PISA ILC-CNR, (2019). Presentazione Computer modeling of reading efficiency . • Images: Pixabay (kerttu, Tumisu, MabelAmber, VictoriaBorodinova ), Flickr.com (Pete Ashton - Umberto Boccioni - States of Mind I The Farewells) 38
Recommend
More recommend
