Scribes or Independent with ICT? Can technology replace human - PowerPoint PPT Presentation

Reliant on Readers, Stuck with Scribes or Independent with ICT? Can technology replace human support in examinations? 9 th European e-Accessibility Forum Cité des sciences et de l'industrie - Universcience – Paris 8 June 2015

CALL Scotland • Communication, Access, Literacy and Learning • Funded primarily by Scottish Government • National Assistive and Communication Technology • Service, research and development www.CALLScotland.org.uk

Scottish School Assessments • Formal examinations sat by most students in Scotland. • Traditional paper format. • 675,361 examinations sat by 132,588 candidates in 2014. • Administered by the Scottish Qualifications Authority (SQA).

SQA Assessment Arrangements “ Assessment arrangements allow candidates who are disabled and/or who have been identified as having additional support needs appropriate arrangements to access the assessment without compromising its integrity .” Introduction to Assessment Arrangements for Schools and Colleges SQA January 2010, revised July 2014 Number of Candidates requesting AA 14,214 10.7% % of total candidates Number of requests for AA 43,344 6.42% % of total examination entries

SQA Assessment Arrangements 2005 Total number of requests 41,454 Assessment Arrangement Number of requests Percentage of requests which included the which included the specific arrangement specific arrangement Extra Time 33,146 79.96% Reader 15,740 37.97% Scribe 14,505 34.99% PA Referral 3,658 8.82% Use of ICT 2,388 5.76% Transcription with correction 1,178 2.84% Coloured Paper 1,108 2.67% Calculator 893 2.15% Transcription without correction 742 1.79% Enlarged Print 737 1.78% Question Paper signed to candidate 92 0.22% Candidate Signs Responses 38 0.09% Use of tape recorder for responses 26 0.06% Braille 19 0.05%

Digital Question Papers • Visually identical to hard copy papers • Read questions with text-to- speech • Use on-screen drawing tools • Type and choose answers on screen

Visual impairment / dyslexia • Zoom in and out • Papers (without answer boxes) can be re-flowed • Set page and text colours • Navigate with keyboard/mouse • Read with text-to- speech

Writing and answering • Type into answer boxes • Draw and annotate • Spellchecker • Word prediction • On-screen keyboards • Switch access • Speech recognition

Text-to-speech voice quality “many of the students commented that the synthetic voice was of poor quality and was difficult to understand ” “The most common suggestion for improvement to the PDF examinations was to improve the quality of the synthetic voice .” Nisbet, P.D., Aitken, S., Shearer, N. (2004) Trial of External Papers in Accessible PDF for Candidates with Additional Support Needs. http://www.adapteddigitalexams.org.uk/Downloads/Reports/

..so we licensed high quality Scottish voices from CereProc in Edinburgh 2008: ‘Heather’ licensed for Scottish schools 2011: ‘Stuart’ developed and licensed > € 2.7 million cost saving compared to schools buying high quality TTS Schools get the voices from: www.CereProc.co.uk http://www.thescottishvoice.org.uk

Low cost -> wider access All schools and students have access at zero cost to: • Adobe Reader • Text reader software • High quality voices • Past papers “Simplify, then add lightness” from SQA’s web Colin Chapman, Lotus site

Can technology replace human support? 25 000 Number of AA 20 000 requests 15 000 Reader 10 000 Scribe 5 000 ICT or Digital 0 Paper 2008 2009 2010 2011 2012 2013 2014

Assessment arrangements 2014 Extra Time 34,494 Transcription with correction 863 Separate Accommodation 24,277 Transcription without correction 421 9,399 341 Reader Calculator Modified Content (e.g. text Scribe 7,105 308 description of images) ICT / word processor 6,786 Referral of script to the Principal 192 Assessor Adapted Certificate (e.g. in Large Digital Question Papers 3,540 68 Print / Braille) Question Paper signed to Rest Period 2,232 32 candidate Taped transcription-Live 1,793 26 Coloured Paper Presentation Prompter / Practical Helper 1,783 Braille 18 Enlarged or Adapted Print 1,064 Candidate Signs Responses 11 Question Papers

Problems and limitations 1. “ Integrating the question papers and answer booklets into one .” 2. “ Pronunciation of some of the longer and geographical terms was poor .” 3. “takes up a lot of paper/ink when printing” 4. “Two machines crashed during Higher English exam .” 5. “give symbols for Science/Maths in order to do more straight on to the digital paper ”

Accommodation and costs • 9,399 reader requests; 7,105 scribe requests • ~ 18,000 individual exams • ~ 27,000 hours • Say average € 28/hour for reader/scribe? = € 756,000 • Say € 14/hour for invigilator? = € 378,000 TOTAL = € 1,134,000 in 2014 (Although it was € 2.26 m in 2013)

Independence • “The pupils' experience of using the digital exams was a very positive one. Pupils adapted to the technology with ease and each found their own preferred way of using it - some pupils used the digital reader only for text that they struggled with in the paper copy whereas others did not look at the paper exam and worked exclusively on the computer. It was this flexibility of use that impressed me.” • “I think the digital exam system is an excellent option for pupils with ASN and gives them more independence .” • Increases candidates’ feelings of independence and control over their speed of progress. Helps with confidence to face next step, e.g. college, work.

Conclusions – PDF Digital papers are: • Accessible for many candidates who require Assessment Arrangements, but they are not a universal solution for all candidates and all assessments. • Low cost to produce and use. • Reliable, familiar and secure. • Accessible on different platforms. • A pragmatic solution to the problem of providing digital versions of assessments. • A good solution for paper-based assessment systems. A viable alternative to human readers and scribes.