When Students Copy: Examining state policies, practices, and consequences of copied text in evidence based writing NCSA 2018 1
Presenters and Moderator Sally Rhodes (Presenter) English Language Arts Coordinator, Test Development Center, Florida Department of Education Jan Reyes Ed.D. (Presenter) Director of Assessment Development, Georgia Department of Education Diane Simaska/Rebekah Baum-Leaman Ed.D. (Presenters) Education Associates, Division of Instructional Quality, Pennsylvania Department of Education Shiloh Cox-Fouch (Presenter) Senior Director Performance Assessment Services, DRC Holly Baker Ph.D. (Moderator) Vice President Education Solutions, DRC
Context and Goals • Copied text versus plagiarism – Using provided sources to support students original words and ideas • Requires a full set of critical reading skills, not just comprehension – Characterized as an “inability to read well enough to understand, analyze, synthesize, and evaluate sources and then use those sources in support of an argument.” Horning, 2011 • Goals – Explore state policies for identifying copied text, collecting information on copied text and reporting – Discuss state initiatives to improve writing instruction to address copied text – Discuss lessons learned and ideas for future development 3
Florida’s Observations of Copied Text Florida Standards Assessment English Language Arts Sally Rhodes FSA ELA Coordinator 4 www.FLDOE.org
Florida Standards Assessment English Language Arts FSA ELA is administered in grades 3-10 with approximately 200,000 students per grade. • Reading Component – two sessions • Writing Component – • Grades 4 – 10 • Text-based writing prompt • Opinion/Argumentation or Informational • One session – 120 minutes • Students receive a combined ELA scaled score for reading and writing. 5 www.FLDOE.org
FSA Writing – Rubric Domains • Clear claim/controlling idea Purpose, Focus, • Effective organization Organization (4) • Logical progression of ideas • Integrated, relevant evidence from sources Evidence and • Thorough, convincing support through elaboration Elaboration (4) • Effective expression of ideas Conventions (2) • Adequate command of basic conventions 6 www.FLDOE.org
What Constitutes a “Copy” Score? To qualify as scorable, a student must give a controlling idea, a reason, and “a little bit more.” • The response demonstrates no original writing . • Often the introduction is the prompt followed by a three-pronged thesis of copied phrases from text. • Each paragraph may have transition words, but followed only by direct copy from texts. • A few words may be substituted but usually only basic synonyms are used (i.e., “wrong” used instead of “incorrect”). 7 www.FLDOE.org
Condition Code - Copy • Between 1% and 9% of operational responses per grade have been coded as copy since 2015. • Student receives 0 of possible 10 raw score points on writing portion of assessment. • Factors into overall FSA ELA score • Students scoring 0 on writing rarely receive a satisfactory Level 3 score (out of 5 levels). • Reported out to students/schools/districts. 8 www.FLDOE.org
Percentage of Responses Receiving Copied Text Condition Code 10 9 8 7 6 5 4 3 2 1 0 2015 2016 2017 2018 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 9 www.FLDOE.org
Mode Effects – Informational vs Argumentation 7 % of responses coded as copied text 6 5 4 3 2 1 0 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Writing by Grade and Mode Informational Argumentation 10 www.FLDOE.org
Approach to Handscoring • Train scorers with a training set including multiple examples of copied text and an example of a “barely scoreable ” response. • Scorers do many read-throughs of text, including in small group discussions. • A response must go through a minimum of three levels of review before any condition code can be applied. • Keep a file of “decision papers” to be used from year to year for consistency. 11 www.FLDOE.org
Automated Scoring Engine • Grades 8 and 9 are given one score from an automated scoring engine and one human score. • The engine is trained to pick up on several dimensions including prompt copy, duplicate text, not enough text, etc. • If identified as problematic by the engine, a response automatically receives two human scores. 12 www.FLDOE.org
Steps Taken by FDOE • Provided an example of an annotated “copied text” condition code in Writing Samplers posted online. • Emphasized the importance of analysis and synthesis of texts as related to writing prompts during statewide meetings. • Enhanced student score reports to show scores by domain and define condition codes. https://fsassessments.org/resources/ 13 www.FLDOE.org
Lessons Learned In future writing development • Prompts should encourage students to analyze and synthesize information from multiple sources . • Make purpose crystal clear – argumentation/opinion, or expository/informational. • Concept must be appropriate for the grade-level Florida student with clear connection between the topic, text set, and prompt. • Avoid overly technical articles/texts. 14 www.FLDOE.org
Questions? Sally Rhodes FSA ELA Coordinator sally.rhodes@fldoe.org (850) 922-2584 ext. 2228 15 www.FLDOE.org
Richard Woods, Georgia’s School Superintendent “Educating Georgia’s Future” gadoe.org Georgia’s Experience wi with th Co Copi pied ed Text xt Jan Reyes, Ed.D. Director of Assessment Development 7/6/2018 16
Hi Hist stor ory y of of Writ iting ing Ass ssess ssment ent in in Geor orgia gia Richard Woods Georgia’s School Superintendent “Educating Georgia’s Future” gadoe.org • Evidence-based writing was introduced into Georgia’s student assessment program in the 2014-2015 school year with the transition to the Georgia Milestones Assessment System. • Prior to Georgia Milestones, writing was assessed through separate performance-based writing assessments in Grades 3, 5, 8, and 11, as required by Georgia law. 7/6/2018 17
Former rmer Writ iting ing Ass ssessment essment Prog ogram am Richard Woods Georgia’s School Superintendent “Educating Georgia’s Future” gadoe.org • Student writing was assessed in four domains: • Ideas (40%) • Organization (20%) • Style (20%) • Conventions (20%) • Performance Levels: • Does not meet • Meets • Exceeds • A performance level of Meets or Exceeds on the Georgia High School Writing Test was a requirement for high school graduation. 7/6/2018 18
Former rmer Writ iting ing Ass ssessment essment Prog ogram am Richard Woods Georgia’s School Superintendent “Educating Georgia’s Future” gadoe.org Types of Grade Time Writing Type of Assessment Level Allowed Assessed Narrative Teacher evaluation; includes all types of 3 Informational Year-long writing Persuasive Narrative Response to assigned prompt for one 5 Informational 120 minutes type of writing Persuasive Expository Response to assigned prompt for one 8 100 minutes Persuasive type of writing 11 Persuasive Response to assigned prompt 100 minutes 7/6/2018 19
Current nt Writ iting ing Ass ssessment essment Richard Woods Georgia’s School Superintendent “Educating Georgia’s Future” gadoe.org • Georgia Milestones is a comprehensive summative assessment program spanning grades 3 through high school. • Students in grades 3 through 8 take end-of-grade (EOG) assessments and high school students take end-of-course (EOC) assessments. • Georgia Milestones measures how well students have learned the knowledge and skills outlined in the state-adopted content standards in English language arts, mathematics, science, and social studies. 7/6/2018 20
Geor eorgia gia Mile ilestones stones Richard Woods Georgia’s School Superintendent “Educating Georgia’s Future” gadoe.org • Features of the Georgia Milestones Assessment System include: • constructed-response items in English Language Arts and mathematics (all grades and courses); • a writing component (in response to text) at every grade level and course within the English Language Arts assessment; • norm-referenced items in all content areas and courses, to complement the criterion-referenced information and to provide a national comparison; and • transition to online administration over time. 7/6/2018 21
En End-of of-Gr Grade ade EL ELA A Ass ssessments essments Richard Woods Georgia’s School Superintendent “Educating Georgia’s Future” gadoe.org • Georgia Milestones English Language Arts (ELA) End-of- Grade (EOG) assessments measure student achievement in grades 3 through 8. • Georgia law mandates that student achievement in reading, as measured as a component of the ELA EOG assessment, be utilized in promotion and retention decisions for students in grades 3, 5, and 8. • Students who fail to demonstrate grade-level achievement on these measures must receive remediation and be offered an opportunity for a retest prior to consideration for promotion to the next grade. 7/6/2018 22
Recommend
More recommend
