STUDENT-LED ASSESSMENT: RUBRIC DEVELOPMENT AND PEER GRADING IN - PowerPoint PPT Presentation

STUDENT-LED ASSESSMENT: RUBRIC DEVELOPMENT AND PEER GRADING IN WRITING ASSIGNMENTS Presented by Brice Particelli, PhD English Department, New York City 2017 Faculty Institute, Pace University

LIST What do you expect from a good Faculty Institute session? What are the central elements or expectations?

STUDENT-LED ASSESSMENT: RUBRIC DEVELOPMENT AND PEER GRADING IN WRITING ASSIGNMENTS A small study on student responses to a writing assignment where student-led assessment was a central element, and a look into pedagogical implications. Students developed the rubric for the assignment, and they graded each other based on that rubric.

PAIRS What strategies do you use to make expectations explicit for your writing assignments? Do you use rubrics to communicate this? (Perhaps use a recent writing assignment as example.)

GUIDING QUESTIONS FOR THE DAY How can we make more aspects of assessment more formative than summative? (assignment sheet, rubrics, peer review, grades, etc.) How do we make expectations more transparent (for the benefit of long-term student learning)? What are the pedagogical implications of letting students develop the assignment’s rubric? What are the pedagogical implications of letting students grade each other?

ENG 201: WRITING IN THE DISCIPLINES • “…writing effective essays and research papers in disciplinary modes and in students' field of interest. It may include interviews, analysis of journal articles, and appropriate documentation style formats…” • Focus on writing about their disciplines: • first a profile of an individual within their discipline (6 weeks) • then a genre study of a genre within their discipline (6 weeks) • and a presentation of how a genre works in their discipline (2 wks) • Goals: • think critically about their field • think critically about how their discipline communicates • write more effectively within the language of their field • write more effectively across disciplines by better understanding why and how each communicates differently

PROFILE ESSAY A “public” genre vs. “occluded” genre Familiar to anyone who reads magazines, blogs, or websites that feature individuals Scaffolded learning: from comfortable to complex. Genre study of the public, moving to self-guided genre study of the occluded Read 20+ broad exemplar texts: from Sports Illustrated to Business Insider to New York Magazine to Foreign Policy to Elle Six-week assignment Genre Study: “Genre as a typified social action that responds to a recurring situation,” as Irene Clark writes, and one that can better be “understood as originating from suitability and appropriateness, rather than from arbitrary formal conventions” (2005).

DEVELOPING CLASS CULTURE • Readings that established metacognitive questions: How do writers learn to write with authority? How do genres function within a discourse community? (i.e. Bartholomae’s “Inventing the University”). Goal: expose “authority,” help students access it. • Read exemplar texts: Without a formal assignment sheet, we began reading profile essays (across fields, styles, and subjects) • Write to question: Weekly Blackboard writing focused on genre study - comparative and focused analyses on audience, structure, intent, etc • Write for emulation/creation: In-class writing to test the waters for their own profile subject, constantly questioning why writers make the choices they do (depending on audience, subject matter, experience, time, etc) • Surveys (3) to broaden discussion: i.e. Survey 1: faith in grades despite stated lack of transparency; reliance on peers to understand assignment; benefits of peer review • Discussions on grading: Questioned how writing is assessed and who determines the goals, professionally and in class. (Asked: Most frequently awarded grade at Pace? )

Rojstaczer & Healy - research on the average grades of hundreds of universities Of all letter grades given in 4-year colleges, 45% are As; 34% are Bs; 14% are Cs.

ASSESSMENT More problem with grades… Often cited study by Brookhart (1991) that concluded that instructors offer a • “hodgepodge grade of attitude, effort, and achievement” Why use a rubric? (To make assessment more meaningful and helpful) In a 2010 review of rubric use in higher education, Reddy and Andrade found that • “One striking difference between students’ and instructors’ perceptions of rubric use is related to their perceptions of the purposes of rubrics. Students frequently referred to them as serving the purposes of learning and achievement, while instructors focused almost exclusively on the role of a rubric in quickly, objectively and accurately assigning grades” (439). This is a problem because research shows “higher achievement and deeper learning by students who have rubrics to guide their work” (439). Similarly, Margaret Heritage wrote, “When used by students as part of a formative • assessment of their works in progress, rubrics can teach as well as evaluate. Used as part of a student-centered approach to assessment, rubrics have the potential to help students understand the targets for their learning and the standards of quality for a particular assignment, as well as make dependable judgments about their own work” (p. 437, 2010).

ASSESSMENT Why get students involved in assessment? • Benefits to writer: Peer-review is frequently used: more eyes on their work, offering more feedback. Students read each other with an assessment goal in mind (i.e. “look at the evidence used, specifically focused on xxxx.”) • Benefits to reviewer: pushes reviewer to better understand their own writing through the critique of others (Lundstrom & Baker 2009) Why get students involved in the grading piece of assessment? • It’s reliable: In a 2006 study of 708 online students across 16 different courses at 4 universities, Cho, et al, found that when at least four peers grade a piece of writing, the aggregate grade is as reliable and valid as a single instructor. And in less that one-third of the courses, did students offer low ratings of reliability and validity for the peer assessment process (2006). • It empowers students: It offers the same benefits as peer-review, but with higher stakes and greater responsibility. Students must embrace becoming experts (with authority) on our version of the genre.

TIC TOC Rubric Development (3wks out): “popcorn” to create consensus; • assignment sheet and rubric sent out after that process (yes…late) “Rater Training” (1-2 wks out): Applied our rubric as a class to an • anonymous student-written profile; Applied it as part of peer review to a peer’s profile; Applied it to their own Grading Day: Students brought in two printed copies. Each graded • two, offering written feedback as well as the filled-out numeric rubric. Rewrites Reminder: To push them toward formative-focused • responses (and remove social pressures) I reminded them that I allow for rewrites. Only 3 took me up on rewrites (few rarely do), but it helped ease the worry. Control: I graded independently, putting my grades in a spreadsheet • before reviewing the peer-graded copies.

RESULTS Average grade offered by students: 87.4% Average grade offered by instructor: 87.9% I sided with the student average in all but three cases, where a peer-grader was particularly harsh, offering a D when the other peer-grader (and I) offered a B- or C+. That resulted in an uptick to a final 88.3% average . Inflated? They’d been informally graded by a peer, informally graded themselves, done two guided peer-review sessions, and gotten direct feedback from me. Plus they’d picked the subject. If they didn’t get a B, they’d probably missed multiple classes (and were lost)…

RESULTS Survey 2: one week before due date (and peer grading): • 89% Agreed or Strongly Agreed that “I understand the basic elements of this genre,” while 11% answered Neutral/No Opinion. No one Disagreed nor Strong Disagreed. • The results for “I understand the requirements of this writing assignment” were even more positive, at 97% Agreed or Strongly Agreed, and 3% Neutral/No Opinion

RESULTS Surveys over time: Students' confidence in their peers' ability to grade their work fairly and competently. Strongly Neutral/No Strongly Survey Survey Question Disagree Agree Disagree Opinion Agree It is possible for students to grade each other's Initial: 2 9 18 8 1 written assignments fairly and competently I am confident my peers will grade my work fairly Pre-Grading: 6 17 11 5 0 and competently Post-Grading: My peers graded my work fairly and competently 1 3 8 15 3 Initial Survey Pre-Grading Survey Post-Grading Survey 20 18 17 18 15 16 14 11 12 9 10 8 8 8 6 5 6 3 3 4 2 1 1 2 0 0 Strongly Disagree Disagree Neutral/No Opinion Agree Strongly Agree

RESULTS Student responses to the Post-grading survey statement "Grading my peers' writing, I learned (check all that apply)" 16 14 12 10 8 15 14 13 6 12 11 9 9 4 2 3 0 How to better A better More about my About the Styles and How to read It reinforced I did not learn evaluate peers understanding own approach subject-matter techniques that I similar texts things I already anything of thoughtfully and of how others to this writing within the essay can incorporate more critically know about value effectively evaluate my assignment into future writing and writing writing reading