ONBOARDING SPECIFICATIONS GRADING Mai Yin Tsoi Mary Beth - PowerPoint PPT Presentation

ONBOARDING SPECIFICATIONS GRADING Mai Yin Tsoi Mary Beth Anzovino Angi Lively Karen Perell-Gerson Omar Villanueva

Problems with Traditional Assessment Truth in Grading • What is an “A”? What is a “C”? • Effort vs. Knowledge/Skills • Purpose of Grades • Effects of Grading • Partial credit – reward for poor work • “Game” 2

What is Specifications Grading? • Outcomes-based assessment • Student choice and control • Mastery vs. non-mastery • Multiple opportunities for mastery Nilson, L. Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time ; Stylus: VA, 2014. 3

Benefits of Specifications Grading • Addresses weaknesses and misconceptions • Grades signify “mastery” – increases course rigor • Clear communication about expectations • Student-driven • Second chances – “gentle fail” • Less “emotional” involvement in grading 4

Ten Professors, Eight Courses, Four STEM Areas PROFESSOR COURSE Mary Beth Anzovino CHEM 2211 – Organic Chemistry I Mai Yin Tsoi CHEM 1211 – Principles of Chemistry I CHEM 2211 – Organic Chemistry I Omar Villanueva CHEM 1211 – Principles of Chemistry I Emily Henary CHEM 1211 – Principles of Chemistry I Cynthia M. Woodbridge PHYS 1112 – Introductory Physics II (Algebra-based) Angi Lively MATH 1111* – Access/College Algebra (Co-requisite model) Mary George Whitney MATH 0989 – Foundations for College Algebra AmyH Erickson MATH 2210 – Calculus II Hank G. Harmon MATH 1111 – College Algebra Karen Perell-Gerson BIOL 2451 – Anatomy & Physiology I 5

Initial Methodology: CHEM 2211 (Organic Chemistry I) • Backwards Design = List of Essential Outcomes (EO) / General Outcomes (GO) Dr. Joshua Ring, Lenoir-Rhyne University • Set up Grading Scale • Wrote Assessments – Wrote multiple versions • Chose dates for Exam Hours • Developed Token System Ring, J. In CCCE ConfChem; ACS Division of Chemical Education; 2016.

Initial Assessment Plan: CHEM 2211 (Organic Chemistry I) Pass 6 EOs + 13-15 GOs A Pass 6 EOs + 11-12 GOs B Pass 6 EOs + 9-10 GOs C Pass 6 EOs + 7-8 GOs D [Pass less than 6 EOs] OR F [Pass 6 EOs and less than 6 GOs] 7

Revised Assessment Plan: CHEM 2211 (Organic Chemistry I) • First round: final exam not integrated - “to keep B, no final!” • Second round: integrated final exam - “needed to keep B!” – Worth ~18.5% of total outcomes – Combined with mastered quizzes  Overall Grade 8

How To Write Objectives • Backwards design of course (Wiggins & McTighe, 2005) – Start with what you want the students to be able to do – Break the “what” into assessable outcomes – Consider what evidence is required to support a claim that student has mastered that outcome or objective • Use active verbs: – Draw Lewis structures & predict molecular polarity – Classify isomeric relationships Wiggins, G. and McTighe, J. Understanding by Design , Expanded 2 nd ed; Association for Supervision and 9 Curriculum Development: Alexandria, 2005.

A “Good” Objective • “Predicting relative acidity & basicity.” (General Outcome, CHEM 2211K) – Use quantitative information to compare acids & determine which is stronger – Use qualitative information to compare acids & determine which is stronger – Draw the mechanism (the “how”) for reactions involving acids & bases – Identify Lewis acids & bases • Can write an assessment item for each of these! 10

How to Grade Assessments and Set Grading Policy • Each question is correct or not – no partial credit – Grading is quicker – immediate feedback for students – No debate over how much partial credit an answer was worth • Instructor sets the benchmark for pass/fail 11

MATH 1111 – Access Algebra (Co-requisite Model) • Developed list of Critical Skills (CS) / General Skills (GS) • Prerequisite Materials in addition to skills used repeatedly in subsequent skills were deemed “Critical” • Set up Grading Scale • Created Multiple Versions of Assessments for Multiple Attempts • Chose dates for Three Exam Hours • Created Supplemental Open Lab time • Developed Token System • Counseled Students Individually for Success Strategies 12

MATH 1111 – Access Algebra (Co-requisite Model) Class format: • CLASS I) Instruction with Gradual Release of Responsibility/Online Homework Assigned • CLASS II) Reinforcing In-Class Practice • CLASS III) Reality Check (5 question quiz – must earn at least 4/5) 13

MATH 1111 – Access Algebra (Co-requisite Model) Typical Class Meeting: 15 min: Reality Check on CS3 30 min: Reinforcing In-Class Collaborative Practice on CS2; Reality Check graded while they work; we go over answers to ICP together 5-10 min: Hand Back Reality Check/Discuss Correct Solutions 5-10 min: Break 60 min: Instruction on New Material/HW Assigned 14

MATH 1111 – Access Algebra (Co-requisite Model) Supplemental Open Lab • Class is scheduled TR 10:15-12:15 • Class is invited to supplemental open lab environment TR 8-9:45  Work on Homework  Earn Tokens  Retake Assessments  Get (some) Individual Instruction/Tutoring/Mentoring 15

BIOL 2451 – Anatomy and Physiology I • Unique course – no calculations so finite number of questions from the material • 3 attempts throughout the semester for general objectives • 5 application objectives for lecture material (case study application of material) • 20 general objectives for lecture material (primarily physiology) • 10 general objectives for laboratory material (primarily anatomy) 16

BIOL 2451 – Anatomy and Physiology I (Assessment Plan) GRADING SCHEME FOR OVERALL LECTURE GRADE IN BIOL 2451K Lab Lecture Grade Requirement Requirement A Pass 5 AOs + 18-20 GOs Pass 3 LRs + 9-10 GOs B Pass 5 AOs + 16-17 GOs Pass 3 LRs + 8 GOs C Pass 4 AOs + 14-15 GOs Pass 3 LRs + 7 GOs D Pass 4 AOs + 12-13 GOs Pass 3 LRs + 6 GOs F [Pass less than 4 AOs] OR [Pass less than 3 LRs] OR [Pass 4 AOs and fewer than 12 GOs] [Pass 3 LRs and fewer than 6GOs] 17

General Observations by Faculty • Less grading time – “fighting for partial credit” • Expectation of student to PREPARE before lecture finally realized • Higher expectations of student mastery • Less “guilt” with grading • Harder, less “sensitive” to student excuses 18

General Observations by Faculty • Need to “PR” the system – A LOT! • “Gentle fail” * – sensitivity to student perception of “fail” • Administration comments: – More feedback – Cheerleading * Nilson, L. Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time ; Stylus: VA, 2014.

Observations by Villanueva in Principles of Chemistry I • Increase in student engagement, attendance, and participation • Students do not beg for “partial” credit or bonus points • High levels of student excitement about course! • Although more frequent assessments, grading effort is lower • Lowest withdrawal (“W”) rate at midterm ever seen (only 1 student withdrew out of 52) 20

Midpoint Student Survey in Principles of Chemistry I “I truly feel like I am mastering “After I pass a quiz, even Do you prefer the content in this class though after several tries, I specifications grading through specs grading (even if feel very confident in what I instead of a "traditional" it takes several tries to pass a have learned in the course” three in-class exams and quiz)” a final? 11% 9% 11% 89% 89% 91% Agree Disagree Yes No Agree Disagree 21 Data obtained from n=45 students

Midpoint Student Survey in Principles of Chemistry I “Specs Grading allows me “I didn't think I could do well in to choose what grade I chemistry or learn chemistry, but “Specs Grading has changed how I can get in the class at the specs grading is changing that (so learn Chemistry” end of the semester (I far)” have control over my own grade if I want to)” 2% 9% 20% 80% 91% 98% Agree Disagree Agree Disagree Agree Disagree 22 Data obtained from n=45 students

Student #14 – Organic Chemistry I 1 2 3 4 5 6 Repeating “Excited” At Retook Retook 12 FINAL student about EO (out of 15) Midterm GRADE tokens quizzes at GO C A least 1 x quizzes at least 1 x STUDENT #14 – ORGANIC 1 23

Student #3 – Organic Chemistry I 1 2 3 4 5 6 Athlete Wary Retook 4 Retook 8 At FINAL about “All EO (out of Midterm GRADE or quizzes at 15) GO C A Nothing” least 1 x quizzes at least 1 x STUDENT #3 - RILEY 24

SPECIFICATIONS GRADING – REINFORCEMENT FOR RETENTION • Reinforcement = “really good base knowledge for Orgo 2” • “…I had to make adjustments on the stuff I wasn’t retaining knowledge on…” 25

Student Comments (so far..) • “It gives me the power back over my own grade.” • “Lots of quizzes…rather than cramming at the last minute.” • “…helps to assess how well you…know the material.” • “…more likely to go back and try and relearn the material. I feel less discouraged.” 26

Student Comments (so far..) • “You aren't stressed about failing the class.” • “It forces me to keep up with the material” • “It is stressful at first, until I got the hang of it..”