Assessing Student Writing Across Programs and Time: (Inter)disciplinary and Programmatic Perspectives Presented by Ann M. Blakeslee, Ph.D., Director, Office of Campus and Community Writing W. Douglas Baker, Ph.D., Associate Dean, College of Arts and Sciences
Session Parts 1. Challenges, potential, and importance of assessing student writing across time and programs 2. EMU’s context and history 3. Our response and the results and outcomes • Faculty investment • Use of campus LMSs and other resources 4. Plans and next steps
First, why are campus-wide writing programs and assessments of those programs and student writing important?
Importance of Campus-wide Writing and Assessment of Writing • Proven importance of writing to help students improve as writers and to support their learning and critical thinking (Anderson et al., 2015; Peer Review 2017) • First-year, intermediate, and advanced [writing in the disciplines (WID)] offerings at many universities (Morris, 2017; Rutz and Grawe, 2017) • Deliberate/non-deliberate scaffolding of writing across the curriculum • Importance—and difficulty—of assessing these initiatives (NCTE, 2014; Sparks et al., 2014; Wardle and Roozen, 2012)
Challenges in and Potential for Assessing Student Writing • Faculty perceptions of responsibility • Disciplinary differences • Programmatic vs. individual • Past/current experiences with “assessment” • Example: “It turns out that the assessment program your college imposed on you was probably never going to improve anything. A new article by an assessment insider [David Eubanks] explains why this is so and suggests that assessors have known for sometime now that assessment does not work.” - The Chronicle of Higher Education , January 12, 2018
EMU Assessment Context and History • General Education Reformed (approved 2006-07) • Plans to assess Gen Ed learning outcomes (2007-12) • Centralized approach: Office of Institutional Effectiveness and Accountability (2009-2012) • Decentralized: Programmatic assessment of student learning to colleges and General Education Program (2012-current) • Developing local capacity and leadership • Soliciting participation (e.g., College of Arts and Sciences, approximately 70% of 134 programs contribute)
Our response to the challenges and to our context and history…
Small Group Share What is the situation on your campus with student writing and the assessment of student writing programs? Discuss in a small group of 3-4.
Gen Ed/WI history… 1. General Education reform – 2002/03; Implementation – 2006/07 Education for Participation in a Global Community • • Effective Communication – first-year writing and speech • Upper-level writing-intensive course requirement 2. FYWP assessment – 2013-16 (three parts) 3. WI assessment – starting in 2015/16; ongoing
Background… 1. Assessment of WI was planned from the outset 2. WI classes are taught and approached in a variety of ways 3. Writing Across the Curriculum (WAC) has supported the development and teaching of WI classes since their inception 4. Colleges, departments, schools, and programs all have different needs and interests and face different challenges and opportunities with WI classes
WI Outcomes • Literacy Strategies that Support Inquiry: Develop and employ successful and flexible reading and writing strategies that support sustained inquiry in a discipline • Rhetorical Awareness: Use writing strategies that achieve the purposes for writing and address expectations of audiences in a disciplinary context • Research Strategies: Formulate research questions and employ strategies for researching and responding to those questions • Genre Awareness: Use discipline-specific genres to communicate info • Disciplinary Conventions: Understand conventions for communicating, disseminating, and interpreting information in a discipline
Our Initiative • Began with WAC/UWC impact survey in 2016 • WI assessment pilot was carried out in 2016 • Dietetics, social work, history • Used AACU value rubric for written communication • Most recent assessment started in winter 2017 • Participation was voluntary – sent invitation to faculty on record for teaching a WI class that semester • 37 initial faculty volunteers • Meetings to explain and orient volunteers to assessment
Components and Rationales • Faculty questionnaire: • Writing assigned, genres taught, strategies used • Understanding of outcomes • Perceptions of effectiveness in teaching and achieving outcomes • Perception of student success in developing proficiency with outcomes • Ease in teaching, allocation/use of class time • Collection and review of syllabi: • Presence/absence of required rationale • Presence/absence of WI/any writing outcomes • Uses of drafts, peer review, instructor feedback, revision, other strategies • Student questionnaire: • Students’ perceptions of proficiency with the outcomes
Components and Rationales • Rubric in Canvas: • Faculty assessment of student success with outcomes on a particular assignment • Post-assessment questionnaire: • Any changes they would make as a result of the assessment • Use of class time after vs. before assessment • Understanding of outcomes after assessment • Perception, after assessment, of • Success and effectiveness in teaching and achieving outcomes • Student success in becoming proficient with outcomes • Ease or difficulty of teaching outcomes • Focus groups: • Students’ experiences with the classes • Faculty members’ experiences with the classes and perceptions…
Approach • Collected course section numbers • Developed and loaded rubric into Canvas • Held information sessions • Developed and distributed questionnaires • Provided written instructions for using the rubric • Provided individualized instruction as needed
Small Group Share What have you, or others on your campus, done (or what would you like to do) to assess writing programs and/or student writing? Why? What are your reasons/rationales? Discuss in a small group of 3-4.
Faculty Questionnaires…
Demographics/Information • Most tenured (17), tenured track (8) • Disciplines: STEM, humanities, social sciences, business/professional • Majority face-to-face (76%) • 21% hybrid/web-enhanced • Majority had taught class more than twice • 46% had taught it 6+ times • Enrollments ranging from 15 to 45 • Most 20-25
Writing • Mix of formal/informal • Genres: research papers (66%), lit reviews, essays, abstracts, summaries, multi-genre projects, other (48%) • Approaches to assignments – 86% scaffold • Drafts (76%) • Revision (68%) • Peer review (64%) • Project proposal, annotated bib (52%) • Rubrics (79%) • Feedback on drafts (83%) • Opportunities for revision (86%); Require revision (60%)
Writing Feedback I'm developing Canvas discussions in which students explore ideas in their formative stages, and students respond to each others' Canvas writings in this way, and I also comment on and grade these postings. For each of the two assigned "analytical essays" (no research), I comment on essays and require/recommend resubmission with revisions. For the final "research" essay, I require that students defend their research questions orally and then they bring in rough drafts for peer review or individual consultation. I also require an "annotated bibliography" for this assignment in the weeks that precede the draft, so that I can help students to find the best research for their ideas (and so that I can assess -- and intervene when necessary -- how well they understand and are able to use their sources). (high touch approach )
Writing Feedback My feedback is typically at the higher and middle levels of concern with an emphasis on what worked/did not work for this assignment and questions to answer or things to be aware of when preparing for the next assignment. To allow students time to review and benefit from the feedback, I work to return feedback within one week of the assignment due date and make sure that major writing assignments are reasonably spaced throughout the term.
Outcomes – Understanding of Excellent Good Average Limited Wtd. Avg. Literacy 38% 52% 7% 3% 1.76 Rhetorical 52% 38% 7% 3% 1.62 Research 61% 25% 14% 0 1.54 Genre 55% 34% 10% 0 1.55 Conventions 69% 28% 3% 0 1.34
Outcomes – Perception of Effectiveness in Teaching Highly Effective Somewhat Not Effective Wtd. Avg. Effective Effective Literacy 28% 45% 21% 7% 2.07 Rhetorical 17% 66% 17% 0 2.00 Research 22% 44% 30% 4% 2.15 Genre 34% 41% 24% 0 1.90 Conventions 29% 43% 25% 4% 2.04
Outcomes – Perceptions of Student Success in Achieving Proficiency with Highly Successful Somewhat Unsuccessful Wtd. Avg. Successful (meets Successful (does not (exceeds expectations) (approaches achieve expectations) expectations) expectations) Literacy 7% 66% 24% 3% 2.24 Rhetorical 3% 72% 24% 0 2.21 Research 19% 33% 33% 15% 2.44 Genre 14% 69% 17% 0 2.03 Conventions 10% 62% 28% 0 2.17
Student Questionnaires…
Recommend
More recommend
