THE OREGON COMMUNITY FOUNDATION Studio to School Initiative Latinos in Oregon: Evaluation Jam Session Trends and Opportunities Qualitative Analysis Part Two in a Changing State April 16, 2018
Logistics • Thank you for patience! • We are recording • If you can’t hear us, check your speaker or switch to the phone. Stay on your computer to see the slides!
Logistics • Everyone is muted, we’ll unmute for Q&A if feasible • Please mute your microphones as well • Use chat box for questions, etc.
Reminders • Today’s jam session slides and recording available on the website: • http://www.studiotoschool.org/info-for- grantees/project-evaluation-worksheets/
Agenda • Today • Thematic analysis vs rubrics • Creating rubrics • Using rubrics
Other Rubric Resources • Authentic assessment webinar • Rubric videos • Website Resources • OCF team!
What is a rubric? • Type of rating scale, like a customer satisfaction survey • Key difference: the levels of a rubric are well defined
Rubrics Novice Proficient Advanced Criteria Description Description Description Criteria Description Description Description Criteria Description Description Description Criteria Description Description Description
Rubrics vs Thematic Analysis Thematic Analysis Rubric Analysis More time spent gathering and More time spent creating interpreting data interpretation tools Inductive and deductive reasoning Primarily deductive reasoning No value attached to analysis Value attached (So what?)
Process
Process
Inductive vs Deductive
Top Down Buckets
Bottom up Buckets
Use both!
Rubrics are more top down Detailed description Data
The “So what?” step vs
How do you know which approach to choose? • Why choose a rubric? • Why choose thematic analysis?
Deciding between rubrics and thematic analysis Rubric Thematic analysis When top-down makes Bottom up or a sense combination is appropriate Levels or stages make Not sensible to develop sense levels Change is Stand-alone effort anticipated/sought in particular ways
Questions
Process of using a rubric Define what “art skills” means for your program Create rubrics to look at Create and collect student work examples of student work Use the rubric you created to assess student work
Focus on arts skills • Today the focus is on using rubrics for arts skills ratings • Can also use them for rating other parts of program performance
Define “art skills” • What do you want students to learn? • What would it look like for students to use those skills? • More than just creating art! • Thinking about art • Reflecting on art
Start with your logic model!
Student Arts Skills “Students will demonstrate an “Increase thematic discussion increased knowledge of and reflection through an storytelling” artistic lens and discuss what art means personally.” “Student’s will improve their arts vocabulary” “Student performance in adjudicated competition “Student’s build skills, improves over time.” knowledge, language and engagement in “Increase in student learning - the arts, music skill, knowledge, ability and education, and appreciation in music.” performance.” “Acquisition or “Students develop their improvement of arts skills demonstrated through media arts ability during student understanding and and after school.” application of the SHOM”
Teacher Ability to Integrate Arts “Core subject teachers gain “Teachers can knowledge, ability, confidently incorporate confidence and arts learning in the commitment to integrate classroom” the arts into core subjects and classrooms” “Increased “Teachers better “Teachers gain teacher integrate the skill and comfort proficiency/ capacity to arts into core in art integrate the curriculum” arts” integration” “Children’s original “Teachers have thinking elicited in arts enhanced their ability to experiences will provoke implement arts new understanding on integration into their classroom.” the parts of staff.”
From Portfolio to outcome Criteria Outcome Criteria Criteria
What art skills should students learn? Outcome: “Student have musical skills and knowledge to successfully participate in high school orchestra.” 1. Ability to refine musical work over time. 2. Understanding context and structure of musical work. 3. Demonstrate technical skill with their instrument. 4. Present/perform musical work.
What art skills should students learn? Outcome: “Student have musical skills and knowledge to successfully participate in high school orchestra.” Novice Proficient Advanced 1. Ability to refine musical Criteria Description Description Description work over time. 2. Understanding context and Criteria Description Description Description structure of musical work. 3. Demonstrate technical skill Criteria Description Description Description with their instrument. Criteria 4. Present/perform musical Description Description Description work.
Develop Descriptions For each rubric criteria, consider: • What would a novice be able to do? • What about a student who has mastered that skill? • How would students’ different levels of mastery look different in their work?
Examples Four examples • Generic rubric for program performance • Generic rubric for working with open ended responses • Rubric looking at the big picture of arts skills • Specific example of one arts skills
Generic Rubric Rating Description Excellent Clear example of exemplary performance or best practice in this domain; no weaknesses Very good Very good or excellent performance on virtually all aspects; strong overall but not exemplary; no weaknesses of any real consequence Good Reasonably good performance overall; might have a few slight weaknesses but nothing serious Barely Adequate Fair performance; some serious (but nonfatal) weaknesses on a few aspects Poor Clear evidence of unsatisfactory functioning; serious weaknesses across the board or on crucial aspects From Evaluation Methodology Basics by E. J. Davidson
Open ended response rubric Rating Description Evidence of a strong positive impact: Very positive comments, with a Excellent substantial number that indicated a very strong impact few if any neutral or negative comments Very good Evidence of noticeable positive impact: a good number of positive comments (few neutral or negative), clearly showing that the program had made a noticeable positive effect on students Evidence of some positive impact: a mix of positive and negative Good comments, skewed somewhat towards the positive; evidence pointing in the right direction but not to a very noticeable impact. Limited or no impact either way: a real mix of comments, no clear skew in Barely Adequate either the positive or negative direction Evidence of some negative impact: a mix of positive and negative Poor comments, skewed somewhat towards the negative; not enough evidence to call this a really noticeable negative impact. From Evaluation Methodology Basics by E. J. Davidson
Generic Arts Rubrics Rating Description Advanced Students at the Advanced level independently identify challenging arts problems based on their interests or for specific purposes, and bring creativity and insight to finding artistic solutions. They are able to use at least one art form as an effective avenue for personal communication, demonstrating a higher level of technical and expressive proficiency. Students at the Accomplished level are -- with minimal assistance -- able to identify or solve Accomplished arts problems based on their interests or for a particular purpose; conduct research to inform artistic decisions; and create and refine arts products, performances, or presentations that demonstrate technical proficiency, personal communication and expression. Students at the Proficient level have developed the foundational technical and expressive Proficient skills and understandings in an art form necessary to solve assigned problems or prepare assigned repertoire for presentation. Students at the Intermediate level are continuing study in a chosen specialized art form. Their Intermediate development continues in artistic understanding and technical and expressive skills enabling the student to begin to independently and collaboratively create, perform and respond at their given skill level. Their presentation and performance opportunities in ensembles at school and in the community increase and students actively participate in rehearsals. Novice Students at the Novice level have started specialization in an art form of their choice. They are beginning to develop the basic artistic understanding and technique necessary to advance their skill level. Their expressive skills may be identified and exploratory work begins. They may participate in presentation and performance opportunities as they are able. Adapted from National Core Arts Standards
Recommend
More recommend
