NAEA Museum Education Division Research Marathon Practical Strategies for Getting Started and Making it Happen #NAEAMusEd16 @NAEAMusEd
Developing a Shared Vocabulary Visitor Studies “The interdisciplinary study of human experiences within informal learning environments. Visitor studies. . . • Follow rigorous methods that adhere to the standards of the social sciences • Draw from and contribute to the theory and practice of social science • Are designed to improve the practices of learning in informal environments.”* Evaluation Research Tests feasibility, effectiveness, impact, ● Tests hypotheses ● or value Seeks to add knowledge to/develop ● Seeks to inform decisions, policies, or ● theory in a given field program development May have an experimental – ● Uses social science research methods “treatment-control” - design * Visitor Studies Association, http://www.visitorstudies.org/glossary-of-terms, and The Definitions Project, http://www.definitionsproject.com/definitions/media/definitions_list_0107.pdf
Presenters Lynn Courtney Head of Planning & Evaluation, Museum of Fine Arts Boston Cecilia Garibay Principal, Garibay Group Jennifer Wild Czajkowski VP Learning & Interpretation, Detroit Institute of Arts
Session Goals Participants will... ● understand a variety of research / evaluation models for a variety of situations ● feel confident and inspired to engage in next-level conversations about research and evaluation in their institutions ● be intrigued to attend the rest of the marathon!
Lynn Courtney Head of Planning and Evaluation #NAEAMusEd16 @NAEAMusEd
Developing a Shared Vocabulary Visitor Studies “The interdisciplinary study of human experiences within informal learning environments. Visitor studies. . . • Follow rigorous methods that adhere to the standards of the social sciences • Draw from and contribute to the theory and practice of social science • Are designed to improve the practices of learning in informal environments.”* Evaluation Research Tests feasibility, effectiveness, impact, ● Tests hypotheses ● or value Seeks to add knowledge to/develop ● Seeks to inform decisions, policies, or ● theory in a given field program development May have an experimental – ● Uses social science research methods “treatment-control” - design * Visitor Studies Association, http://www.visitorstudies.org/glossary-of-terms, and The Definitions Project, http://www.definitionsproject.com/definitions/media/definitions_list_0107.pdf
In-House Evaluation from Soup to Nuts A Case Study: Meeting with the “clients” Evaluation questions: ➲ How satisfied are Playdates families? What works? What doesn’t? ➲ What are toddlers learning? ➲ What should we change? ➲ 7
In-House Evaluation from Soup to Nuts A Case Study: Designing the study Observations Exit Interviews? Survey Cards? Who? When? How many? Online surveys? 8
In-House Evaluation from Soup to Nuts A Case Study: Designing the study Observations Parents and caregivers Twice a month At least 75 Online surveys 9
In-House Evaluation from Soup to Nuts A Case Study: Collecting the Data ➲ Onsite recruitment Online survey link sent ➲ within 1 week of visit 2 follow-up emails ➲ 10
In-House Evaluation from Soup to Nuts A Case Study: Collecting the Data Timeline Planning & Data Final survey design Data collection analysis report September October April May June 11
In-House Evaluation from Soup to Nuts A Case Study: Data analysis & reporting: what did we learn? ➲ 69% of Playdates children are 2 years old or younger 12
In-House Evaluation from Soup to Nuts A Case Study: Data analysis & reporting: what did we learn? ➲ High satisfaction levels ➲ But. . .Younger children were not always engaged; elements were not always well-integrated; parents want more help with art making 13
In-House Evaluation from Soup to Nuts A Case Study: Data Analysis & Reporting: “My daughter STILL talks about the crown she made at an MFA what did we learn? Playdates last summer. She thinks the Chihuly sculpture looks like ➲ We identified indicators of celery.” learning in young children “She's young, but she was looking and pointing at the artwork.” “The next day my daughter sang every line of a song the instructor sang!” 14
In-House Evaluation from Soup to Nuts A Case Study: Disseminating Results ➲ Meet with program staff: give them usable results ➲ Evaluation Highlights ➲ Present at Museum staff meetings 15
In-House Evaluation from Soup to Nuts A Case Study: What did we change? ➲ Professional development for gallery educators ➲ Design for 2-year-olds and adapt for older children Work with the guards ➲ Stronger connections: theme, ➲ art, gallery activity, art making 16
Resources and Capacity Each year we conduct… With… ➲ 1 full-time evaluator ➲ 2 – 4 major studies ➲ 2 work-study students at ➲ 4 – 6 smaller studies 14 hours per week ➲ 1 paid summer intern at ➲ 1 major planning 14 hours per week process ➲ 1 summer Evaluation ➲ Various ad hoc Assistant at 14 hours projects per week 17
When to Call in an External Evaluator ➲ When there is funding ➲ When you need specialized skills ➲ To boost credibility ➲ When there is a large, ongoing project that is beyond your capacity 18
Expanding Uses of Evaluation Cecilia Garibay Principal, Garibay Group #NAEAMusEd16 @NAEAMusEd
Summative Remedial Formative Front-End Concept Development Post-Opening Preview Full Implementation Launch
Listening to Community Voices • Evaluation as a process for creating accessible, relevant places and experiences • Evaluation as deeply collaborative process
• How will what we learn help you make better decisions for the specific project/your organization? • How will this study help you better serve your visitors and community members?
• Offered deeper interpretation and historical context. • Made changes to placement of artwork from original plan. • Added response/reflection areas. • Issues raised by participants became talking points in interpretation. (A shift in who gets to tell the story!)
Examine processes to ● identify factors that lead to . outcomes Articulate interrelationships ● Develop models ●
“It was a wonderful experience. We can talk about one picture with other classmates and we can try to discuss pictures with the vocabulary we had…. Somehow I feel very connected to each other.” “When you have friends around you it Empowerment helps to motivate [you] to talk about the the picture… sometimes you want to add something…and you also learn from what other people say.” Sharing “I found that when you are sharing ideas about art it doesn’t matter the right or English wrong because it’s your feelings and it’s Community Skills your opinion. And through this program— I think we feel free—I don’t worry about right or wrong.”
I know what this is and want I don’t know what this is but want to Interesting to know more. know more. Unfamiliar Familiar Uninteresting I know what this is and don’t want to I don’t know what this is but don’t want to know more. know more.
Characteristics Characteristics that afford engagement that impede engagement Scale Ordinariness ● ● Evidence of age Dense, small type (documents) ● ● Details Archaic type (documents) ● ● Intriguing mysteries Formal language or jargon ● ● Tangible expression of a time Small size ● ● Relationship to a real person Difficult topics ● ● Readily identifiable, but unique ● Representative of a specific Chicago place ● Connection to a well-known event/story ● Select opportunities based Select strategies to on characteristics overcome impediments Discuss conservation process Emphasize uniqueness of ordinary objects ● ● Help explore details Highlight important parts of document ● ● Discuss how to solve mysteries Translate formal language ● ● Create a feeling of time Call attention to small items ● ● Provide interpretation related to difficult topics ●
• These kinds of evaluations are collaborative endeavors between museum team, external evaluators, visitors, and community members. • These uses of evaluation provide significant opportunities to deepen organizational learning beyond one program/exhibit. • They require a willingness to engage in a deep, reflective process.
Jennifer Wild Czajkowski VP Learning & Interpretation #NAEAMusEd16 @NAEAMusEd
Research with Universities Possible partnership formats: find academic partner who teaches a research class ● course buyout: pay university to keep instructor out of class ●
Research with Universities: Finding a Partner NAEA resources for museum / higher ed partnerships: Professional Learning Through Research working group ● Higher Ed leadership/membership ● Research Commission Interactive Cafe ● Future projects? NAEA database of higher ed members with research interests ●
Recommend
More recommend
