Andrea M. Karkowski, Jody S. Fournier, Richard M. Ashbrook, Terry D. - PowerPoint PPT Presentation

Andrea M. Karkowski, Jody S. Fournier, Richard M. Ashbrook, Terry D. Lahm Higher Learning Commission Annual Conference ● Chicago, IL ● April, 2013

 Identify Diverse Views on Assessment and Learning  Develop Student Learning Goals Programs/Majors  Align Student Learning Goals to IDEA Objectives  Map IDEA Objectives Across Required Courses  Help Students Improve their Learning  Document the Results  Use Results to Improve Courses and Curricula

Photo by A. M. Karkowski of Dr. Terry Lahm, September 17, 2012, Capital University

Photo by A. M. Karkowski of Lanee Williams, Jordan Aires, Jessica Heck, Andre Cousin, Capital University, September 17, 2012

Janet Leigh in Psycho, Directed by Alfred Hitchcock, Universal Pictures

 Direct Measures  What students learned or can actually do  More difficult assessment to do and more highly valued  Example: Capstone Project  Indirect Measures  What students and faculty think students learned  Easier assessment to do and undervalued  Example: NSSE and LSSE

How can the IDEA survey be used as an indirect assessment measure?

OBJECTIVE 1: Gaining factual knowledge (terminology, classification, methods, trends) OBJECTIVE 2: Learning fundamental principles, generalizations, or theories OBJECTIVE 3: Learning to apply course material (to improve thinking, problem solving, and decisions) OBJECTIVE 4: Developing specific skills, competencies, and points of view needed by professionals in the field… OBJECTIVE 5: Acquiring skills in working with others as a member of a team OBJECTIVE 6: Developing creative capacities (writing, inventing, designing, performing in art, music, drama) OBJECTIVE 7: Gaining a broader understanding and appreciation of intellectual/cultural activity… OBJECTIVE 8: Developing skill in expressing oneself orally or in writing OBJECTIVE 9: Learning how to find resources for answering questions or solving problems OBJECTIVE 10: Developing a clearer understanding of, and commitment to, personal values OBJECTIVE 11: Learning to analyze and critically evaluate ideas, arguments, and points of view OBJECTIVE 12: Acquiring an interest in learning more by asking questions and seeking answers

How do we develop student learning goals for programs and majors?

Develop Learning Goals for Programs and Majors

University Learning Department Student APA Recommendations Goals Learning Goals (Halonen et al., 2006) Develop a sense of Weigh evidence, tolerate values that guides Sensitivity to ethical ambiguity, act ethically, and personal decision issues reflect other values that are the making underpinnings of psychology Recognize, understand, and Recognition of ethnic, Develop intercultural respect the complexity of cultural, social and competencies sociocultural and international gender diversity diversity

Students will develop… A broad and deep knowledge base incorporating both social and natural  science aspects of psychology Understanding of different theoretical traditions  Methodological competence in statistics, research design and assessment  methods including the use of technology for these purposes Practical research, laboratory, or field experience to generalize a scientific  style of thought to the realm of application Effective communication skills  Sensitivity to ethical issues  A recognition of ethnic, cultural, social and gender diversity 

What is the connection between program or major student learning goals and IDEA Objectives?

(HANDOUT) DEPARTMENT IDEA OBJECTIVE LEARNING GOAL  A broad and deep knowledge  Objective 1 : Gaining factual base incorporating both knowledge (terminology, social and natural science classification, methods, aspects of psychology trends)  Understanding of different  Objective 2 : Learning theoretical traditions fundamental principles, generalizations, or theories

(HANDOUT) DEPARTMENT IDEA OBJECTIVE LEARNING GOAL  Methodological competence  Objective 4 : Developing in statistics, research design specific skills, competencies, and assessment methods and points of view needed by including the use of professionals in the field most technology for these closely related to this course purposes  Objective 9 : Learning how to find resources for answering questions or solving problems

(HANDOUT) DEPARTMENT IDEA OBJECTIVE LEARNING GOAL  Practical research,  Objective 3 : Learning to laboratory, or field apply course material (to experience to generalize a improve thinking, problem scientific style of thought solving, and decisions) to the realm of application  Objective 11 : Learning to analyze and critically evaluate ideas, arguments, and points of view

(HANDOUT) DEPARTMENT IDEA OBJECTIVE LEARNING GOAL  Effective communication  Objective 5 : Acquiring skills skills in working with others as a member of a team  Objective 8 : Developing skill in expressing oneself orally or in writing

(HANDOUT) DEPARTMENT IDEA OBJECTIVE LEARNING GOAL  Sensitivity to ethical issues  Objective 10 : Developing a clearer understanding of, and commitment to, personal values  A recognition of ethnic,  -- NONE -- cultural, social and gender diversity

Three IDEA Objectives Do Not Align with Department Learning Goals  Objective 6: Developing creative capacities (writing, inventing, designing, performing in art, music, drama)  Objective 7: Gaining a broader understanding and appreciation of intellectual/cultural activity (music, science, literature, etc.)  Objective 12: Acquiring an interest in learning more by asking questions and seeking answers

How do we know which IDEA objectives are addressed in which courses for a program or major?

(HANDOUT) Required Courses for Psychology Major IDEA Objective 100 110 200 210 220 300 310 320 330 340 350 370 410 1. E E I E E E E I I 2. E I E I E E E E 3. I I E E E I I E I I E E 4. I E E E I E I E I 5. I I 6. This IDEA Objective is not aligned with department learning goals and thus is not assessed. 7. This IDEA Objective is not aligned with department learning goals and thus is not assessed. 8. E E E I I 9. E I 10. I I I 11. I E I I I E 12. This IDEA Objective is not aligned with department learning goals and thus is not assessed.

Photo by A. M. Karkowski of M. Torello, S. Sagaria, J. Fournier, J. Loopstra, S. Saleem, K. Bell, September 17, 2012

How do we ensure that students understand the connections between course activities and learning goals and objectives?

 Dr. L. Dee Fink  Capital Alumnus  Creating Significant Learning Experiences  Emphasized making explicit the link between learning goals and course activities

(HANDOUT) How the Learning Learning Department Goal IDEA Objective Objective Specific Course Activity Objective is to Course Assessed Students demonstrate IDEA Objective 4 : • Develop • Participation methodological Developing specific • APA Style Paper Research competence in skills, competencies, Design, conduct, Studies (Method) statistics, research and points of view and analyze design and assessment needed by empirical research methods including the professionals in the studies to test a use of technology for field most closely hypothesis • Collect and • APA Style Paper these purposes related to this course Analyze Data (Results) • Complete IRB IDEA Objective 10 : CITI Training Developing a clearer Apply ethical Students demonstrate • Research Ethics • IRB Protocol sensitivity to ethical understanding of, reasoning in a Discussion Assignment issues and commitment to, research context • Research Ethics personal values Class Activity

Metacognition Understanding Learning Goals  Linking New and Prior Learning  Building Knowledge Structures that Facilitate Future Learning  Student View of Metacognition Requires Work  Benefits of Metacognition Metacognition  Academic Success (Gourgey, 1998; Nietfeld et al., 2005) 

Teach Students About Course and Department Learning Goals and Objectives  How Learning Goals are Themes Across the Curriculum  Syllabi for Required Psychology Courses Include a Map the Department Learning Goals for Required Courses  Seemingly Different Courses Can Achieve the Same Learning Goal  Identify when Course Activities Target Particular Learning Goal Understand that Learning Goals Are Achieved by Accumulation of Many  Different Learning Activities

When students have built and can use the metacognitive structures, ratings of the IDEA Objectives then provide one way to assess whether students perceive that learning goals are being met.

How do we document?

Assumptions  Our primary concern is whether or not students are achieving the course objectives  A single data point (i.e., a data from a single class) is not useful without the larger context

Student Ratings of Meeting Course Objectives (page 2 of the IDEA Feedback Form)

Excel Spreadsheet  http://www.capital.edu/psychology-resources/  Can tailor to individual classes  Can graph classes over time

(HANDOUT) Directions Three Tabs

(HANDOUT) Course Number IDEA Objectives