What to Look For: Major Design or Integrating Experience • Evidence all students complete major design, capstone or integrating experience (if criteria require) that draws on previous courses and incorporates standards and realistic constraints. • Student work should demonstrate via a complete project report or some other mechanism. 36
What to Look For: Assessment Materials • Examples of instruments used and actual data collected. • Summaries of the data with results reported in usable form. • Recommendations for program improvement based on data. • Implementation and results. 37
Documentation – APPM II.A.6 • Institutional catalogs and similar publications must clearly indicate the programs accredited by the commissions of ABET as separate and distinct from any other programs or kinds of accreditation. • Each accredited program must be specifically identified as “accredited by the _________ Accreditation Commission of ABET, http://www.abet.org.” • Each ABET- accredited program must publicly state the program’s educational objectives and student outcomes. • Each ABET-accredited program must publicly post annual student enrollment and graduation data per program. Handout: 3-4 38
39
Module 4: Sunday Team Meeting 40
Sunday Team Meeting Agenda • Agenda Review • Visit Schedule • Program Evaluator Pre-Visit Briefings Program Strengths Potential Program Shortcomings Recommended Action • Display Material and Facility Findings • Agree on Common Findings and Plans to Investigate • Handout 4-1 Debrief 41
Actual Visit Sunday Meeting • Usually involves review of multiple programs with one PEV per program. • Discussions will focus on common shortcomings across programs and what team needs to do to ensure consistent evaluation and recommended action. • Usually two meetings: Before review of display materials and facility tours. After review of display materials and facility tours. 42
43
Module 5: Interviews 44
Why Interview? • Direct interface with members of program: Facilitates understanding of program. Provides an opportunity to clarify items in Self-Study and to probe for further details. Effective way to identify problems not addressed in Self-Study. 45
Panel Discussion 46
Advisor Interview 47
Managing Problem Interviews: Exercise In your table groups: • Draw card from scenario card deck. • Share ideas for dealing with situation: What would you say? • Facilitator: Capture ideas on sticky-notes. • Post ideas/responses for each scenario on appropriate flipchart in room. Handout: Interview Cards 48
The Art of Asking Questions • All interview questions should be DESIGNED to elicit detailed response: Determine and clarify objectives BEFORE interview. Prepare questions. 49
The Art of Asking Questions (cont’d) • Begin with: Why? How? What? When? Where? • Avoid leading questions that assume an answer. • Prompt for specific information: “ Show me” 50
The Art of Asking Questions (cont’d) • Turn interviewee into teacher: “Can you explain that to me?” • Ask an obvious question to establish interviewee as expert. • Periodically verify what you heard: “Do I understand you to say…?” • Remember, silence is golden. 51
Student Interviews • Suggest interview of junior/senior level class (~30 minutes without instructor present) • Briefly describe ABET and Comprehensive Review • Possible questions: How well are laboratory facilities and equipment working? How satisfactory and available are computer laboratories? How helpful are enrollment and career advising? How do courses from Chem, Phys, and Math support engineering courses? Handout 5-1 52
53
Module 6: Monday Night Meeting 54
Monday Night Meeting • Each PEV Candidate in turn brief team on potential finding including: Criterion to be cited Evidence gathered Compliance level • Discuss evidence, criteria, and compliance as team and develop consensus for finding. • You have maximum of 60 minutes. 55
Actual Monday Team Meeting • Similarities with simulation • Differences with simulation 56
57
Module 7 Writing Exit Statements 58
Types of Statements Handout 7-1 59
Types of Findings • Statement will generally include one or more types of findings: Strength Deficiency Weakness Concern Observation • Format will vary depending on type of visit: General Review Interim Visit 60
General Review 61
Outline of Exit Statement • Introduction • Program Strengths • Program Shortcomings • Deficiencies (in order of Criteria) • Weaknesses (in order of Criteria) • Concerns (in order of Criteria) • Observations NOTE: ETAC does not require an Exit Statement. The PAF serves as the Exit Statement. Handout 7-2 62
Introduction Statement • Include information such as: Type or special characteristics of program. Emphasis area(s). Number of enrolled students. Size of most recent graduating class. Number of faculty members. Other information that could be helpful to next team. 63
Introduction: Example The industrial engineering BS program prepares students for careers in system design, optimization, and ergonomics. The program has 150 students, and 14 full-time faculty members support the program. The program had 39 graduates in the 2014-2015 academic year. All courses in the major are available by distance education, but students must come to campus or have facilities available through their employers for laboratory experiences in two courses. 64
Program Shortcoming Statements • Contents: Criterion/policy citation What was observed Effect on program 65
Criteria/Policy Citation • Criteria/Policy Citation State which criterion or policy applies Quote excerpts as needed (No quote marks) 66
What Was Observed & Effect • What was observed: Describe what was observed (evidence). Describe how your finding points to lack of compliance with criterion or policy or points to lack of strength of compliance. • Effect: Describe negative effect it has on the program (D or W) with respect to criterion. Describe potential future effect on program (C) with respect to criterion. 67
Deficiency: Example (Part 1) Students Criterion Criterion 1 requires that the program must have and enforce procedures to ensure and document that students who graduate meet all graduation requirements. 68
Deficiency: Example (Part 2) While most students who graduated successfully completed all requirements, there were a few students who did not pass all of the required courses designated by the program. Students can graduate from the program without passing all the required courses. Thus, the program is not in compliance with the Student criterion. 69
Weakness: Example (Part 1) Program Educational Objectives Criterion Criterion 2 requires that there must be a documented, systematically utilized, and effective process, involving program constituencies, for the periodic review of these PEOs that ensures they remain consistent with the institutional mission, the program’s constituents’ needs, and these criteria . 70
Weakness: Example (Part 2) The program has relied on a review of the PEOs every three years by the faculty only. The other constituencies (alumni, employers) were not involved in this review. Without involvement of all constituencies, the program cannot ensure that its PEOs meet the needs of all constituencies. Thus, strength of compliance with Criterion 2 is lacking. 71
Concern: Example (Part 1) Institutional Support Criterion Criterion 8 requires that the resources available to the program must be sufficient to acquire, maintain, and operate infrastructures, facilities, and equipment appropriate for the program, and to provide an environment in which student outcomes can be attained. 72
Concern: Example (Part 2) At present, it appears that resources are adequate to support the program. However, there have recently been large reductions in the operating budget of the program. If these budgetary reductions continue, future compliance with this criterion may be jeopardized. 73
Statement Critiquing: Exercise Handouts 7-3 & 7-4 74
Additional Guidelines for Writing Statements • Must be based on conclusions from evidence found by team to be credible and significant. • Names of individuals or titles that identify individuals must not be used. • Avoid attributing opinions, conclusions or recommendations to other individuals or groups of individuals besides Commission. Poor: The team discovered that … Better: Evidence indicated that … 75
Additional Guidelines for Writing Statements (cont’d) • Provide enough detail that program and institution will know precisely what is inadequate. • Provide enough detail to Commission to justify type of finding. • Provide enough detail that team making the next general or interim review will be able to determine amount of progress that has been made since original finding. 76
Additional Guidelines for Writing Statements (cont’d) • If wording does not add to or clarify finding, do not include it. • Avoid use of acronyms or other abbreviations. • Avoid combining findings on two criteria into one statement. 77
Program Audit Form [PAF] • ONLY document left with institution at end of the visit. • MUST be consistent with Exit Statement. • Level of compliance. • Wording to describe what was observed and effect on program. Handout 7-5 78
Program Audit Form [PAF] (cont’d) 79
Program Audit Form [PAF] (cont’d) 80
Format of Exit Statement • Introduction • Program Strengths • Program Shortcomings Deficiencies (in order of Criteria) Weaknesses (in order of Criteria) Concerns (in order of Criteria) • Observations 81
Reviewing Your Statement • Use statement rubric to carefully review your exit statement before turning in to Team Chair. • Ask another PEV to review statement for clarity and consistency with PAF. • Read statement out loud to your fellow team members. 82
Statement Writing Summary Table Discussion Handout 7-6 83
84
Module 8 Homework 85
Homework Assignment • Read two Judgment Scenarios . • Update your PEV Worksheet to reflect new information from today’s activities: Interviews, display materials, information from Dean, and team discussions. • Draft your Exit Statement following statement format and writing guidelines. Introduction, and shortcomings based on your team consensus for Upper State University. • Complete the Program Audit Form . Handouts 8-1 & 9-1 86
Homework Assignment (cont’d) • You will email completed Exit Statement and Program Audit Form to ABET Staff person whose email address will be provided. • PAF for your commission is available online in Module 7. • Homework will be evaluated by your table Support Facilitator using statement rubric. • We will discuss Judgment Scenarios Sunday morning. Handouts 8-1 & 9-1 87
Day 1, Closing • Day One once-around the room What do you leave with today? What’s “top of mind” about the day ? Thank you for your time, energy, and attention today! 88
Welcome Back! Day 2 PEV F-2-F Training Do the right thing. It will gratify some people and astonish the rest. ~ Mark Twain 89
Agenda – Day 2, Morning Module Name Timing Day 2 Overview 8:00-8:10 a.m. 9) Context for Judgment 8:10-9:00 a.m. 10) Informal Program Debrief and Exit 9:00-10:05 a.m. Meeting Break 10:05-10:20 a.m. 11) Writing Exit Statements 10:20-11:00 a.m. 12) Connection to PEV Competencies 11:00-11:20 a.m. 13) Post-Visit Work 11:20-11:29 a.m. 14) Post-Training 11:29-11:38 a.m. 15) Closing Exercise 11:38-12:15 a.m. 90
Module 9 The Context for Judgment 91
Context for Judgment • Evaluation of program for accreditation requires PEV judgment on extent to which each criterion is satisfied. • PEV judgment is based on evidence obtained during evaluation process. • Judgment was demonstrated in exit statements written last night for Upper State University as discussed yesterday. 92
Context for Judgment (cont’d) Throughout case study: • Where did you use evidence as basis for judgment? • How and when did you see evidence-based judgment applied by others? • What did you struggle with? • Why is evidence-based judgment important? 93
Context for Judgment (cont’d) • Accreditation has moved away from rigid “bean counting” process to one that is based on continuous quality improvement processes. • ABET Criteria are based on principles of Continuous Quality Improvement (CQI). 94
Context for Judgment (cont’d) An educational program CQI process should reflect clear understanding of: • Mission • Constituents • Objectives • Outcomes • Processes (internal practice to achieve outcomes) • Facts (data collection) • Evaluation (interpretation of facts) • Action (feedback to support decision making and improve processes) 95
Judgment Scenario: Exercise Handout 9-2 96
Context for Judgment (cont’d) • Many questions of judgment center around assessment and continuous improvement. • When such issues arise, ask: Are assessment processes adequate to determine achievement of each of outcomes? Are assessment processes robust enough to identify shortcomings and achievements? Are assessment processes sustainable? Will assessment processes lead to program improvements if appropriate? 97
Program Educational Objectives • Definition: Program educational objectives are broad statements that describe what graduates are expected to attain within a few years of graduation. Program educational objectives are based on the needs of the program’s constituencies. • Key Words: Broad Statements Expected to Attain Needs of Constituencies 98
PEOs: Important Concepts • Should be Forward Looking • Should use Expected to Attain language and not Preparing Graduates to Achieve • Must periodically review PEOs • Must ensure PEOs remain consistent with institutional mission, program’s constituents’ needs, and the criteria 99
Summary • Judgment centers on observed evidence that supports achievement of each criterion. • When issues arise about specific components of criteria, ask: Does evidence support my decision? What additional evidence could I seek? 100
Recommend
More recommend
