Preparing an Item for Consensus Meeting 1. Review the Criteria questions 2. Assign 4-6 key factors 3. Read applicant’s response 4. Review team members’ Independent Reviews SYNTHESIZE WRITE SCORE REVISE PREPARE Synthesize team Draft 6-10 feedback Score the item based Revise comments Prepare to present members’ IR into comments based on on your consensus based on feedback your items during 6-10 comment topics your chosen topics review from your backup the team’s consensus meeting Develop rationale: Use TEST Provide feedback to B/O/M, evaluation your partner factors, and initials 28
Individual Exercise Set your timetable: (pgs. 93-97) • Review Criteria questions 2 min. • Identify 4-6 relevant key factors 5 min. • Review your homework and the team’s IR to find commonalities and conflicts for your assigned item 30 min. • Select 6 topics (most important key messages) to become strength and OFI comments • Note your rationale for each comment • Write 2 comments (1 strength + 1 OFI) 15 min. • Score the item 5 min. 29
Preparing an Item for Consensus Meeting 1. Review the Criteria questions 2. Assign 4-6 key factors 3. Read applicant’s response 4. Review team members’ Independent Reviews SYNTHESIZE WRITE SCORE REVISE PREPARE Synthesize team Draft 6-10 feedback Score the item based Revise comments Prepare to present members’ IR into comments based on on your consensus based on feedback your items during the 6-10 comment topics your chosen topics review from your backup team’s consensus meeting Develop rationale: Use TEST Provide feedback to B/O/M, evaluation your partner factors, and initials 30
Exercise • Pass your comments to the person to your right • In the right-hand column, give feedback to your teammate: – Is the comment Criteria based? – Can you understand it? What would improve it? – What is the topic sentence? – Are there examples from the application? – So what? • As a table, agree on the item score • Agree on one key learning to share 32
33
Overall and Multiple Questions
In Which Scoring Range… (Le) Must the applicant’s results be responsive to the multiple questions? (T) Are “some” adverse trends acceptable? (C) Is the applicant in the “early stages” of obtaining comparative data? (I) Are projections required? 35
“ If everyone is moving forward together, then success takes care of itself.” Henry Ford CONSENSUS MEETING
The Importance of Teamwork 37
What is Consensus? • Everyone can support the team’s decision. • When you leave the meeting will you say “we decided” or “they decided”? 38
Preparing an Item for Consensus Meeting 1. Review the Criteria questions 2. Assign 4-6 key factors 3. Read applicant’s response 4. Review team members’ Independent Reviews SYNTHESIZE WRITE SCORE REVISE PREPARE Synthesize team Draft 6-10 feedback Score the item based Revise comments Prepare to present members’ IR into comments based on on your consensus based on feedback your items during the 6-10 comment topics your chosen topics review from your backup team’s consensus meeting Develop rationale: Use TEST Provide feedback to B/O/M, evaluation your partner factors, and initials 39
Purpose of Consensus • Reach agreement on: – Key factors – Strengths and OFIs – Scores • Agree on site visit issues • Begin to develop key themes 40
Sample Roles Examiner Item Lead Back-up Other 1 1.1, 1.2, 7.4 6.1, 6.2, 7.1 Team leader Scorebook 2 2.1, 2.2 5.1, 5.2, 7.3 editor Back-up team 3 3.1, 3.2, 7.2 4.1, 4.2, 7.5 leader 4 4.1, 4.2, 7.5 3.1, 3.2, 7.2 Criteria cop 5 5.1, 5.2, 7.3 2.1, 2.2 Arrange logistics 6 6.1, 6.2, 7.1 1.1, 1.2, 7.4 Time keeper 41
Topics for Consensus Meeting • Changes to the language of a comment that affect: – The meaning of the comment – The A-D-L-I or Le-T-C-I assessment factors that pertain – Basic/Overall/Multiple level of a comment – Distribution of strengths and OFIs at B/O/M – The score – Key themes • Comments that are not “key” or Criteria-based • If a “key” area to address is missed • Conflicts between strengths & OFIs, or comments in different categories • Potential site visit issues 42
NOT Topics for Consensus Meeting • Anything that does not alter the meaning of a comment or the score • Grammatical errors – Typos – Misspellings – Word format changes – Extra words or words left out – Sentences or clauses that are redundant – Substituting acronyms for entity names, and the reverse – Separating the clauses of a lengthy sentence to make shorter sentences. 43
Exercise 44
Tips for Consensus Meeting • Be prepared • Don’t be defensive – check your ego at the door! • Begin with an ice breaker • Set ground rules – Does silence mean agreement? • Use a round robin approach to be sure everyone’s voice is heard • Only discuss areas of disagreement 45
“If you just focus on the smallest details, you never get the big picture right.” Leroy Hood KEY THEMES
Key Theme Overview • Executive summary of feedback report for applicant’s senior leaders • Strategic in nature • Summarize the most vital issues for the organization as a whole – e.g., what must the organization do to become/remain competitive and ensure long term sustainability? • Typically prepared by team leader or scorebook editor with support from team members • Used by examiners and judges to get overview of applicant maturity 47
What Is a Key Theme? • A synthesis of multiple comments – Don’t just cut and paste • A perception or observation that: – Recurs and is expanded upon throughout the scorebook – Is common (cross-cutting) to more than one item/category – Is of high value or significance to the applicant – May address a core value of the Criteria – Is traceable to individual item comments 48
1.1 – PEP and + annual goal plans, Communication processes Key Theme 2.1 – deployment of + strategy to workforce Valuing Employees 5.2 – Leadership + System reinforces open culture and engagement 49
Key Themes Reflect a. The most important strengths or outstanding practices (of potential value to other organizations) found in processes b. The most significant opportunities, concerns, or vulnerabilities found in processes c. The most significant strengths (related to data, comparisons, and integration) found in results d. The most significant opportunities, vulnerabilities, and/ or gaps (related to data, comparisons, and integration) found in results 50
Sample Key Theme (a) LOTS has implemented multiple approaches that demonstrate that it values its employees, strengthening the organization’s core competency of a mission-driven workforce. Beginning at the top, the leadership team models ethical behavior and uses multiple touchpoints to communicate with the workforce. Workforce members’ engagement in the success of the organization is supported through the cascading of strategic goals and by addressing their needs via various benefit-plan options and promoting a safe operating environment. Additionally, LOTS has designed the Learning and Development System to improve workforce learning and then uses multidisciplinary meetings and Rounding for Outcomes to capture and disseminate valuable workforce knowledge. All of these workforce-focused approaches highlight the philosophy “We take care of ‘Our People’ so they can take care of others.” 51
Steps in Developing Key Themes 1. Capture initial impressions on your first read- through of the application (IR) 2. Use the “key” icon to flag potential key themes during consensus review 3. Scorebook editor or other team member develops draft topics for key themes for team review prior to consensus meeting and discussion during consensus meeting 4. TL and scorebook editor finalize key themes after item comments are finalized (after site visit) 52
Core Values as Key Themes • Core values may become the topic of a key theme strength: – Use core values to provide value – as the basis of a strength key theme comment – Do not use core values to assess the applicant – NOT as the basis of an OFI key theme comment 53
Other Sources of Key Themes • Process items – A-D-L-I evaluation factors – ++ and – comments • Results items – Le-T-C-I evaluation factors – ++ and -- comments • Scoring guidelines • Scoring band descriptors 54
Class Discussion Does LifeBridge demonstrate any of the Baldrige core values? What other potential key themes might be relevant for LifeBridge? 55
Table Exercise Instructions Using the Consensus Scorebook, assist your table anchor in drafting a key theme on the Key Themes Worksheet (p. 115-117) Key Table Theme Where to Look Suggested Topics • 1 & 5 A Process Strengths Systems Perspective (Cat 1-6) • Data and Information processes, Integration • 2 & 6 B Process OFIs Missing cycles of learning for key processes (Cat 1-6) • Lack of processes to promote innovation and intelligent risk-taking • 3 & 7 C Results Strengths Favorable comparisons for many key results (Cat 7) • Near perfect performance levels for most important measures • 4 & 8 D Results OFIs Missing results (Cat 7) • Adverse trends 56
“What people say, what people do, and what they say they do are entirely different things.” Margaret Mead SITE VISIT PREPARATION
Examiner Evaluation Cycle Complete Pre-work & Training Finalize Independent Feedback Review Report Site Visit Consensus Review Review 58
Site Visit Review Review for Conflict Click “Done”! • Criteria • Key Factors • Analyze: PROCESS Prepare Log on to the system Consensus items (ADLI) 12 process • 6-10 Observations items • Score Participate in Consensus meeting Review the Framework • Criteria Prepare Site Visit • Key Factors issues Read the Application • Analyze: RESULTS (LeTCI) 5 results Complete Site Visit • 6-10 Observations items Create Key Factor List • Score Prepare Final Scorebook 59
Purpose of a Site Visit • To learn more about the applicant – Not just to get your questions answered! • To clarify OFIs • To verify role model strengths • To allow the applicant to tell its story • To meet the applicant’s expectations 60
Site Visit Phases Planning On Site Post Site Visit 61
Site Visit Phases Planning On Site Post Site Visit 62
Prior to Planning Phase… • Begin logistics as soon as teams are formed – Dates of availability for team – Dates of availability for applicant – Travel arrangements • Potential site visit dates: – September 15 – October 4 63
Stair Steps to Excellence X X 64
Planning Reread Org Profile/ Key Factors, and Key Themes to get grounded Phase Discuss Site Visit agenda and approach during consensus meeting Watch site visit video Develop site visit issues and complete Scorebook Navigator form, SVI worksheets Send site visit issues to applicant 10-14 days prior to site visit 65
Site Visit Video 66
Selecting Site Visit Issues • Use site visit issues to – Verify role model strengths and – Clarify OFIs • How many site visit issues do I need? – As a general rule, prepare a site visit issue for each OFI and double strength (++) – Your team leader will assist you in determining the correct number of site visit issues based on • the length of the site visit • the size of the organization 67
Sample Site Visit Issues “Verify LOTS’ use of the Workforce Planning Process (Figure 5.1-1) to address workforce change management.” “Clarify how LOTS ensures its workforce represents the diverse ideas and cultures of its hiring communities.” 68
69
70
Complete the Form in Scorebook Navigator • The comment and rationale will auto-populate • Enter any documents you would like to review • Enter whom you will interview • Write ONLY the SVI question (verify/clarify) in the “potential questions” box • You will complete the “Findings” box later • Only complete this form for the comments you have selected as site visit issues! 71
72
73
74
Next Steps • Once team members have completed the site visit forms in Scorebook Navigator, the team leader will pull a report including: – Requested Documents – Potential Interviewees – SVI Questions (Verify/Clarify) • Team leader edits and formats this list 10-14 days prior to the site visit • Team members then go back to prepare questions on the SVI Worksheet (Word) 75
The Site Visit Issue Worksheet • Complete a separate Site Visit Issue (SVI) Worksheet for each site visit issue identified – Download template from TNCPE website – Sample SVI for Item 5.1a(2) on pg. 133 • Complete the entire front page of the worksheet prior to site visit: – Site visit issue (verify or clarify statement) – Comment(s) affected (cross-walk to Scorebook Navigator) – Strategy (A-D-L-I or Le-T-C-I questions) • You will use the worksheet to conduct your interviews and take notes on site 76
What Questions Should I Ask? • Use your process evaluation factors (A-D-L-I) to “move the applicant to the right” • If your site visit issue clarifies the applicant’s approach , be prepared with follow-up questions regarding deployment, learning, and integration. • This approach works for results too (Le-T-C-I) most of the time. 77
Let’s Review… • Complete SVI form in Scorebook Navigator first – Documents, interviewees – Site Visit Issue only (verify/clarify questions) • Then complete a SVI worksheet in Word for each issue • The applicant will receive your site visit issues (verify/clarify statements) and requested documents but not your prepared A-D-L-I questions or strategy • Prioritize your SVI worksheets in case you run out of time 78
Assignment Table Item 1 3.2 2 4.2 3 3.2 4 4.2 5 3.2 6 4.2 7 3.2 8 4.2 79
Table Exercise • Review Criteria questions, key factors and comments for your item. • Using the Consensus Scorebook, make note of key processes LifeBridge uses to address Criteria questions. • Identify one OFI site visit issue and complete: – The SVI form in Scorebook Navigator (poster) • Documents • Interviewees • SVI question (verify/clarify) – The SVI Worksheet in Word (poster) • A-D-L-I questions 80
Remember… Trust the process! 81
What about Results Items? • Process works the same way • Identify SVI Issues • Complete SVI form in Scorebook Navigator – Documents – Interviewees – Verify/Clarify Questions • Complete SVI Worksheet (Word) – Still “move to the right” (Le-T-C-I) WHERE APPROPRIATE – You may discover lots of your “missing results” 82
Without continual growth and progress, such words as improvement, achievement, and success have no meaning. Benjamin Franklin THE SITE VISIT
Site Visit Phases Planning On Site Post Site Visit 84
On-Site Opening Meeting Phase Category interviews Review of documents Team caucus Closing Meeting Remember, after site visit, “it is not clear” means you didn’t do your job! 85
Sample Level 3 Site Visit Schedule Day 1 Day 2 Follow-up Category 1 8:00 am Opening Meeting Follow-up Category 6 Category 1, Results Follow-up Category 2 9:30 am Category 6, Results Follow-up Category 3 11:30 am Lunch/Caucus Lunch/Caucus Category 2, Results Follow-up Category 4 Noon Category 3, Results Follow-up Category 5 Category 4, Results Visit other sites; review documentation 2:00 pm Category 5, Results Final Caucus 4:00 pm Examiner Caucus Closing Meeting 86
Opening Meeting • Introductions are tricky • Team Leader presents TNCPE Opening Meeting slides • Applicant may have a presentation • Come ready to work! 87
Site Visit Interview Tips • If the applicant gives examples or anecdotes, ask a follow up question • At the end of an interview ask “Is there anything else you’d like to tell us?” • Follow process questions with corresponding results questions • Use evaluation factors (A-D-L-I and Le-T-C-I) to guide your questions • Interview in pairs: one person conducts the interview and the other takes notes • Be committed to be an examiner for the entire site visit 88
Ask Open-Ended Questions • Tell me how it works • How often do you do it? Who participates? • Walk me through . . . • Show me how . . . • How has it changed? Have you always done it this way? • How do you know? (vs. Do you know?) (Be prepared to ask your questions more than one way) 89
Assignments Table Item 1 1.1 2 2.2 3 3.1 4 6.1 5 1.1 6 2.2 7 3.1 8 6.1 90
Exercise – Site Visit Part 1 • Individually, s elect an OFI from the Consensus Scorebook for your assigned item – Try not to double up! Make sure all OFIs are covered around the table. • Complete the Scorebook Navigator form (pg. 142) – Documents to review – Who to interview – Site Visit Issue (Verify/Clarify question) • Complete a Site Visit Issue Worksheet with questions you will ask the applicant (pg. 141) – A-D-L-I 91
Exercise – Site Visit Part 2 • Take turns interviewing your table anchor using your SVI worksheet • Choose a back-up to take notes • Listen to other site visit interviews for information that may impact your SVI • Table anchors—give feedback to the interviewer. Consider question phrasing and interview style. - Share best practices • Save for the next exercise! 92
“Beginning in itself has no value, it is an end which makes beginning meaningful, we must end what we begun.” Amit Kalantri AFTER THE SITE VISIT
Site Visit Phases Planning On Site Post Site Visit 94
Post-Site Enter site visit findings in Scorebook Navigator Visit Revise comments and key themes Phase Complete score summary form Complete summary of sites visited form Complete Award Recommendation form 95
Site Visit Findings • Use your SVI Worksheet (Word) to take notes during site visit • Team leader will collect these when complete (and save in case there are questions) • Document your evidence/findings in the Scorebook Navigator form with a conclusion: – OFI remains – Delete OFI – OFI becomes a strength 96
97
98
Possible Findings • Keep OFI, no change • Edit OFI – change from approach to deployment. They demonstrated an approach but workgroups xyz are not using it yet. • Change OFI to strength – process is in place, fully deployed, with cycles of learning. • Delete OFI – evidence on site presented to show this has been addressed. – Question: Can you identify a new OFI using ADLI, LeTCI, or B/O/M? • Don’t forget to save! 99
100
Recommend
More recommend
