How Do I Evaluate Process Items? Process items are evaluated using four factors: • Approach • Deployment • Integration ADLI • Learning
Approach (A) “Approach” comprises • Methods used to carry out Process • Appropriateness of methods vs requirements • Effectiveness of the methods • The degree to which the approach is repeatable and based on reliable data/information (i.e., systematic) • Common Language (CL): HOW do you address the question
Deployment (D) “Deployment” is the extent to which • Your approach addresses item requirements relevant and important to the organization • Your approach is applied consistently • The approach is used by all appropriate work units • CL: Who needs to understand the approach
Learning (L) Learning comprises • The refinement in approach; cycles of evaluation/improvement • The encouragement of breakthrough change through innovation, and, • The sharing of refinements and innovation with other work units and processes in the organization • CL: Are we getting smarter/better?
Integration (I) “Integration” is the extent which • Your approach is aligned with organizational needs (OP/ processes) • Your measures, information, and improvement systems are complementary across processes and work units • Your plans, processes, results, analyses, learning, and actions are “harmonized” across processes/work units to support goals • CL: Agreement/Alignment with VMVG
Category 7 Examination
Category 7 Evaluated by LeTCI Results: 45% of the Total Score Levels – Le Trends – T Comparisons – C Integration - I
Results Evaluation Factors LeTCI • Levels – Current performance; meaningful measurement scale • Trends – Numerical data; shows the direction/rate of improvements (slope over time) • Comparisons – Performance relative to that of other appropriate organizations, competitors or organizations that are similar; relative to industry leaders/benchmarks • Integration – Measures (segmentation) addressing important performance requirements relating to customers, products/services, markets, processes, or action plans identified in OP and in process items; future performance; harmonization across processes and work units to support organization-wide goals
Segmentation Segments can be defined by, among other things: ▪ Customers (students/families for Education; patients/families for HC) ▪ Market/Product offerings ▪ Location ▪ Workforce group (employees, tenure, admin, hourly, etc.) ▪ Size of the group in question The applicant defines their segments (OP – P1); we assess the results of those segments
Comparisons Comparisons can be defined by, among other things (Organizational Profile OP – P2 Competition): • Inside the industry • Competitive comparisons • Outside the industry
QTF/Baldrige Examination 6 Step Process TRUST THE PROCESS! (KEY TEACHING POINT)
OP – The Preliminary Step • Organizational Profile, Eligibility Forms • Reader’s digest version of application • Sets stage for what the applicant says is important • Do not read into or take away • Develop Key Factors from what the applicant says is important
Step 1: Read the Criteria. Read Baldrige Framework for Award Level QTF Pioneer, Engagement, Commitment, or Progress Level Criteria
Step 2: Determine Most Relevant Key Factors Four to Six Key Factors taken from OP, Eligibility, or from Application
Step 3: Read & Analyze the Application Read the Application Mark as appropriate
Step 4: Identify Strengths/OFIs Around 6 comments Strengths and OFIs
Step 5: Write Feedback Ready Comments Remember: NERD! N – nugget of importance E – evidence/example to support comments R – relevance to the applicant D – Done!
Step 6: Determine the Scoring Range and Score Best Fit Don’t Block a Winner Tie goes to applicant
Scoring: Not like your previous education! Go to Process Scoring Page 34
UNDERSTANDING SYSTEMATIC APPROACH
UNDERSTANDING DEPLOYMENT
UNDERSTANDING IMPROVEMENT/LEARNING/INNOVATION
UNDERSTANDING INTEGRATION
Go to Results Scoring Page 35 What is different about Results Scoring?
Questions to this Point??
So let’s begin our Examination/Evaluation Independent Review We will follow the exact same steps that you will follow
Preliminary Step: Importance of the Organizational Profile • Reader’s digest version of the application • Sets the stage for what the applicant says is important • Frames our comments (feedback report), strengths/OFIs, scoring, and Key Themes • Assists with our scoring band descriptors • Do not read into or take away
How to Determine Key Factors • About 18-20 KFs • Obtained from OP, Eligibility Form, and Application • What is important? – OP – Eligibility Form – Application • Do not rewrite the OP • Walk through with me
P.1 Organizational Descriptions: What are your key organizational characteristics? In your response, answer the following questions: P.1a Organizational Environment (1) Product Offerings What are your main HEALTH CARE SERVICE offerings? What is the relative importance of each to your success? What mechanisms do you use to deliver your HEALTH CARE SERVICES? (2.0, 3.0, 6.0, 7.1) (2) MISSION, VISION, and VALUES What are your stated MISSION, VISION, and VALUES? What are your organization’s CORE COMPETENCIES, and what is their relationship to your MISSION? (1.0, 2.0, 7-1-7-5)
6 6 P.1 Organizational Descriptions: What are your key organizational characteristics? (3) WORKFORCE What is your WORKFORCE profile? What recent changes have you experienced in WORKFORCE composition or your WORKFORCE needs? (5.0, 7.3) • What are your WORKFORCE or employee groups and SEGMENTS, the educational requirements for different employee groups and SEGMENTS, and the KEY drivers that engage them in achieving your MISSION and VISION? (1.0, 2.0, 5.0, 7.3) • What are your organized bargaining units (union representation)? What are your organization’s special health and safety requirements? (4) Assets What are your major facilities, technologies, and equipment?
P.1 Organizational Descriptions: What are your key organizational characteristics? (5) Regulatory Requirements What is the regulatory environment under which you operate? What are the KEY applicable occupational health and safety regulations; accreditation, certification, or registration requirements; industry standards; and environmental, financial, and HEALTH CARE SERVICE delivery regulations? (7.0)
P.1 Organizational Descriptions: What are your key organizational characteristics? b . Organizational Relationships (1) Organizational Structure What are your organizational structure and GOVERNANCE system? What are the reporting relationships among your GOVERNANCE board, SENIOR LEADERS, and parent organization, as appropriate? (1.0, 7.4) (2) PATIENTS, OTHER CUSTOMERS, and STAKEHOLDERS What are your KEY market SEGMENTS, PATIENT and other CUSTOMER groups, and STAKEHOLDER groups, as appropriate? What are their KEY requirements and expectations for your HEALTH CARE SERVICES, other CUSTOMER support services, and operations? What are the differences in these requirements and expectations among market SEGMENTS, PATIENT and other CUSTOMER groups, and STAKEHOLDER groups? (3.0, 7.2)
P.1 Organizational Descriptions: What are your key organizational characteristics? (3) Suppliers and PARTNERS What are your KEY types of suppliers, PARTNERS, and COLLABORATORS? What role do they play • In your WORK SYSTEMS, especially in producing and delivering your KEY HEALTH CARE SERVICES and PATIENT and other CUSTOMER support services; and in enhancing your competitiveness? (2.2, 3.0, 4.0, 6.0, 7.2) • What are your KEY mechanisms for two-way communication with suppliers, PARTNERS, and COLLABORATORS? What role, if any, do these organizations play in contributing and implementing INNOVATIONS in your organization? What are your KEY supply-chain requirements? (3.0)
P.2 Organizational Situation: What is your organization’s strategic situation? P.2 Organizational Situation: What is your organization’s strategic situation? In your response, include answers to the following questions: a. Competitive Environment (1) Competitive Position What is your competitive position? What are your relative size and growth in the health care industry or the markets you serve? How many and what types of competitors do you have? (1.0, 2.2, 7.0)
P.2 Organizational Situation : What is your organization’s strategic situation? (2) Competitiveness Changes What KEY changes, if any, are affecting your competitive situation, including changes that create opportunities for INNOVATION and collaboration, as appropriate? (1.0, 2.2, 7.0) (3) Comparative Data What KEY sources of comparative and competitive data are available from within your health care industry? What KEY sources of comparative data are available from outside your industry? What limitations, if any, affect your ability to obtain or use these data? (1.0, 2.2, 7.0)
P.2 Organizational Situation: What is your organization’s strategic situation? b. Strategic Context • What are your KEY STRATEGIC CHALLENGES and ADVANTAGES in the areas of HEALTH CARE SERVICES, operations, societal responsibilities, and WORKFORCE? (1.0, 2.0, 7.0) c. PERFORMANCE Improvement System • What are the KEY elements of your PERFORMANCE improvement system, including your PROCESSES for evaluation and improvement of KEY organizational projects and PROCESSES? (1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0)
Exercise: Determine Key Factors Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Discuss at table as a group your thoughts on the key 20 factors for this applicant from Organizational Profile Tables 1/9: P1a(1)(2); Tables 2/10: P1a(3)(4)(5); Tables 3/11: P1b(1)(2); Tables 4/12: P1b(2)(3); Tables 5/13: P2a(1)(2); Tables 6/14: P2a(3); Table 7: P2b; Table 8: P2c 3. Read and Highlight Key Factors from the OP and select 20 a few to write down; Table Anchors guide this process 10 4. Record the requirements on chart paper 5. Report Out (1 minute per table) 5 55 Minutes Total
IR Key Factors (each Examiner) • Post your own individual KF on Scorebook Navigator • Team Leader will post the final consolidated KF for the team to use on Scorebook Navigator before moving forward with IR
Step 1: Read the Criteria
Exercise: Step 1 Read the Criteria Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Discuss at your table, (group) your thoughts on the key 20 requirements for your assigned Item from Baldrige Framework Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 3. Review individually your assigned Criteria Item 10 10 4. Record the requirements on chart paper 5. Report Out (1 minute per table) 10 50 Minutes Total
Step 2: Determine Most Relevant Key Factors for Criteria
Exercise: Step 2 Determine the Most Relevant Key Factors Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Review individually your assigned Criteria Item 15 3. Discuss at your table as a group and come to 20 agreement on the relevant 4 to 6 key factors (from Team Leaders list) for your assigned item. Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 4. Write your selected key factors on a flip chart for use in 10 subsequent exercises (capture key words and phrases) 5. Report Out (1 minute per table) 5 50 Minutes Total
Step 3: Read and Analyze the Application
Exercise: Step 3 Read and Analyze the Application Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Review individually your assigned Criteria Item 15 3. Discuss at your table as a group your observations on 20 the applicant ’ s response against the Criteria questions. Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 4. Report Out (1 minute per table?) 5 40 Minutes Total
Step 4: Identify Strengths/OFIs
Exercise: Step 4 Identify S’s/OFI’s Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Select potential S/OFIs individually. Then, use a round 20 robin approach to discuss individual S/OFIs identified 3. From this list, discuss at your table and select as a 25 group, a total of around 4/6 potential S/OFIs. Then select 2 S/OFIs that you feel are most important. Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 60 Minutes Total 5 4. Record these on chart paper 5. Report Out (1 minute per table) 10
Our Key Product
What Feedback Comments Should Do: • W hat the applicant does well (strengths) and what it needs to improve upon to take it to the next level OFIs (opportunities for improvement) • Use Criteria language • ADLI and LeTCI • Use OP narrative and KF • Use VMVG • Use Scoring language
Well- Written Comments: “NERD” • N – Nugget – Up front comment based on the Item Criteria or a Key Factor (i.e., briefly connects to what the applicant says is important) • E – Evidence or Example - State what the applicant has (Strength) or is missing or not addressed (OFI) • R – Relevance – Why is this comment important? Include a ctionable “so what” (not prescriptive) information; help the applicant to improve. • D – Done – Read to see if comment makes sense and helps the organization understand its current status and improvement possibility
Comment Writing Elements of a well written comment
How to Give Useful Feedback Do: Don’t: • Base your comments on the Criteria • Be judgmental or prescriptive • Reference the evaluation factors • Forget linkages • Include an opening “ nugget ” to • Forget the Key Factors give the comment significance • Stray from the Criteria • Keep each comment to single issue • Use embedded S/OFI comments • Make Key Factor references • Make conflicting strength and • Be accurate; check your statements OFI statements • Recognize the page limitations • Forget to: • Give benefit of the doubt – Check your score against the balance and content of comments • Be polite in tone – Check your facts • Include a few examples – Check grammar and spelling • Check spelling and grammar • Forget you’re providing a service
Step 5: Write Feedback Ready Comments
Sample Process Strength Nugget • 1.1b By using and improving a variety of mechanisms (Figure 1.1-2) to communicate with the workforce and community, senior Relevance leaders deliver on a key driver of workforce engagement. The mechanisms share and reinforce the organization’s vision, mission, Examples and core values. Several cycles of learning have resulted in the expanded use of social media, the addition of a county director to support communication flow from senior leaders and throughout the county, and revisions to the website to enhance transparency.
Sample Process OFI • 2.1b(2) The applicant’s strategic objectives Nugget and action plans (Figure 2.1-2) do not appear to address all strategic challenges. For example, financial objectives (e.g., to decrease administrative/indirect patient costs) do not Examples align with action plans to improve collection rates and relative value units, and no action plans align with strategic challenges related to recruiting paid staff members. The lack of Relevance alignment between strategic objectives, action plans, and strategic objectives may limit the applicant’s ability to maintain its competitive position and fulfill its mission.
Sample Results Strength • 7.2a(1) Patient and family satisfaction Nugget results — such as aggregate patient satisfaction, satisfaction with medical services, and satisfaction with dental services Examples (Figures 7.2-1 through 7.2-3) — have equaled or exceeded the top-decile level since 2013. These results reflect the applicant’s positive competitive position and support its mission Relevance to provide easy, timely access to high-quality, safe health care services responsive to diverse cultural and socioeconomic needs, regardless of ability to pay.
Sample Results OFI • 7.3 Results are missing for areas related to the Nugget applicant’s strategic challenge of staff recruitment and retention. For example, results Examples are not provided for recruitment of health care professions and physicians; for some drivers of workforce engagement, including comfort with reporting errors or unsafe acts, protection from health and safety hazards, and a flexible work Relevance schedule; and for measures of workforce safety. Without a focus on the strategic challenge of staff recruitment and retention, patients may not be able to “become the healthiest in the state.”
Exercise: Step 5 Write Feedback Ready Comments Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Draft feedback ready S/OFI comments (split table into 25 2 groups; 1 feedback ready comment - S and 1 OFI) Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 3. Give feedback to the other group and then redraft the 10 comment, using their input 10 4. Record the final feedback ready comment on chart paper 5. Report Out (1 minute per table) 10 55 Minutes Total
Step 6: Determine the Scoring Range and Score
Look at Scoring Guidelines in Baldrige Framework Pages 34 Process and 35 Results
Introduction to Scoring • Process Scoring Guidelines and Results Scoring Guidelines • Use Glossary for Definitions always • Process Scores Address: Approach, Deployment, Learning, Integration (ADLI) • Results Scores Address: Levels, Trends, Comparisons, Integration (LeTCI) 8.0 8.5 6.5
Exercise: Step 6 Scoring Activity Time (min) 1. Select a scribe, timekeeper and reporter (1) 2. Using your S/OFI comments from Step 4 and the 15 Scoring Guidelines, discuss as a group the most appropriate scoring range for your assigned Item. Table 1: 1.1; Table 2: 1.2; Table 3: 2.1; Table 4: 2.2; Table 5: 2.2; Table 6: 3.1; Table 7: 3.2; Table 8: 4.1; Table 9: 4.2; Table 10: 5.1; Table 11: 5.2; Table 12: 6.1; Table 13: 6.2; Table 14: 5.2 3. Next, discuss the appropriate score within that range 10 4. Record the scoring range and score on chart paper 10 and be prepared to discuss why you chose that score 40 Minutes Total 5. Report Out (1 minute per table) 5
Learnings?
Follow the Six Step Process Complete for Category/Items as assigned to your table
Recommend
More recommend
