zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA "Reviewed - PowerPoint PPT Presentation

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA "Reviewed November 2014" 2D Barcode Pilot Lessons Learned & Early Findings Marshall Gaddis Deloitte Consulting zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA 44

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Outline Lessons Learned • Vaccination Data • User Experience Analysis Methodology Early Pilot Findings • Vaccination Data Overview • Pilot Supply Chain • Metrics • Next Steps

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Lessons Learned: Vaccination Data 1. Data cleansing • Intentionally added information C3699AA – HOSP • Reduction in data quality (P)C3699AA 1.Connected 2. Consistency between EMR and IIS data 2.Transfer EMR IIS • Reporting requirements 3.Double Entry • Non-identical datasets 4. IIS Only 3. Name standardization 1.PEDIARIX-VFC • As written at administration 2.DTAP/IPV/HEP B • Any field with free-form entry 3.VFC: DTAP-POLIO- PEDIARIX HEP B … 101.DTAP-HEP 8-IPV

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Lessons Learned: User Experience 1. Common Challenges • Within administration and inventory Only small percent 2D vaccines Administration Administration Difficulties scanning Inconsistent supply 2D vaccines Minor Different process Only small percent 2D vaccines Moderate Inconsistent supply 2D vaccines Major Inventory Inventory Having to open box to scan Different process other vaccines Difficulties scanning 0 50 100 150 Respondents (#) Preliminary results

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Lessons Learned: User Experience (cont.) zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA 2. Incorporating Scanners with EMRs • Configuration and capability • Inventory versus Administration Administration: 2D Scanning Frequency 50 (N=152) 3. Pilot Supply Chain Respondents (%) • Infrequent 2D product 40 30 • Decreases likelihood to scan 20 10 0 Less than 1 to 3 4 to 6 7 to 9 10 or more 1 Frequency of scanning per week Preliminary results

Analysis Methodology

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Early Pilot Findings: Vaccination Data Overview Data V olume zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA EMR IIS • Nearly one million records to Baseline Learning Baseline Learning analyzed 297,000 718,000 344,000 960,000 Records • Robust estimates 146 32,000 58 26,000 • More 2D records 2D Records per site in EMR 160 118 204 204 Providers Note: Figures rounded. After exclusions: missing vaccination date, invalid vaccine type (e.g., PPD test), withdrawn site EMR IIS Site distribution • Subset of sites provided all phases 84 7* 112* • Before-and-after comparison • Use for analyses sites sites sites *Note: Numbers above represent count of sites with both Baseline and Learning data for either EMR or IIS. IIS data came from different sets of 204 sites in Baseline vs. Learning. Preliminary results

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Early Pilot Findings: Pilot Supply Chain • Percent of each vaccine that has converted to 2D barcodes in EMR data zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Note: Data shown are from EMR vaccinations Preliminary results

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Early Pilot Findings: Metrics Data elements • Lot Number, Expiration Date, Product Code Metrics • Completeness and Accuracy Upward trend before looking at potential confounders • Isolate effect of technology from other causes • Impact of percentage increase on one million records IIS Lot Metrics EMR Lot Metrics (where lot number is scanned) (where lot number is scanned) 100% 100% 90% 90% 80% 80% 816,000 674,000 288,000 667,000 265,000 648,000 289,000 192,000 70% 70% 60% 60% 50% 50% Complete Accurate Complete Accurate Base Learn Base Learn Number of records Number of records written vertically written vertically Preliminary results

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA Early Pilot Findings: Next Steps Maturity Data Collection • All sites finish by end of April Analysis • Calculate effect of technology after removing potential confounders (e.g., Site variation, observation bias, seasonality) • Examine across demographic variables • Statistical tests and adjustments (e.g., Bonferroni) Baseline Data Learning Data Maturity Data Immunizer Immunizer Recruiting Installation Awardee WFA 1 WFA 2 Recruiting Pilot Installation Scheduling Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Sep 2011 May 2013

Questions?