Florida Association of Professors of Educational Leadership (FAPEL) - PowerPoint PPT Presentation

Florida Association of Professors of Educational Leadership (FAPEL) Florida Educational Leadership Examination (FELE) Spring Meeting January 19, 2016 Tallahassee, Florida 1 www.FLDOE.org

Postsecondary Assessment Team for Today Phil Canto, Bureau Chief Dr. Suleyman Olgar, Scoring and Reporting Supervisor Lauren White, Scoring and Reporting Team Dan Moore, Test Development Team 2 www.FLDOE.org

Today’s Postsecondary Assessment Presentation  Present status of current FELE program  Discuss status of research for “Next Gen” FELE  Review 2015 data (critical implementation year)  Review closely the FELE holistic scoring process  Provide sample materials and inform  Q & A 3 www.FLDOE.org

Educator Quality ‐ BHAG “Every Florida teacher and leader is prepared, developed, supported, and supervised by educators to make teaching better.” 4 www.FLDOE.org

FELE Program 411 • No planned substantive changes to current FELE content, structure, passing scores, or examination fees in 2015 or 2016; • Ongoing enhancement and expansion of M/C item pools for all three subtests ‐ We constantly monitor to ensure quality control; • With recent SBE changes to school grades, there are no impacts to FELE items and prompts. 5 www.FLDOE.org

FLDOE Research for Next Generation FELE • FLDOE is evaluating several options for performance based or enhanced assessment(s) • May impact FELE and/or other exams in the future (current or new) • End of 2016 to 2017 before pilot project could begin with statewide inputs • Normal test development process is at least 2 years • Biggest question is, “What is the current FELE not giving us that a performance assessment might?” 6 www.FLDOE.org

FELE 3.0 Program Changes • January 1, 2014 (New Test) • FELE 3.0 Examination • Subtest 1: Leadership for Student Learning • Subtest 2: Organizational Development • Subtest 3: Systems Leadership • Effective January 1, 2015 (New Format & Passing Scores) • Subtest 3: Systems Leadership • Multiple ‐ Choice Section • Written Performance Section (Examinees must pass both sections independently to pass FELE Subtest 3) 7 www.FLDOE.org

FELE Recommended Passing Scores, Projected Pass Rates vs. 2015 Operational Pass Rates Operational Pass Rates* Recommended Recommended Projected FELE Subtest N % Number Correct Percentage Correct Passing Rates Subtest 1: Leadership for 48/70 69% 58% 1,617 63% Student Learning Subtest 2: Organizational 48/69 70% 55% 1,568 58% Development Subtest 3: System Leadership: Multiple- 36/55 66% 1,620 50% Choice Section 34% Subtest 3: System Leadership: Written- 7/12 59% 1,618 46% Performance Section * Operational pass rates are calculated for the first-time examinees taking FELE subtests from January 2, 2015 to December 31, 2015 with the approved passing scores at the November 2014 SBE meeting which became effective by January 1, 2015. Note: FELE Subtest 3 written performance assessment is holistically scored by two human raters using a scoring rubric of 1–6 range. FELE Subtest 3 written performance assessment score is the sum of the two ratings. 8 www.FLDOE.org

FELE 3.0 First-Time Examinee Pass Rates (2014 vs. 2015) 2014 2015 FELE 3.0 N Pass Rate Mean Score N Pass Rate Mean Score FELE Subtest 1: 1,495 84% 219 1,617 63% 202 Leadership for Student Learning FELE Subtest 2: 1,485 90% 225 1,568 58% 200 Organizational Development FELE Subtest 3: System Leadership 1,485 82% 221 1,620 50% 197 (Multiple-Choice Section) FELE Subtest 3: System Leadership N/A N/A N/A 1,618 46% 7 (Written Performance Section) FELE Assessment Level 1,475 73% N/A 1,523 22% N/A 9 www.FLDOE.org

FELE 3.0 Best Attempt Examinee Pass Rates (2014 vs. 2015) 2014 2015 FELE 3.0 N Pass Rate Mean Score N Pass Rate Mean Score FELE Subtest 1: 1,495 87% 219 1,617 70% 202 Leadership for Student Learning FELE Subtest 2: 1,485 93% 226 1,568 69% 200 Organizational Development FELE Subtest 3: System Leadership 1,485 86% 222 1,620 63% 197 (Multiple-Choice Section) FELE Subtest 3: System Leadership N/A N/A N/A 1,618 59% 7 (Written Performance Section) FELE Assessment Level 1,475 81% N/A 1,523 42% N/A 10 www.FLDOE.org

FELE Performance by Competency (2014 ‐ 2015) Test Name Competency Name 2014 2015 Florida Educational Leadership Examination (FELE) N % Correct N % Correct 69.7% 69.8% 01 (17-18) Facilitate student learning goals 02 (18) Prioritize student learning via leadership 70.6% 74.6% FELE Subtest 1: 1,495 1,572 03 (15-17) Develop/implement instructional Leadership for Student Learning 71.5% 71.6% framework 68.4% 70.5% 04 (17) School environment for student learning 01 (11-12) Recruitment and induction practices 72.2% 74.1% 02 (22-23) Faculty/staff development and retention 69.5% 71.6% FELE Subtest 2: 1,485 1,522 Organizational Development 03 (16-17) Develop leadership w/in the organization 70.9% 71.9% 70.3% 71.0% 04 (17-18) Personal/professional behavior practices 62.7% 67.2% 01 (13-14) Decision-making processes 02 (14) School operation management practices 69.8% 68.5% FELE Subtest 3: System Leadership 1,485 1,574 (Multiple-Choice Section) 03 (14) Financial and resource management 64.4% 66.9% 64.8% 66.2% 04 (13) School legal practices and applications FELE Subtest 3: System Leadership 05 (1) Effective communication practices N/A N/A 1,471 62.6% (Written Performance Section) 11 www.FLDOE.org

Score Verification • Score verification eligibility: • A non ‐ passing score of 190 or higher • Written Performance component Number of Items Number of Status Number of Sessions Reviewed Changes FELE 2014 2015 2014 2015 2014 2015 Subtest 1: Leadership for Student Learning 11 25 119 197 0 0 Subtest 2: Organizational Development 3 22 59 163 1 0 Subtest 3: Systems Leadership 23 23 254 (14) 212 1 1 Multiple-Choice Section Subtest 3: Systems Leadership 0 42 0 45 0 0 Written Performance Section 12 www.FLDOE.org

FELE Written Performance Assessment (WPA) & Holistic Scoring • Interpretation of data and writing ability as it relates to the data analysis and target audience; • Organization, sentence structure, and mechanics, as well as the accuracy of data interpretation • An examinee response is judged for its total effect; • Rater makes an overall evaluation taking multiple performance factors into consideration • Responses are rated independently by two raters and sent through a discrepancy resolution process if rater scores vary by more than 1 point. 13 www.FLDOE.org

FELE WPA Responses & Holistic Scoring Process • Scoring Rubric identifies performance characteristics to be judged, includes the criteria for evaluating those features, and describes how performance varies across the scoring scale • Anchor Set is used to calibrate raters to the rubric • Supplemental Rating Criteria lists the critical elements of data use by score points • Calibration/Practice Set includes pre ‐ scored responses and used for rater calibration prior to operational scoring 14 www.FLDOE.org