National Aeronautics and Space Administration NASA STEM Engagement Performance Assessment & Evaluation Briefing August 9, 2019
NASA STEM Engagement Performance Assessment & Evaluation Briefing Agenda NASA STEM Engagement Performance Assessment, • NASA STEM Engagement FY 2018 Evaluation, & Information Management (PAEIM) Team Performance Data Summary (Preliminary) • NASA STEM Engagement Performance & Rob LaSalvia, Chief, NASA Glenn Office of Education Evaluation Strategy Rick Gilmore, Evaluation Manager • Higher Education Challenges, Paragon TEC Competitions, and Internships Multiple Dr. Catherine Graves, Program Manager/Sr. Evaluation Specialist Case Study Summary of Findings Dr. Tara Strang, Evaluation Specialist Dr. Anne Kotynski Gooding, Evaluation Specialist • NASA Office of STEM Engagement Clarence Jones, Technology Coordinator/OEPM Data Manager (OSTEM) Programs Principal Investigator (PI) Effective Practices Panel Discussion Session 2
F OOD FOR T HOUGHT Ask Questions At: jsc.cnf.io Don’t forget to vote responses up! 3
NASA STEM Engagement FY 2018 Performance Data Summary (Preliminary) 4
FY 2018 NASA STEM Engagement Performance Assessment Internships and Research and Collaborators Fellowships Development 7,205 2,214 In Fiscal Year 2018, NASA provided 7,205 internships and fellowships to higher education students across all institutional categories and levels. These significant awards provided a total of over $14M in direct financial support to higher education students. NASA’s performance in providing 27% Underrepresented opportunities for learners to contribute to NASA’s aeronautics, space, and science NASA’s Office of STEM Engagement 73% missions and work is assessed across peer- Non-underrepresented collaborators are funded and unfunded reviewed publications and technical paper and located in all 50 states, DC, GU, PR, presentations directly resulting from and VI. Collaborators include: government research funded by NASA STEM Engagement agencies, industry, formal and informal 27.3% of higher education internships grants and awards to higher education education institutions including museums, institutions. and fellowships were awarded to racially 1,324 science centers, planetariums, and youth- or ethnically underrepresented student serving organizations, non-profit, and participants, compared to 24.5% for the other education organizations. national average of STEM degree Collaborators extend the reach of NASA enrollees. STEM engagement opportunities by Space Grant, MUREP, and EPSCoR grantee supporting the execution of an Additionally, 39.6% of the Agency’s higher and awardee institutions reporting 1,324 opportunity. In FY 2018 OSTEM education internships and fellowship peer-reviewed publications and technical collaborated with 2,214 institutions and positions were filled by women. papers and presentations in FY 2018. organizations. 4 5
NASA STEM Engagement Performance and Evaluation Strategy 6
Legislative Directives that Guide the Office of STEM Engagement’s Work Public Law 85-568 Public Law 114-329 (S. 3084) The National Aeronautics and Space Act of 1958. American Innovation and Competitiveness Act. Created NASA and directed that the agency should pursue several goals, including The act authorizes creation of an interagency advisory panel and working groups to the pursuit of educational activities. consider education for STEM fields and supports the coordination of citizen science and crowdsourcing by Federal agencies to accomplish their missions. Public Law 111-352 (H.R. 2142) Public Law 114-264 (S. 1550) Government Performance and Results Act – Modernization Act Program Management Improvement Accountability Act (PMIAA). (GPRAMA) of 2010. Requires quarterly performance assessments of Government programs for purposes Enhances accountability and best practices in project and program management of assessing agency performance and improvement. throughout the federal government. H.R. 4174 Foundations for Evidence-Based Policymaking Act of 2017. Reauthorization Act of 2015 (H.R. 1806) America COMPETES Act. Requires the measurement of the federal government’s effectiveness through the use of program evaluation, continuous improvement, policy-relevant research, and Funds investments in science and engineering research and STEM education from cost-benefit analyses by qualified researchers and institutions. kindergarten to the postdoctoral level. Established the National Science and Technology Council’s Committee on STEM Education (CoSTEM) to: (1) collaborate with the STEM Education Advisory Panel and other outside stakeholders to ensure H.R. 4887 the engagement of the STEM education community, and (2) review evaluation Grant Reporting Efficiency and Agreements Transparency (GREAT) Act. measures used for federal STEM education programs. Requires the creation of a comprehensive and standardized data structure, or “taxonomy”, covering all data elements reported by recipients of federal awards, including both grants and cooperative agreements. 7 7
Strategic Alignment of Studies (Illustrative) 8
Strategic Alignment of Studies (Illustrative) 9
Performance Assessment & Evaluation Overview 10 10
FY19 & FY20 Office of STEM Engagement Performance Assessment Model 11
NASA Higher Education Challenges, Competitions, and Internships Multiple Case Study Summary of Findings 12
Introduction QUESTIONS: PURPOSE: 1. In what ways are NASA STEM engagement activity higher The overarching purpose of this basic research study was to education participants contributing to NASA’s aeronautics, define the key constructs underpinning NASA’s goals for STEM space, and/or science missions? engagement in relation to higher education activities. NASA’s goals for STEM engagement put forth that STEM engagement 2. What evidence-based and practice-based effective strategies activities should: for STEM learning are used in the design and implementation of NASA STEM engagement higher education activities? A. Enable contributions to NASA’s work; 3. What evidence-based and practice-based effective strategies B. Build a diverse, skilled future STEM workforce; and for the recruitment and retention of participants from groups historically underrepresented or underserved in STEM fields C. Strengthen understanding of STEM through powerful are used in the design and implementation of NASA STEM connections to NASA. engagement higher education activities? UTILIZATION-FOCUSED APPROACH: Designed for the actual use and application of evaluation findings (Patton, 2015) 13 13
Methodology & Data Collection EVALUTION QUESTIONS DATA SOURCE ANALYSIS • Open-ended questionnaires Qualitative In what ways are NASA STEM engagement activity • Document (content) analysis of activity plans, performance and evaluation reports, Analysis higher education participants contributing to NASA’s presentations, and mission directorate strategic planning documents, goals, and priorities aeronautics, space, and/or science missions? • Literature review Qualitative • Open-ended questionnaires What evidence-based and practice-based effective Analysis • Document (content) analysis of activity plans, performance and evaluation reports, strategies for STEM learning are used in the design presentations, and mission directorate strategic planning documents, goals, and and implementation of NASA STEM engagement priorities • Literature review higher education activities? • Open-ended questionnaires Qualitative What evidence-based and practice-based effective • Document (content) analysis of activity plans, performance and evaluation reports, Analysis strategies for the recruitment and retention of presentations, and mission directorate strategic planning documents, goals, and participants from groups historically underrepresented or priorities • Literature review underserved in STEM fields are used in the design and implementation of NASA STEM engagement higher education activities? 14
Recommend
More recommend
