A Brief History Of Leadership In Assessing Undergraduates’ Learning Richard J. Shavelson & Leta Huang Stanford Education Assessment Lab Website: http://www.stanford.edu/dept/SUSE/SEAL/ Invited Talk Improving Quality and Equity in Education: Inspiring a New Century of Excellence in Teaching and Assessment Carnegie Foundation for the Advancement of Teaching & ETS June 1, 2006
Overview • The Foundation’s vision and role for “objective testing” • Framework for comparing learning assessments and locating vision • Brief history of college learning assessment – Origins of objective testing: 1900-1933 – Assessment of learning in graduate education—The GRE: 1933-1947 – Rise of the test providers: 1948-1978 – Era of external accountability: 1979-present • Concluding comments
The Foundation’s Vision And Role For “Objective Testing” Of Learning
The Beginning Of Assessing Student Learning In Colleges • Today’s call for campuses to account for their students’ learning appears almost as if there were no past…of course there was… • The Foundation was arguably the ringleader under the leadership of its first president, Henry Pritchett • Pritchett: – Was concerned with the quality of higher education – Recognized the potential impact the emergence of “objective testing” might have on monitoring quality, especially with the publication of E.L. Thorndike’s Mental and Social Measurements , and Henry Pritchett – Had a Foundation staff member who The Foundation’s agreed and avidly pursued the cause 1 st President
William S. Learned: Instruments Of Testing Reform • William S. Learned was that staff member who shared Pritchett’s passion for education quality and objective testing, became the instrument through which The Foundation transformed higher education learning assessment • Learned along with Columbia College’s Ben D. Wood: – Led a large-scale assessment of college learning in the state of Pennsylvania, – Parlayed this experience into the development of the Graduate Record Examination, and William S. Learned – Germinated the idea of a “National Foundation Staff Examination Board”—a national testing agency—which 20 years later became the Educational Testing Service.
The Foundation’s Vision In Assessing Student Learning Walter Jessup, The Foundation’s 3 rd President, put their vision for learning assessment and higher-education improvement in words that sound remarkably familiar today: “The central problems [in improving higher education] are three in number: first, the setting up of generally accepted standards of achievement; secondly, the devising of methods of measuring this achievement and holding pupils [now organization’s] to Walter Jessup performance; and thirdly, the introduction of The Foundation’s such flexibility in educational offerings that each 3 rd President individual [organization] may receive [deliver] the education from which he is able to derive the greatest benefit”
Framework for Comparing Learning Assessment The Foundation’s Offspring at ETS Current Claimed Approach—At Least Some Do The Foundation’s Initial Approach Source: Shavelson & Huang (2002)
Origins Of Objective Testing Of Undergraduates’ Learning: 1900-1933
Initial Objective Testing Of Undergraduates’ Learning: 1916- • Objective tests were seen as a scientific improvement over previous oral and written examinations characteristic of learning assessment in earlier years • Learned uses objective tests at the University Missouri’s Experimental School focusing on content knowledge in arithmetic, spelling, penmanship, reading, and English composition • Thorndike’s studies MIT engineering students using objective tests in E.L. Thorndike mathematics, English and physics Educational Psychologist • Learned and Ben D. Wood conduct the Pennsylvania Study—the first large-scale objective testing of learning in higher education
Views Of College Learning: Prevalent Today As Well • The Foundation’s Vision of Learning (Learned & Wood): – Accumulation of breadth and depth of content knowledge – “Must be a relatively permanent and available equipment of the student” – “Must be so familiar and so sharply defined that it comes freely to mind when needed and can be depended upon as an effective cross- fertilizing element for producing fresh ideas” • Competing Vision—Progressive Ben D. Wood Education Association (e.g. 8 Year Study) Learned’s Collaborator – Reasoning and problem solving – Life tasks – Moral, self and social development
The Pennsylvania Study: Large- Scale Statewide Learning Assessment • Objective testing of declarative and procedural knowledge ( achievement) of State’s college seniors (70% or 4,580) and high-school seniors (75% or 26,500) in 1928 • College learning of high-school seniors tested longitudinally in 1930 and again in 1932 • Testing Time: 12 hours in 1928; roughly half that later • Number of Test Items: 3,200 in 1928; roughly half that later
Noteworthy Features Of The Pennsylvania Study • Set clear conception of what was meant by learning in undergraduate education • Distinguished learning and achievement • Comprehensively covered content domain of general undergraduate education • Developed testing technology linked to Thomas J. Watson conception of learning IBM’s 1st President including IBM machine scoring
Assessment Of Learning In Graduate Education: 1933-1947
Motivation For Assessing Learning In Graduate Education • As Pennsylvania study wound down in late 1930s, Learned and Wood parlayed this experience, and their staff, into doing the same for graduate schools • Learned argued that the demand for graduate education had increased following the depression, that: – The AB degree had “ceased to draw the line between the fit and the unfit” and – Something more than number of college (“Carnegie”) credits was needed on which to base admission and graduate-student quality decisions • Answer: The GRE – Initially tested students in graduate school – Rapidly moved to become a selection measure for graduate school
GRE’s Rapid Growth • 1936-1938 Versions: 8 Profile content knowledge tests—e.g., mathematics, physical sciences, social studies, literature and fine arts, foreign language [dropped in first revision], and a verbal Number of Institutions 200 factor 150 • 1939: Advanced tests in 16 academic 100 major fields added 50 • 1946: General Education section 0 (effectiveness of expression and general 1937 1940 1945 1947 Ten Years of Growth education index) added to Profile tests (1937: Harvard, Yale, Columbia & Princeton) • 1949: GRE Aptitude Tests—Q and V – Grew out of the Profile’s mathematics and verbal factor tests – Offered as an alternative to the plethora of tests available from the GRE • Beginning of shift away from measuring content knowledge to measuring verbal and quantitative reasoning aptitudes
Rise Of The Test Providers: 1948-1978
Motivation For Rise Of Test Providers • End of WWII and GI Bill rapidly expands demand for higher education • In response supply increases: – Number of colleges and universities mushroom – Number of test providers • ETS established in 1948 • ACT established in 1959
Motivation For Creating ETS • A response to The Foundation’s – Testing burden with the GRE program – Financial burdens created by its innovative faculty retirement program (for which The Foundation was created) • A response to demand after WWII • A means to preserve “meritocracy”
Origins Of ETS: Learned & Wood … Again! • These events simply rekindled the notion of a “national testing service” that can be traced back to Learned and Wood who, as early as 1931 had envisioned a national college testing program, but the idea had “… languished, partly for lack of funds…” • Their vision was a national testing service overseeing: – A neatly organized education pyramid from pre-college at the base through college to graduate education at the apex – From base to apex, the pyramid increasingly relied on “ merit ” as measured by scientifically sound “objective” tests , and • According to Historian Ellen Lagemann, to blunt a challenge to objective testing launched by the Progressive Education Association (PEA) and the Eight Year Study beginning in 1933
Creation Of ETS • Notion of national testing service languished between 1938-1945 • Learned reopened the matter not only with respect to the GRE but also “to the work of ‘other enterprises of a related nature’: the College Entrance Examination Board, the Co-operative Tests Service of the American Council on Education, and the Education Henry Chauncey Records Bureau” ETS’ First President • CEEB and Cooperative Tests Service joined the Foundation in creating ETS • Carnegie Corporation provides $750,000 in start up for ETS • The Foundation leaves the testing business…
Recommend
More recommend
