I nternational research assessm ent revisited - A com parison betw - PowerPoint PPT Presentation

I nternational research assessm ent revisited - A com parison betw een the research perform ance of KTH and selected UK universities • Folke Snickars, Faculty Board, KTH • Björn Marklund, Rector’s Office, KTH • Tom Letcher, Evidence Ltd, UK 1

KTH Research Performance Assessment Project • Evidence Ltd has experience from UK RAE • Interesting method to bench learn in research assessment • Access to data in UK Higher Education Research Yearbook • KTH has asked Evidence to create test KTH footprints • Comparison is with 10 selected UK universities as peers • Aim is to be proactive concerning research assessment • Bibliometrics foreseen as part of national funding system • Methods and indicators for an internal allocation system • Project may serve as model for School dialogues 2

Performance Indicators • Research degrees Licentiate • Staff –Professors • Research degrees PhD –Associate professors –Research associates • Research funding –Senior and junior lecturers –Government grants –Research staff –Research Council grants –Funded doctorate students –Government Agency grants –Foundation grants • Bibliometrics –Private sector grants –Publications –EU –Rebased impact 3

Subjects for Assessment • KTH • UK HEA • Mathematics • Mathematics • Chemistry • Chemistry • Information Technology • Computer Science • Engineering Physics • Physics • Electrical Engineering, • Electrical and Electronic Electronics and Photonics Engineering • Chemical Engineering • Chemical Engineering • Biotechnology • Biological Science • Engineering Mechanics • Mechanical Engineering • Material Science • Metallurgy and materials • Civil Engineering and • Civil Engineering Architecture • Built environment • Engineering and Business • Business and management management Studies • Environmental Engineering • Environmental Science • Urban Studies • Town and Country Planning • Engineering Science (other) • General Engineering 4

Background Data • KTH Annual Report • Data from Statistics Sweden (SCB) • ISI Thomson University Publication Database 1996-2005 • UK RAE 2001-2004 – Imperial College, Sheffield, Leeds, Southhampton, Birmingham, Manchester, Surrey, Edinburgh, Strathclyde, College London 5

6 Publishers Three Perspectives Universities Financiers

Physics vs Engineering Physics KTH Imperial Coll London Tot al 1 1 Per FTE 8 2 8 2 7 3 7 3 6 4 6 4 5 5 Univ Leeds 1 1 Research active staff 2 Research fellows 8 2 3 Research assistans 4 Research degrees (PhD) 7 3 5 Studentships 6 4 6 RGCI 7 Articles in ISI journals 5 8 Rebased Impact 7

Civil Engineering KTH Imperial Coll London Tot al 1 Per FTE 1 8 2 8 2 7 3 7 3 6 4 6 4 5 5 Univ Leeds 1 Research active staff 1 2 Research fellows 8 2 3 Research assistans 4 Research degrees (PhD) 7 3 5 Studentships 6 RGCI 6 4 7 Articles in ISI journals 5 8 Rebased Impact 8

KTH Mechanical, Aeronautical and Manufacturing Imperial Coll London Tot al Engineering 1 Per FTE 1 8 2 8 2 7 3 7 3 6 4 6 4 5 5 Univ Leeds 1 1 Research active staff 8 2 2 Research fellows 3 Research assistans 7 3 4 Research degrees (PhD) 5 Studentships 6 4 6 RGCI 5 7 Articles in ISI journals 8 Rebased Impact 9

KTH Imperial Coll London Metallurgy and Materials Tot al 1 Per FTE 1 8 2 8 2 7 3 7 3 6 4 6 4 5 5 1 Research active staff Univ Leeds 2 Research fellows 3 Research assistans 1 4 Research degrees (PhD) 8 2 5 Studentships 6 RGCI 7 3 7 Articles in ISI journals 6 4 8 Rebased Impact 5 10

Physics London Surrey Strathclyde Southampton Sheffield Manchester Leeds Imperial Edinburgh Birmingham KTH 0 50 100 150 200 250 300 AcademicStaff ResearchActiveStaff 11

Physics London Surrey Strathclyde Southampton Sheffield Manchester Leeds Imperial Edinburgh Birmingham KTH 0 0,5 1 1,5 2 2,5 3 3,5 4 Yearly publications 2001-2005/R&D active staff 2004 12

Physics London Surrey Strathclyde Southampton Sheffield Manchester Leeds Imperial Edinburgh Birmingham KTH 0,00 0,50 1,00 1,50 2,00 2,50 3,00 AverageRebasedImpact2001_2005 AverageRebasedImpact1996_2001 13

Physics London Surrey Strathclyde Southampton Sheffield Manchester Leeds Imperial Edinburgh Birmingham KTH 0 1000 2000 3000 4000 5000 6000 7000 8000 Articles2001_2005 Articles1996_2001 14

KTH School Indicators • Number of applicants for academic positions • Number of teachers with doctoral degree • Number of female teachers • Number of applicants to undergraduate programs • Student’s average grading of courses • Throughput of undergraduate students after three semesters • Inbound and outbound undergraduate exchange students • Number of graduates holding a position after six months • Study time for completing PhD degree • Average impact value of publications • Number of citations • Research income from competitive funding agencies • Income from commissioned research and education 15

Some Lessons Learned • Possible to make comparison of publication work • Comaprable data on staff and funding with difficulty • Shows the need to be careful with information systems • Creates a background for internal work on indicators • Shows the need to be knowledgeable in the subject areas • KTH present in broad range of subjects • KTH shows particular strength in core engineering fields • KTH publication performance varies over time • KTH exhibits same level of productivity as UK peers • Productivity varies substantially across universities in field 16