19th International Conference on Computational Statistics Paris - France, August 22-27 Contextual factors of the external effectiveness of the university effectiveness of the university education: a multilevel approach Matilde Bini European University European University of of Rome Rome Leonardo Grilli, Carla Rampichini Department of Statistics “G. Department of Statistics “G. Parenti Parenti”, University of Florence ”, University of Florence
Aim of the What is the effectiveness of university education? research Dataset The effectiveness of university education is a measure of the y Method of ability of the university system to achieve educational aims analysis Main results • Occupational status after degree (1= Yes, 0= No) • Duration of unemployment (time to first job) Duration of unemployment (time to first job) Conclusions • Wage or job satisfaction Aim of this research Check the possibility of measuring the external effectiveness of universities (or course programs of all universities) In order to make fair comparisons among different universities (or course programs) we take account of both • the characteristics of the individuals • and the economic and social context factors of the Italian regions To this aim we apply Multilevel (mixed, random effects) models
Data set Aim of the research From last survey on job opportunities of the Italian graduates in Dataset 2004 conducted by the Italian National Institute of Statistics in 2004, conducted by the Italian National Institute of Statistics in Method of analysis 2007 (Istat, 2008) Main results Peculiarity : two different groups of individuals with respect to the y g p p Conclusions reform about teaching organization DM n.509/99 (decree law) � 26570 graduates from degree programs before � 20730 graduates from degree programs after After the decree law DM n.509/99 the new teaching system should have affected the ‘performance’ of universities in terms of capacity to prepare young people to needs of job market and also changed the propensity of people to search for a job after g p p y p p j the degree This study is accomplished using graduates from degree This study is accomplished using graduates from degree programs after this decree law
Variables considered in the analysis V i bl Aim of the research Response variable Occupational status at the interview (about three years after Dataset degree) (1=Y g ) ( es, 0=No) , ) Method of analysis External contextual covariates measured at Covariates measured at individual level regional level (Istat, 2008) • age Main results a) macroeconomic measures • gender g • Conclusions Gross Domestic Product per inhabitant • marital status ( gdp ) • address (residence) during the studies • productivity of labour ( prod_lab ) • kind of the degree b) job market measures • final mark • (youth) unemployment rate ( unempl ) • occupational condition during the studies • quota of irregular labour ( q_lab ) • other studies or training jobs after the c) measure of production structures degree • number of firms per inhabitant • course program changes p g g ( n_firms ) f • military service • average number of employees per • social background firm ( empl_firm ) • kind of the high school attended d) measure of innovation and technology • quota of innovative firms f f ( innov_firms ) Remark: the only available information e) measures of the degree of culture pertaining characteristics of course programs • quota of family expenses for cultural or universities i e internal contextual or universities, i.e. internal contextual entertainments ( q_family ) i ( f il ) variables, are the cluster means of individual f) measure of quality of life level covariates • the poverty rate ( q_life )
� To estimate the probability to get a job Aim of the research Dataset the sample size was reduced to 13424 units by eliminating all graduates who at the date of the interview: li i i ll d h h d f h i i Method of analysis � did have the same job before their degree Main results � were unemployed but at the same time were not Conclusions interested in searching for a job Multilevel approach M ltil l h � To evaluate how much the contextual covariates affect the probability to get a job b bili j b Level 2 � course programs by universities (546 clusters) p g y ( ) Level 1 � graduates (13424 graduates)
T Two level Logit model l l L it d l Aim of the research ∑ ∑ ∑ ∑ Dataset π π = = β β + + β β + + δ δ + + log ( ) log ( ) it it x x z z u u Method of 0 ij h hij l lj j analysis h l ∑ ∑ ∑ ∑ Main results β β + + β β + + δ δ + + exp( exp( ) ) x x z z u u 0 h h hij h l l l lj j Conclusions = = π = + h l ( 1| , ) E y x u ∑ ∑ β + β + δ + ij ij j ij 1 exp( ) x z u 0 h hij l lj j h h l l l- th macro-economic probability to covariate for the j- th covariate for the j- th get job t j b h- th covariate for cluster the i- th graduate iid σ σ of the j- th cluster ∼ 2 (0 (0, ) ) u u N N j u 2 nd level random effect Remark: all the 2 nd level factors beyond observed characteristics 2 d l R k ll th l f t b d b d h t i ti are included in u j
Model A without external contextual variables contextual variables M d l Model results 1 lt 1 Aim of the research Dataset Method of analysis Main results Conclusions Labels: • age of graduates ( age2 ) Baseline graduate (when the model is null) • graduation within institutional time ( graduate_in_time, 1 if yes) � less than 23 years old y • occupational condition during the studies occupational condition during the studies In the fit, we � not graduated within institutional time ( working_student , 1= if at work during the also added the studies) � never worked during the studies mean of the h - • educational level of father th covariate of � not graduated father ( graduate_father) the j th cluster the j -th cluster • cluster mean of graduate_father � not attended courses after degree * z (proportion) ( pctg_graduate_father) h j • attended courses after getting the degree( after_degree )
Model B with external contextual variables: unempl_2005 p _ Model results 2 M d l lt 2 Aim of the research Dataset Method of analysis Main results Conclusions As regards the macro-economic variables measured in the 2005 year at Regional level, only the unemployment rate ( unempl_05 ) is significant significant.
Aim of the Regional unemployment rate effect research Dataset the probability to be employed for the baseline the probability to be employed for the baseline Method of analysis graduate (86.84%) reduces of about 6% when the Main results unemployment rate increases from 6% (average unemployment rate increases from 6% (average Conclusions value) to 10% the unexplained cluster variability is partly explained by the regional unemployment rate ⎛ ⎞ σ − σ 2 2 ⎜ ⎜ m od m od ⎟ ⎟ = u B u A = − j j j j *100 *100 20.9% 209% I I ⎜ ⎟ Δ σ 2 ⎝ ⎠ m od u A j
Aim of the Probabilities to get job for a baseline graduate in research “bad” and “good” Universities: a classification Dataset π i = exp( η ij + u j )/[1+ exp( η ij + u j )] Method of analysis Main results • Very good univ. u j = + 2 σ u Conclusions • Good univ. u j = σ u • Medium univ. u j = 0 • Bad univ u = - σ Bad univ. u j = σ u • Very bad univ. u j = -2 σ u Baseline graduate B li d t + ∑ η = β δ � less than 23 years old z ij 0 l lj � not graduated within institutional time l � never worked during the studies g � not graduated father � not attended courses after degree
U Unemployment rate effect l t t ff t Aim of the research Dataset Method of analysis Main results Conclusions
U Unemployment rate effect: regional differences l t t ff t i l diff Aim of the research Probability plot for graduates in Economics Dataset Method of analysis Main results Conclusions
Concluding remarks Aim of the research � The use of contextual characteristics improves the Dataset ranking of universities (or course programs) f Method of analysis � Need to improve this measure including Main results � Internal contextual variables (characteristics of � I t l t t l i bl ( h t i ti f Conclusions institutions) � External contextual variables are observed only at � External contextual variables are observed only at regional level. Need to have information for “local” territories (counties) Work in progress p g � Same analysis using the graduates before the decree law n.509/99
THANK YOU FOR YOUR THANK YOU FOR YOUR ATTENTI ON ATTENTI ON ☺
Recommend
More recommend
