Chapter 12. Evaluation Research Chapter 12. Evaluation Research - PDF document

What are the topics appropriate for What are the topics appropriate for Chapter 12. Evaluation Research Chapter 12. Evaluation Research evaluation research? evaluation research? � Evaluation research (program evaluation )’s Evaluation research (program evaluation )’s � � What is a social intervention? What is a social intervention? � uniqueness lies in its purpose: to evaluate the impact uniqueness lies in its purpose: to evaluate the impact � An action taken within a social context for the An action taken within a social context for the � of social intervention. of social intervention. purpose of producing some intended result purpose of producing some intended result � Topics covered in this chapter Topics covered in this chapter � � What topics are appropriate for evaluation What topics are appropriate for evaluation � � Topics appropriate for evaluation research Topics appropriate for evaluation research � research? � Issues related to conducting evaluation research Issues related to conducting evaluation research research? � � Formulating the Problem: Issues of Measurement Formulating the Problem: Issues of Measurement � � Occurring social intervention Occurring social intervention � � Types of Evaluation Research Designs Types of Evaluation Research Designs � � The Social Context The Social Context � Planned social intervention Planned social intervention � � � Common Problems associated with evaluation research Common Problems associated with evaluation research � � Social Indicators Research Social Indicators Research � What are the procedure for What are the procedure for How to formulate the problem? How to formulate the problem? evaluation research? evaluation research? � Formulating the problem Formulating the problem � � Specify the intervention Specify the intervention � � Specify the intervention Specify the intervention � � Ex. DARE ( Drug Abuse Resistance Education) program: A drug Ex. DARE ( Drug Abuse Resistance Education) program: A drug � � Specify the intended result of such intervention Specify the intended result of such intervention � prevention program that uses uniformed police officers in the prevention program that uses uniformed police officers in the � Specify the population Specify the population � schools to warn students away from drugs. schools to warn students away from drugs. � Measurement issues Measurement issues � � Specify the intended results of the intervention Specify the intended results of the intervention � Measure the outcome (response variable) Measure the outcome (response variable) � � � Measure the context and other related variables Measure the context and other related variables - - aspects of the context aspects of the context � � More than one intended result possible for a specific More than one intended result possible for a specific � of an experiment that might affect the experiment. of an experiment that might affect the experiment. project project � Mode of observation Mode of observation � � Ex. Reduce drug use among school Ex. Reduce drug use among school- -aged children aged children � Classical experimental design Classical experimental design � � � Quasi Quasi- -experimental design experimental design � � Specifying the population Specifying the population � � Evaluating success/failure Evaluating success/failure � � Population affected by the social intervention Population affected by the social intervention � Cost Cost- -benefit analysis benefit analysis � � � Agreement Agreement � Ex. Middle school /High school children Ex. Middle school /High school children � � What are the issues related to What are the issues related to What are the common modes of What are the common modes of measurements? measurements? observation? observation? � Specify outcomes to be measured Specify outcomes to be measured � Classical experiment design Classical experiment design � � � Can employ more than one measurement Can employ more than one measurement � The ultimate method to determine causal relationships The ultimate method to determine causal relationships � � � Ex. rate of tobacco use, rate of marijuana use, rate of cocaine Ex. rate of tobacco use, rate of marijuana use, rate of cocaine use, use, � Quasi Quasi- -experimental designs experimental designs � � rate of ecstasy use. rate of ecstasy use. � Time Time- -series design series design - - studying the processes occurring over time. studying the processes occurring over time. � � Measure study contexts and other variables Measure study contexts and other variables � Such analysis is useful when one can only have an experimental Such analysis is useful when one can only have an experimental group without a control group group without a control group � The context within which the study occurs is critical. The context within which the study occurs is critical. � � Nonequivalent control groups: Nonequivalent control group design Nonequivalent control groups: Nonequivalent control group designs s Hence variables related to the context must be identified Hence variables related to the context must be identified � can be employed when it is possible to find an alternative control can be employed when it is possible to find an alternative contr ol and measured as well. and measured as well. group similar to the experimental group, though this control gro group similar to the experimental group, though this control group is up is � Ex. If in the middle of your evaluation research there is a nati Ex. If in the middle of your evaluation research there is a national onal � not randomly selected. not randomly selected. TV campaign against drug use, then this campaign should be taken TV campaign against drug use, then this campaign should be taken into consideration, especially if the mode of observation is tim into consideration, especially if the mode of observation is time e- - � Multiple Time Multiple Time- -Series designs Series designs � series (see later slides) series (see later slides) � Qualitative evaluations Qualitative evaluations � � Other variables that may affect the outcome Other variables that may affect the outcome � � Field interviews, unstructured questions Field interviews, unstructured questions � � Ex. Family background of the school Ex. Family background of the school- -age children; Local culture. age children; Local culture. � 1