Collecting Data Marc H. Mehlman marcmehlman@yahoo.com University of New Haven Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 1 / 28
Table of Contents Design of Experiment, DOE 1 Sampling Design 2 Inference 3 Ethics 4 Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 2 / 28
Observation vs. Experiment When our goal is to understand cause and effect, experiments are the only source of fully convincing data. The distinction between observational study and experiment is one of the most important in statistics. An observational study observes individuals and measures An observational study observes individuals and measures variables of interest but does not attempt to influence the variables of interest but does not attempt to influence the responses. The purpose is to describe some group or situation. responses. The purpose is to describe some group or situation. An experiment deliberately imposes some treatment on An experiment deliberately imposes some treatment on individuals to measure their responses. The purpose is to study individuals to measure their responses. The purpose is to study whether the treatment causes a change in the response. whether the treatment causes a change in the response. In contrast to observational studies, experiments don’t just observe individuals or ask them questions. They actively impose some treatment in order to measure the response. 5 Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 3 / 28
Confounding Observational studies of the effect of one variable on another often fail because of confounding between the explanatory variable and one or more lurking variables. A lurking variable is a variable that is not among the A lurking variable is a variable that is not among the explanatory or response variables in a study but that may explanatory or response variables in a study but that may influence the response variable. influence the response variable. Confounding occurs when two variables are associated in Confounding occurs when two variables are associated in such a way that their effects on a response variable cannot be such a way that their effects on a response variable cannot be distinguished from each other. distinguished from each other. Well-designed experiments take steps to avoid confounding. 6 Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 4 / 28
Design of Experiment, DOE Design of Experiment, DOE Design of Experiment, DOE Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 5 / 28
Design of Experiment, DOE Definition The individuals on which the experiment is done are the experimental units . When the units are human beings, they are called subjects . The explanatory variables of the experiment are called factors . Often factors are administrated at different levels , ie, strengths. A specific experimental condition (combination of levels from the different factors) applied to a subset of the units is called a treatment . If there exists a collection of experimental units that are not exposed to any of the factors, that group is called the control group . Example Scientists are interested in the effects of two factors in crop yield, namely irrigation and fertilizer. They consider four levels of irrigation; none, once a week, three times a week and daily. They consider three levels of fertilizer, none, moderate application and heavy application. By combining all levels, the scientists come up with twelve treatments. They then apply each treatment to nine of 108 plots of crop land. Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 6 / 28
Design of Experiment, DOE When doing medical experiments using humans, it was often noticed that subjects who knew they were in a control group and hence receiving no treatment did worse than those who were receiving treatment even when the treatment was known to be ineffective! This phenomenon is called the placebo effect , named after the method frequently used to deal with this effect. Subjects who do not take pills in a medical study know they are not getting treatment, so all subjects are given identically looking pills, only some subjects are given placebos , sugar bills with no medical effects. Subjects are not told which treatment they are getting. This does not remove the placebo effect from the experiment, but the placebo effect will now be the same for all treatments. The power of suggestion, ie the placebo effect, is quite strong in humans. A mother’s kiss on her child’s seemly critical wound can often render the scratch painless. Similarly placebos can have positive effects with adults suffering from more serious maladies. Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 7 / 28
Design of Experiment, DOE Randomized Comparative Experiments The remedy for confounding is to perform a comparative experiment in which some units receive one treatment and similar units receive another. Most well-designed experiments compare two or more treatments. Comparison alone isn’t enough. If the treatments are given to groups that differ greatly, bias will result. The solution to the problem of bias is random assignment. In an experiment, random assignment means that In an experiment, random assignment means that experimental units are assigned to treatments at random, that experimental units are assigned to treatments at random, that is, using some sort of chance process. is, using some sort of chance process. 11 Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 8 / 28
Design of Experiment, DOE Randomized Comparative Experiments In a completely randomized design, the treatments are In a completely randomized design, the treatments are assigned to all the experimental units completely by chance. assigned to all the experimental units completely by chance. Some experiments may include a control group that receives Some experiments may include a control group that receives an inactive treatment or an existing baseline treatment. an inactive treatment or an existing baseline treatment. Group Group 1 1 Random Random Experimental Experimental Assignme Assignme Units Units nt nt Group Group 2 2 12 Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 9 / 28
Design of Experiment, DOE Principles of Experimental Design Randomized comparative experiments are designed to give good evidence that differences in the treatments actually cause the differences we see in the response. Principles of Experimental Design Principles of Experimental Design 1. Control for lurking variables that might affect the response, most 1. Control for lurking variables that might affect the response, most simply by comparing two or more treatments. simply by comparing two or more treatments. 2. Randomize: Use chance to assign experimental units to 2. Randomize: Use chance to assign experimental units to treatments. treatments. 3. Replication: Use enough experimental units in each group to 3. Replication: Use enough experimental units in each group to reduce chance variation in the results. reduce chance variation in the results. An observed effect so large that it would rarely occur by chance is An observed effect so large that it would rarely occur by chance is called statistically significant. called statistically significant. A statistically significant association in data from a well-designed A statistically significant association in data from a well-designed experiment does imply causation. experiment does imply causation. 14 Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 10 / 28
Design of Experiment, DOE Cautions About Experimentation The logic of a randomized comparative experiment depends on our ability to treat all the subjects the same in every way except for the actual treatments being compared. In a double-blind experiment, neither the subjects nor those who In a double-blind experiment, neither the subjects nor those who interact with them and measure the response variable know which interact with them and measure the response variable know which treatment a subject received. treatment a subject received. The most serious potential weakness of experiments is lack of realism. The subjects or treatments or setting of an experiment may not realistically duplicate the conditions we really want to study. 15 Marc Mehlman Marc Mehlman (University of New Haven) Collecting Data 11 / 28
Recommend
More recommend
