Basic Experimental Design Basic Concepts in Experimental Design - PowerPoint PPT Presentation

Basic Experimental Design Basic Concepts in Experimental Design Prof. Dr. Luc Duchateau Ghent University 2018 UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 1 / 26

Basic concepts in experimental design Overview Overview What do we want to test? Specifying statistical hypotheses What tools do we use? Random assignment Blocking Blinding What are the building blocks of an experiment? Experimental unit versus observational unit Replication versus repeated measures How much resources do we need? Power analysis UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 2 / 26

Basic concepts in experimental design Specifying statistical hypotheses Specifying statistical hypotheses Scientific hypothesis TRANSLATION Statistical hypothesis UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 3 / 26

Basic concepts in experimental design Specifying statistical hypotheses Example: Mastitis trial in dairy cows Observe somatic cell count (SCC) every 3 hours in 24 hours timespan Observe milk reduction 24 hours after infusion 10 4 CFU Low E.Coli MASTITIS infusion 10 6 CFU High UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 4 / 26

Basic concepts in experimental design Specifying statistical hypotheses Mastitis: Scientific question Evaluate the effect of the infusion dose on SCC! → FAR TOO GENERAL → NOT TESTABLE → TRANSLATION REQUIRED UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 5 / 26

Basic concepts in experimental design Specifying statistical hypotheses Mastitis: (Rather specific) statistical hypotheses → SCC is on average higher in high dose group H 0 : µ H,Av = µ L,Av H a : µ H,Av > µ L,Av versus → Maximum SCC is higher in high dose group H 0 : µ H,M = µ L,M versus H a : µ H,M > µ L,M → SCC increases faster in high dose group H 0 : β H = β L H a : β H > β L versus → time to attain a certain SCC treshold value is less in high dose group H 0 : τ H = τ L versus H a : τ H < τ L UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 6 / 26

Basic concepts in experimental design Specifying statistical hypotheses Mastitis: More general hypotheses → Compare SCC at each time point BEWARE of multiple testing! → Global hypotheses with dose, time and their interaction in the model UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 7 / 26

Basic concepts in experimental design Specifying statistical hypotheses Mastitis: one- versus two-sided statistical hypotheses Compare two species, Escherichia coli and Staphylococcus aureus for their effect on milk production reduction ρ → Is there a difference between the two species? H 0 : ρ E = ρ S versus H a : ρ E � = ρ S → Is the reduction larger with Escherichia coli ? H 0 : ρ E = ρ S H a : ρ E > ρ S versus → Are the two species equivalent? UGent STATS VM H 0 : | ρ E − ρ S | = ∆ H a : | ρ E − ρ S | < ∆ versus Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 8 / 26

Basic concepts in experimental design The tools Randomisation All variables with possible effect on response are distributed randomly over treatment groups → guard against unknown confounders In experiments, treatments are randomly assigned as compared to observational studies To show causal relationship, randomisation is required! Observational studies can only show associations UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 9 / 26

Basic concepts in experimental design The tools Blocking Experimental units can be heterogeneous Group experimental units in blocks of more homogeneous units Optimal is when all treatments appear in a block ⇒ Compare treatments within block UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 10 / 26

Basic concepts in experimental design The tools Example blocking Difference in milk reduction with high and low CFU dose of E. coli Herds are the blocks; cows are more similar within herd Herd 1 Herd 2 Herd 3 L H L H L H 0.9 7.4 2.3 6.6 3.4 8.8 2.0 6.8 2.7 7.1 2.7 8.3 2.0 7.1 1.3 6.2 2.9 7.9 2.2 6.7 1.6 7.8 3.0 8.2 2.0 7.9 2.1 7.2 3.4 8.1 UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 11 / 26

Basic concepts in experimental design The tools Blinding (1) Blind investigator for assessing response (2) Blind subject for treatment received ⇓ Takes care of placebo effect (1) and (2): DOUBLE BLIND UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 12 / 26

Basic concepts in experimental design Building blocks of an experiment Experimental versus observational units Experimental unit: entity to which a treatment is randomly assigned Observational unit: entity to which a response variable is measured Sometimes experimental unit = observational unit Sometimes more than 1 type of experimental unit UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 13 / 26

Basic concepts in experimental design Building blocks of an experiment Mastitis: experimental unit=observational unit Infusion Dose Low High One herd L L Experimental unit = cow = observational unit H H UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 14 / 26

Basic concepts in experimental design Building blocks of an experiment Mastitis: experimental unit � = observational unit herd 1 herd 2 L L H H L L H H Dose assigned to herd → Herd = experimental unit Cow = observational unit UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 15 / 26

Basic concepts in experimental design Building blocks of an experiment Mastitis: experimental unit � = observational unit herd 1 herd 2 L L H H L L H H Dose assigned to herd → Herd = experimental unit Cow = observational unit UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 15 / 26

Basic concepts in experimental design Building blocks of an experiment Mastitis: experimental unit � = observational unit herd 1 herd 2 L L H H L L H H Dose assigned to herd → Herd = experimental unit Cow = observational unit herd 1 herd 2 herd 3 herd 4 L L H H H H L L UGent L L H H H H L L STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 15 / 26

Basic concepts in experimental design Building blocks of an experiment Mastitis: two types of experimental units Dose + Parity Low High Heifer Multiparous H herd 1 L herd 2 L herd 3 H herd 4 M M H M M H H H H H H M M H M M Experimental unit for DOSE: HERD Experimental unit for PARITY: ANIMAL UGent Observational unit: ANIMAL STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 16 / 26

Basic concepts in experimental design Building blocks of an experiment Replication versus Repeated measure Random assignment of treatment → REPLICATION Different observations on 1 experimental unit → REPEATED MEASURES in time/space Replication → information about variation between experimental units Repeated measure → better assessment of experimental unit UGent Var(mean) < Var(observation) STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 17 / 26

Basic concepts in experimental design Building blocks of an experiment Repeated measures in time Cow with HIGH Cow with LOW infusion dose infusion dose SCC at 3h SCC at 3h SCC at 6h SCC at 6h SCC at 9h SCC at 9h UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 18 / 26

Basic concepts in experimental design Building blocks of an experiment Repeated measures in time Cow with HIGH Cow with LOW infusion dose infusion dose SCC at 3h SCC at 3h SCC at 6h SCC at 6h SCC at 9h SCC at 9h DO NOT ANALYSE SUCH DATA UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 18 / 26

Basic concepts in experimental design Building blocks of an experiment Repeated measures in space Vaccination for mastitis herd 1 herd 2 all LOW dose all HIGH dose → No replications, only repeated measures Difference between HIGH and LOW infusion dose = difference between two herds UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 19 / 26

Basic concepts in experimental design Power analysis Power analysis Objective of experiment: Reject H 0 Failure to reject H 0 can be due to (1) H 0 is correct (2) Bad luck, unrepresentative sample (3) Large difference, but not enough evidence (3) can be avoided by power analysis UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 20 / 26

Basic concepts in experimental design Power analysis Sample size determination The required sample is determined by 4 parameters Probability of type I error α Probability of type II error β , or power=1- β The variance between experimental units σ 2 True underlying difference, e.g., for comparing two means: ∆ = µ 1 − µ 2 UGent STATS VM Prof. Dr. Luc Duchateau (UGent) Basic Experimental Design 2018 21 / 26