E xpe rime nts De sig n a nd Ana lysis F o tis E . Pso mo po ulo - PowerPoint PPT Presentation

E xpe rime nts De sig n a nd Ana lysis F o tis E . Pso mo po ulo s CODAT A-RDA Advanc e d Bio info rmatic s Wo rksho p, 20-24 Aug ust 2018, T rie ste , I taly

A sho rt intro … to me  2  Bio info rma tic s a nd Da ta Mining  to o ls a nd pipe line s to a ddre ss do ma in-spe c ific q ue stio ns  g e no me -a wa re me tho ds Bio info rma tic s  Bio info rma tic s a nd Clo ud Co mputing  wo rkflo ws a nd pipe line s o n c lo ud infra struc ture s Clo ud Da ta  sta nda rdiza tio n a nd re usa b ility Co mputing Mining E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

Bio a na lytic s Gro up @ I NAB|CE RT H 3 Re se a rc h  NGS Wo rkflo ws  Omic s Da ta I nte g ra tio n  Da ta Mining T ra ining Pe o ple NGS Da ta Ana lysis using 1 st So ftwa re Ca rpe ntry  Ma ria K o to uza , PhD Stud e nt Clo ud Co mputing (Oc t 2015) Wo rksho p (Oc t 2016)  Ma ria T sa yo po ulo u, PhD Stud e nt  Atha na ssio s K intsa kis, PhD Stud e nt E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018 CE RT H Main Building

Why do we pe rfo rm e xpe rime nts? 4 Go to www.me nti.c om and use the c ode 45 89 48 E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

Wha t is a n e xpe rime nt? 5 An e xpe rime nt is c ha ra c te rize d b y the tr e atme nts a nd ime ntal units to b e use d, the wa y tre a tme nts a re e xpe r assigne d to units, a nd the r e sponse s tha t a re me a sure d. 1. E xpe rime nts a llo w us to se t up a dire c t c o mpa riso n b e twe e n the tre a tme nts o f inte re st. 2. We c a n de sig n e xpe rime nts to minimize a ny b ia s in the c o mpa riso n. 3. We c a n de sig n e xpe rime nts so tha t the e rro r in the c o mpa riso n is sma ll. 4. Mo st impo rta nt, we a re in c o ntro l o f e xpe rime nts, a nd ha ving tha t c o ntro l a llo ws us to ma ke stro ng e r infe re nc e s a b o ut the na ture o f diffe re nc e s tha t we se e in the e xpe rime nt. Spe c ific a lly, we ma y ma ke infe re nc e s a b o ut c a usa tio n. E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

Co mpo ne nts o f a n E xpe rime nt 6 T re a tme nts, units, a nd a ssig nme nt me tho d spe c ify the e xpe rime nta l de sig n  An a lte rna tive de finitio n is:  “tre a tme nt de sig n” is the se le c tio n o f tre a tme nts to b e use d  “e xpe rime nt de sig n” is the se le c tio n o f units a nd a ssig nme nt o f tre a tme nts  No te tha t the re is no me ntio n o f a me tho d fo r a na lyzing the re sults.  a na lysis is not pa rt o f the de sig n  Ho we ve r: it is o fte n use ful to c o nside r the a na lysis whe n pla nning a n e xpe rime nt. E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

Why T hink Ab o ut E xpe rime nta l De sig n? 7 E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

Crisis in Re pro duc ib le Re se a rc h 8 E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018 http://ne ilfws.g ithub .io /Pub Me d/pmre trac t/pmre trac t.html

Co nse q ue nc e s o f Po o r E xpe rime nta l 9 De sig n…  Cost o f e xpe rime nta tio n. We ha ve a re spo nsib ility to do no rs!  L e c ious ma te ria l imite d & Pr e sp. c linic a l sa mple s.  Immor talization o f da ta se ts in pub lic da ta b a se s a nd me tho ds in the lite ra ture . Our b a d sc ie nc e b e g e ts mo re b a d sc ie nc e .  E ns o f e xpe rime nta tio n: a nima ls a nd c linic a l sa mple s. thic al c onc e r Slide s adapte d fro m “De sig ning F unc tio nal Ge no mic s E xpe rime nts fo r Suc c e ssful Analysis”, b y Ro ry Stark, 18/09/2017, CRUK-CI E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

So , wha t is a g o o d e xpe rime nta l de sig n? 10 Go to www.me nti.c om and use the c ode 45 89 48 E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

A g o o d e xpe rime nt de sig n 11  No t a ll e xpe rime nta l de sig ns a re c re a te d e q ua l!  A g o o d e xpe rime nta l de sig n must 1. Avo id syste ma tic e rro r 2. Be pre c ise 3. Allo w e stima tio n o f e rro r 4. Ha ve b ro a d va lidity  L e t’ s se e the se a spe c ts o ne a t a time ! Slide s adapte d fro m Gary W. Oe hle rt, “A F irst Co urse in De sig n and Analysis o f E xpe rime nts”, 2010 - I SBN 0-7167-3510-5 E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

1. De sig n to a vo id syste ma tic e rro r 12  Co mpa ra tive e xpe rime nts e stima te diffe re nc e s in re spo nse b e twe e n tre a tme nts.  I f a n e xpe rime nt ha s syste ma tic e rro r, the n the c o mpa riso ns will b e b ia se d, no ma tte r ho w pre c ise o ur me a sure me nts a re o r ho w ma ny e xpe rime nta l units we use . I f re spo nse s fo r units re c e iving tr e atme nt one a re me a sure d with instr ume nt A a nd re spo nse s fo r tr e atme nt two a re me a sure d with instr ume nt B , the n we do n’ t kno w if a ny o b se rve d diffe re nc e s a re due to tre a tme nt e ffe c ts o r instrume nt misc a lib ra tio ns. E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

2. De sig n to inc re a se pre c isio n 13  E ve n witho ut syste ma tic e rro r, the re will b e ra ndo m e rro r in the re spo nse s, a nd this will le a d to ra ndo m e rro r in the tre a tme nt c o mpa riso ns.  E xpe rime nts a re pre c ise whe n this ra ndo m e rro r in tre a tme nt c o mpa riso ns is sma ll.  Pre c isio n de pe nds o n the size o f the ra ndo m e rro rs in the re spo nse s, the numb e r o f units use d, a nd the e xpe rime nta l de sig n use d. E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

3. De sig n to e stima te e rro r 14  E xpe rime nts must b e de sig ne d so tha t we ha ve a n e stima te o f the size o f ra ndo m e rro r. We will se e those in pra c tic e la te r.  T his pe rmits sta tistic a l infe re nc e :  fo r e xa mple , c o nfide nc e inte rva ls o r te sts o f sig nific a nc e .  We c annot do infe r ! Sa dly, e xpe rime nts e nc e without an e stimate of e r r or tha t c a nno t e stima te e rro r c o ntinue to b e run. E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

4. De sig n to wide n va lidity 15  T he c o nc lusio ns we dra w fro m a n e xpe rime nt a re a pplic a b le to the e xpe rime nta l units we use d in the e xpe rime nt.  I f the units a re a c tua lly a sta tistic a l sa mple fro m so me po pula tio n o f units, the n the c o nc lusio ns a re a lso va lid fo r the po pula tio n.  Be yo nd this, we a re e xtra po la ting , a nd the e xtra po la tio n mig ht o r mig ht no t b e suc c e ssful. We c o mpa re two diffe re nt drug s fo r tre a ting a tte ntio n de fic it diso rde r a nd o ur sub je c ts a re pr e - adole sc e nt boys fro m our c linic . We mig ht ha ve a fa ir c a se tha t o ur re sults wo uld ho ld fo r pre -a do le sc e nt b o ys e lse whe re , • b ut e ve n tha t mig ht no t b e true if o ur c linic ’ s po pula tio n o f sub je c ts is unusua l in so me wa y. T he re sults a re e ve n le ss c o mpe lling fo r o lde r b o ys o r fo r g irls. • E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

K e e ping a c o mmo n vo c a b ula ry 16 1. T re a tme nts 2. E xpe rime nta l units 3. Re spo nse s 4. Me a sure me nt units 5. Ra ndo miza tio n 6. Co ntro l 7. F a c to rs 8. Co nfo unding 9. E xpe rime nta l E rro r 10.Blinding E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

T e rms a nd c o nc e pts (1/ 5) 17 e atme nts a re the diffe re nt pro c e dure s we wa nt to c o mpa re . 1. T r  diffe re nt kinds o r a mo unts o f fe rtilize r in a g ro no my  diffe re nt lo ng dista nc e ra te struc ture s in ma rke ting  diffe re nt te mpe ra ture s in a re a c to r ve sse l in c he mic a l e ng ine e ring ime ntal units a re the thing s to whic h we a pply the tre a tme nts. 2. E xpe r  plo ts o f la nd re c e iving fe rtilize r  g ro ups o f c usto me rs re c e iving diffe re nt ra te struc ture s  b a tc he s o f fe e dsto c k pro c e ssing a t diffe re nt te mpe ra ture s E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018

T e rms a nd c o nc e pts (2/ 5) 18 Re sponse s a re o utc o me s tha t we o b se rve a fte r a pplying a tre a tme nt to a n 3. e xpe rime nta l unit (a me a sure o f wha t ha ppe ne d in the e xpe rime nt; we o fte n ha ve mo re tha n o ne re spo nse )  nitro g e n c o nte nt o r b io ma ss o f c o rn pla nts  pro fit b y c usto me r g ro up  yie ld a nd q ua lity o f the pro d uc t pe r to n o f ra w ma te ria l e me nt units (o r re spo nse units) a re the a c tua l o b je c ts o n whic h the 4. Me a sur re spo nse is me a sure d. T he se ma y diffe r fro m the e xpe rime nta l units.  (e .g . in d iffe re nt fe rtilize rs o n the nitro g e n c o nte nt o f c o rn pla nts) Diffe re nt fie ld plo ts a re the e xpe rime nta l units, b ut the me a sure me nt units mig ht b e a sub se t o f the c o rn pla nts o n the fie ld plo t, o r a sa mple o f le a ve s, sta lks, a nd ro o ts fro m the fie ld plo t. E xpe rime nt De sig n a nd Ana lysis Mo nda y, Aug ust 20th 2018