what s in it for me what s in it for me
play

What's in it for me? What's in it for me? Transparently organizing - PowerPoint PPT Presentation

Apr 12, 2023 •12 likes •312 views

What's in it for me? What's in it for me? Transparently organizing your research from start to fi nish Candice C. Morey MRC-CBU Open Science Day Adopting open research practices Scientists endorse openness, but most don't prioritize it

  1. What's in it for me? What's in it for me? Transparently organizing your research from start to fi nish Candice C. Morey MRC-CBU Open Science Day

  2. Adopting open research practices · Scientists endorse openness, but most don't prioritize it · Perceived to be a lot of work · Rewards fuzzy: - Idealistic only? - Maybe important later, but not vital now? 2/30

  3. Strategic concerns 3/30

  4. Strategic concerns · Won't it slow me down? 4/30

  5. Strategic concerns · Why should someone else bene fi t from my work? 5/30

  6. Strategic concerns · What if I fail? 6/30

  7. Science is a collaborative e ff ort You are working together to achieve something bigger. 7/30

  8. Why PIs need in-lab transparency 8/30

  9. Why PIs need in-lab transparency 9/30

  10. Why PIs need in-lab transparency Each student thinks their project is theirs, but it is always part of something bigger. 10/30

  11. Why PIs need in-lab transparency · To ensure consistency in training researchers · To know the provenance and protect the security of data · To stem information loss that comes with turn-over · To make it easy to share publicly when its time 11/30

  12. Components of my open-lab work fl ow · Lab handbook · Open Science Framework · Scripted analysis · Pre-registration 12/30

  13. 1. My lab handbook https://ccmorey.github.io/labHandbook/ 13/30

  14. Why have a lab handbook? · End theory-of-mind games when training new students · Articulate a standard to aspire to - So students know what I would consider "professional" (i.e., worthy of high marks) 14/30

  15. Why make it public? · To get feedback · To be helpful · So you might in fl uence standards · Enrich descriptions of your published method 15/30

  16. 2. The Open Science Framework · OSF: A tool you can use to organize, back-up, and eventually share fi nished components · Sharing your fi nished products != working totally in public - Custom software, stimuli, protocols: they're fi nished when study is ready to be run - Pre-registrations: Before analyzing new data - Anonymized data: when the data set is closed to new members, entries - Analysis code and papers that are ready for scrutiny from someone 16/30

  17. Nice features of OSF · It's free · Control project members, visibility · Can get DOI assigned to project · Good tool for supervision - can see quickly where project stands 17/30

  18. How I use it · Students are introduced to OSF at start · They are added to an existing project or create their own · Finished products (materials, data, writing) are uploaded · They "register" project at important milestones - Creates a back-up - Provides a timestamp in case we need it · The OSF page supplements whatever we publish 18/30

  19. 3. Scripted Analyses · Not part of my training! · Work fl ow I learned in my training was not easy to reproduce 19/30

  20. Principles of scripting · Processing data shouldn't lead to information loss · Script starts from anonymized raw data - Every time · Everything done to the data is recorded in the script - Is documented, can be changed e ffi ciently 20/30

  21. Advantages of scripting · Once you have a routine, it is much faster · Changes can be made quickly and easily · You have a record of every step performed · The analysis is reproducible · Your colleagues (inside and outside the lab) can see exactly what was done 21/30

  22. 4. Pre-registration · Documenting detail of method and analysis plan before analyzing (or collecting) data 22/30

  23. Why it is helpful · Prevents rushing into data collection · Better ensures that students collecting data understand what they are doing (and can provide useful criticism ahead of time) · Makes it clear to students how to proceed with data analysis · Prevents p-hacking 23/30

  24. Does pre-registration slow projects down? · Probably. · But also - prevents wasting time running sub-optimal designs - helps catch confounds - improves pedagogy · Inculcates a lab culture where it is clear that speaking up about design is welcome 24/30

  25. Why these things spec fi cally? · Adopted them because I saw they would solve speci fi c problems I experience directing a lab run largely on short-term student labor - Uncertainty about what students had done in the lab - Or with the data - Wanting to make better use of their e ff orts · Didn't adopt them wholesale, immediately - Was always gradual, picked something I could manage, built on it later 25/30

  26. Openness in the lab makes openness outside it easier · When you are con fi dent about - the provenance of the data - the reproducibility of the analyses - the quality of the design · Less anxiety about sharing outwards · Sharing publicly has been bene fi cial - Increases citations - Reputation bene fi ts - Goodwill from others who appreciate your resources 26/30

  27. Transparency is for your team 27/30

  28. Transparency is also for Science 28/30

  29. Transparency is also for Science But it's mostly for you. 29/30

  30. Thanks for your attention! My data and materials are publicly available on Open Science Framework (https://osf.io/4xwa8) Blogging at The Mnemonic Lode, candicemorey.org Twitter: @CandiceMorey Editor-in-chief, www.journalofcognition.org 30/30

Recommend

More recommend