Opening Up Scientific Methodology for Collaboration NFAIS OA - PowerPoint PPT Presentation

Opening Up Scientific Methodology for Collaboration NFAIS OA Conference, October 3, 2017 @lteytelman lenny@protocols.io

The Problem - Communication of methods Lessons from the Cancer Reproducibility project (January 2017): Scientists and journals need to get better at describing their methods and sharing data …. in many cases because they [protocols] had been developed by students and postdocs who were no longer with the lab. The hardest part, by far, was figuring out exactly what the original labs actually did. Scientific papers come with methods sections that theoretically ought to provide recipes for doing the same experiments. But often, those recipes are incomplete, missing out important steps, details, or ingredients. In some cases, the recipes aren’t described at all; researchers simply cite an earlier study that used a similar technique…

The Problem - Communication of methods Lessons from the Cancer Reproducibility project (January 2017): Scientists and journals need to get better at describing their methods and sharing data …. in many cases because they [protocols] had been developed by students and postdocs who were no longer with the lab. The hardest part, by far, was figuring out exactly what the original labs actually did. Scientific papers come with methods sections that theoretically ought to provide recipes for doing the same experiments. But often, those recipes are incomplete, missing out important steps, details, or ingredients. In some cases, the recipes aren’t described at all; researchers simply cite an earlier study that used a similar technique… 1. Missing method details (“contact authors” & “as reported elsewhere”)

The Problem - Communication of methods Lessons from the Cancer Reproducibility project (January 2017): Scientists and journals need to get better at describing their methods and sharing data …. in many cases because they [protocols] had been developed by students and postdocs who were no longer with the lab. The hardest part, by far, was figuring out exactly what the original labs actually did. Scientific papers come with methods sections that theoretically ought to provide recipes for doing the same experiments. But often, those recipes are incomplete, missing out important steps, details, or ingredients. In some cases, the recipes aren’t described at all; researchers simply cite an earlier study that used a similar technique… 1. Missing method details (“contact authors” & “as reported elsewhere”) 2. Once published, impossible to correct and keep up-to-date

A few examples

Small typo in a protocol. Question from researcher. Author replies and corrects in a new version.

Small typo in a protocol. Question from researcher. Author replies and corrects in a new version.

Traction (adoption) Private protocols: 4,000 Public protocols: > 2,000 Cumulative public protocols 2500 2000 1500 1000 500 0

Traction (partnerships) 80 60 Total partners 40 20 0 Journals (50) Reagent Vendors (19) Biotech Accelerators (10)

Traction (partnerships) 250 200 Total partners 150 100 50 0 Journals (180) Reagent Vendors (19) Biotech Accelerators (10)

Traction (adoption) Scientists creating private protocols (monthly) 250 200 150 100 50 0

Business Model Private groups Monthly dues to keep protocols visible only to group members Vendor analytics Subscription fee to access aggregated usage statistics

Long-term preservation Public APIs: apidocs.protocols.io

Prerequisite Questions (The classic for/non-profit debate is misguided.) What is the business model? How secure is it? Are are they sustainable? Are they open access? Will they sell my e-mail? Do they have public APIs? Do they do backups? What are long-term preservation plans?

Prerequisite Questions (The classic for/non-profit debate is misguided.) What is the business model? How secure is it? Are are they sustainable? Are they open access? Will they sell my e-mail? Do they have public APIs? Do they do backups? What are long-term preservation plans?

Acknowledgements Alexei Irina Anjuli Lenny Susan Stoliartchouk Makkaveeva Manche Teytelman Pastore CTO, cofounder CFO, cofounder Partnerships CEO, cofounder Commerce Monika Yulia Nick Sergey Denise Vladimir Khassan Kurnosova Gulev Alekseev Ting Frolov Development Proj Manager Development Development Development UI/UX