transfer learning and applications in computational
play

Transfer Learning and Applications in Computational Biology 1 - PowerPoint PPT Presentation

Nov 08, 2022 •245 likes •1.22k views

Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius Kloft, 1 , 3 Gunnar R atsch, 2 Gabriele Schweikert Nico G ornitz, 1 Memorial Sloan-Kettering Cancer Center, NY, USA 2 Technical University of

  1. Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius Kloft, 1 , 3 Gunnar R¨ atsch, 2 Gabriele Schweikert Nico G¨ ornitz, 1 Memorial Sloan-Kettering Cancer Center, NY, USA 2 Technical University of Berlin, Germany 3 New York University, NY, USA

  2. Memorial Sloan-Kettering Cancer Center Frequent words of abstracts from publications 1998-2004. [ wordle.net ] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 2

  3. Memorial Sloan-Kettering Cancer Center Frequent words of abstracts from publications 2005-2012. [ wordle.net ] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 3

  4. Memorial Sloan-Kettering Cancer Center Learning About the Central Dogma of Biology Goal: Learn to predict what these processes accomplish: Given the DNA, . . . , predict all gene products f (DNA, ) = RNA g (RNA, ) = protein Estimating f , g amounts to cracking the codes of transcription, epigenetics, splicing, . . . � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 4

  5. Memorial Sloan-Kettering Cancer Center Learning About the Central Dogma of Biology Goal: Learn to predict what these processes accomplish: Given the DNA, . . . , predict all gene products f (DNA, ) = RNA g (RNA, ) = protein Estimating f , g amounts to cracking the codes of transcription, epigenetics, splicing, . . . � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 4

  6. Memorial Sloan-Kettering Cancer Center Learning About the Central Dogma of Biology Goal: Learn to predict what these processes accomplish: Given the DNA, . . . , predict all gene products f (DNA, ) = RNA g (RNA, ) = protein Estimating f , g amounts to cracking the codes of transcription, epigenetics, splicing, . . . � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 4

  7. Memorial Sloan-Kettering Cancer Center Learning About the Central Dogma of Biology Goal: Learn to predict what these processes accomplish: Given the DNA, . . . , predict all gene products f (DNA, ) = RNA g (RNA, ) = protein Estimating f , g amounts to cracking the codes of transcription, epigenetics, splicing, . . . � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 4

  8. Memorial Sloan-Kettering Cancer Center Learning About the Central Dogma of Biology Three things will be crucial: Biological insights , RNA) N Many observations of the system: (DNA, i =1 Empirical inference to estimate Θ: f Θ (DNA, ) = RNA � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 4

  9. Memorial Sloan-Kettering Cancer Center Learning About the Central Dogma Goal: Estimate f to predict RNAs Need: Good inference method Omit (f Inputs (DNA, ) Outputs (complete transcriptome) Challenges: 1 RNA only partially known 2 Factors Omit (f only partially known 3 Improved inference methods � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 5

  10. Memorial Sloan-Kettering Cancer Center Recent Machine Learning Work � Develop fast, accurate and interpretable learning methods 1 Large scale sequence classification [R¨ atsch et al., 2006a; Sonnenburg et al., 2010, 2007; Toussaint et al., 2010] 2 Analysis and explanation of learning results [R¨ atsch et al., 2006b; Sonnenburg et al., 2008; Zien et al., 2009] 3 Sequence segmentation & structure prediction [R¨ atsch et al., 2007; Schweikert et al., 2009; Zeller et al., 2008] 4 Transfer & Multitask learning [Schweikert et al., 2008a; Widmer et al., 2011, 2012, 2010a; Widmer and R¨ atsch, 2011; Widmer et al., 2010c] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 6

  11. Memorial Sloan-Kettering Cancer Center Recent Machine Learning Work � Develop fast, accurate and interpretable learning methods 1 Large scale sequence classification [R¨ atsch et al., 2006a; Sonnenburg et al., 2010, 2007; Toussaint et al., 2010] 2 Analysis and explanation of learning results [R¨ atsch et al., 2006b; Sonnenburg et al., 2008; Zien et al., 2009] 3 Sequence segmentation & structure prediction [R¨ atsch et al., 2007; Schweikert et al., 2009; Zeller et al., 2008] 4 Transfer & Multitask learning [Schweikert et al., 2008a; Widmer et al., 2011, 2012, 2010a; Widmer and R¨ atsch, 2011; Widmer et al., 2010c] 8 7 k−mer Length 6 5 4 3 2 1 −30 −20 −10 0 10 20 30 Position � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 6

  12. Memorial Sloan-Kettering Cancer Center Recent Machine Learning Work � Develop fast, accurate and interpretable learning methods 1 Large scale sequence classification [R¨ atsch et al., 2006a; Sonnenburg et al., 2010, 2007; Toussaint et al., 2010] 2 Analysis and explanation of learning results [R¨ atsch et al., 2006b; Sonnenburg et al., 2008; Zien et al., 2009] 3 Sequence segmentation & structure prediction [R¨ atsch et al., 2007; Schweikert et al., 2009; Zeller et al., 2008] 4 Transfer & Multitask learning [Schweikert et al., 2008a; Widmer et al., 2011, 2012, 2010a; Widmer and R¨ atsch, 2011; Widmer et al., 2010c] 8 7 10 k−mer Length 6 Log-intensity 5 4 3 5 2 1 transcript −30 −20 −10 0 10 20 30 Position 0 � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 6

  13. Memorial Sloan-Kettering Cancer Center Recent Machine Learning Work � Develop fast, accurate and interpretable learning methods 1 Large scale sequence classification [R¨ atsch et al., 2006a; Sonnenburg et al., 2010, 2007; Toussaint et al., 2010] 2 Analysis and explanation of learning results [R¨ atsch et al., 2006b; Sonnenburg et al., 2008; Zien et al., 2009] 3 Sequence segmentation & structure prediction [R¨ atsch et al., 2007; Schweikert et al., 2009; Zeller et al., 2008] 4 Transfer & Multitask learning [Schweikert et al., 2008a; Widmer et al., 2011, 2012, 2010a; Widmer and R¨ atsch, 2011; Widmer et al., 2010c] 8 7 10 k−mer Length 6 Log-intensity 5 4 3 5 2 1 transcript −30 −20 −10 0 10 20 30 Position 0 � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 6

  14. Memorial Sloan-Kettering Cancer Center Many algorithms implemented in Shogun toolbox (GPL, ≥ 1000 users) � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 7

  15. Memorial Sloan-Kettering Cancer Center Roadmap Motivation from computational biology TSS Donor Acceptor Donor Acceptor polyA/cleavage DNA TIS Stop Empirical comparison of domain adaptation algorithms Algorithms for hierarchical multi-task learning Algorithms for learning task relations Fast(er) Algorithms Discussion & Conclusion � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 8

  16. Memorial Sloan-Kettering Cancer Center A Core CompBio Problem: Gene Finding DNA genic intergenic pre-mRNA exon intron exon intron exon mRNA 5' UTR 3' UTR cap polyA Protein Given a piece of DNA sequence Predict gene products including intermediate processing steps Predict signals used during processing Predict the correct corresponding label sequence with labels � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 9

  17. Memorial Sloan-Kettering Cancer Center A Core CompBio Problem: Gene Finding TSS polyA/cleavage DNA Splice Splice Splice Splice pre-mRNA Donor Acceptor Donor Acceptor mRNA TIS Stop cap polyA Protein Given a piece of DNA sequence Predict gene products including intermediate processing steps Predict signals used during processing Predict the correct corresponding label sequence with labels � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 9

  18. Memorial Sloan-Kettering Cancer Center A Core CompBio Problem: Gene Finding TSS Donor Acceptor polyA/cleavage Donor Acceptor DNA TIS Stop pre-mRNA mRNA cap polyA Protein Given a piece of DNA sequence Predict gene products including intermediate processing steps Predict signals used during processing Predict the correct corresponding label sequence with labels � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 9

  19. Memorial Sloan-Kettering Cancer Center A Core CompBio Problem: Gene Finding TSS Donor Acceptor polyA/cleavage Donor Acceptor DNA TIS Stop pre-mRNA mRNA cap polyA Protein Given a piece of DNA sequence Predict gene products including intermediate processing steps Predict signals used during processing Predict the correct corresponding label sequence with labels � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology Courant Institute@NYU February 7, 2013 9

Recommend

Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius

Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius

Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius Kloft, 1 , 3 , 4 Gunnar R atsch, Nico G ornitz, Gabriele Schweikert 1 Memorial Sloan-Kettering Cancer Center, NY, USA 2 Microsoft Research, Los

1.04k views • 76 slides
Effective transfer learning for clinical applications Benjamin van der Burgh (LIACS) OVERVIEW

Effective transfer learning for clinical applications Benjamin van der Burgh (LIACS) OVERVIEW

Effective transfer learning for clinical applications Benjamin van der Burgh (LIACS) OVERVIEW 1. Transfer learning in NLP 2. Experiments on Dutch data 3. Well-being tracking using clinical journals 2 PROJECT BACKGROUND Physiotherapists

464 views • 25 slides
Industrial Transfer Learning Introduction to Industrial Transfer Learning Industrial Transfer

Industrial Transfer Learning Introduction to Industrial Transfer Learning Industrial Transfer

Industrial Transfer Learning Introduction to Industrial Transfer Learning Industrial Transfer Learning Motivation Machine Learning in Manufacturing Decision Support Automation Process Control Self-Optimization Predictive Quality Predictive

742 views • 24 slides
Introduction to Machine Learning Deep Learning Applications Barnabs Pczos Applications

Introduction to Machine Learning Deep Learning Applications Barnabs Pczos Applications

Introduction to Machine Learning Deep Learning Applications Barnabs Pczos Applications Image Classification (Alexnet, VGG, Resnet) on Cifar 10, Cifar 100, Mnist, Imagenet Art Neural style transfer on images and videos

1.19k views • 83 slides
Transfer Learning Eu Wern Teh What are we covering? Why transfer learning? Fine

Transfer Learning Eu Wern Teh What are we covering? Why transfer learning? Fine

Transfer Learning Eu Wern Teh What are we covering? Why transfer learning? Fine Tuning: how? why? Example Practise: Ants and Bees dataset Caltech 101 dataset Why Transfer Learning? Speedup Training

584 views • 24 slides
Improving Domain-specific Transfer Learning Applications for Image Recognition and Differential

Improving Domain-specific Transfer Learning Applications for Image Recognition and Differential

Improving Domain-specific Transfer Learning Applications for Image Recognition and Differential Equations M.Sc. Thesis in Computer Science and Engineering Candidates: Alessandro Saverio Paticchio, Tommaso Scarlatti Advisor : Prof. Marco

710 views • 42 slides
Transfer learning and domain adaptation Semi-supervised and transfer learning Myth : you cant

Transfer learning and domain adaptation Semi-supervised and transfer learning Myth : you cant

Day 2 Lecture 5 Transfer learning and domain adaptation Semi-supervised and transfer learning Myth : you cant do deep learning unless you have a million labelled examples for your problem. Reality You can learn useful representations

183 views • 15 slides
Heat Transfer Heat Transfer Introduction Practical occurrences, applications, factors

Heat Transfer Heat Transfer Introduction Practical occurrences, applications, factors

Heat Transfer Heat Transfer Introduction Practical occurrences, applications, factors affecting heat transfer Categories and modes of heat transfer Conduction In a slab and across a pipe Convection Free

516 views • 39 slides
Machine Learning HMM applications in computational biology Central dogma CCTGAGCCAACTATTGATGAA

Machine Learning HMM applications in computational biology Central dogma CCTGAGCCAACTATTGATGAA

10-701 Machine Learning HMM applications in computational biology Central dogma CCTGAGCCAACTATTGATGAA DNA transcription mRNA CCUGAGCCAACUAUUGAUGAA translation Protein PEPTIDE 2 Biological data is rapidly accumulating Transcription

703 views • 36 slides
Applications of Machine Learning in Computational Biology Narges Razavian New York University

Applications of Machine Learning in Computational Biology Narges Razavian New York University

Applications of Machine Learning in Computational Biology Narges Razavian New York University Slides thanks to James Galagan@Board Institute Su-In Lee@Univ of Washington Rainer Breitling@ Univ of Glasgow Christopher M. Bishop @ ECCV

804 views • 41 slides
1. Computational Fluid a. Computational Fluid Dynamics is in the domain of Computational Science

1. Computational Fluid a. Computational Fluid Dynamics is in the domain of Computational Science

1. Computational Fluid a. Computational Fluid Dynamics is in the domain of Computational Science b. Applications i. Computational Fluid Dynamics have many applications. Some applications include: 1. Automotive Aerodynamics 2. Designing HVAC

192 views • 3 slides
Meta-transfer Learning for Few-shot Learning Yaoyao Liu Tianjin University and NUS School of

Meta-transfer Learning for Few-shot Learning Yaoyao Liu Tianjin University and NUS School of

NUS-Tsinghua-Southampton Centre for Extreme Search Meta-transfer Learning for Few-shot Learning Yaoyao Liu Tianjin University and NUS School of Computing OUTLINE Research Background Methods Meta-transfer Learning Hard-task

455 views • 33 slides
Transfer Learning Approach for Botnet Detection based on Recurrent Variational Autoencoder

Transfer Learning Approach for Botnet Detection based on Recurrent Variational Autoencoder

Transfer Learning Approach for Botnet Detection based on Recurrent Variational Autoencoder Jeeyung Kim Scientific Data Management Research Group Computational Research Division Lawrence Berkeley National Laboratory 2020 SNTA, 06/02/2020 J.

423 views • 21 slides
Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization

413 views • 28 slides
Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and

Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and

Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and contextualized word embeddings Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Institute of Computational Perception

1.11k views • 51 slides
Parameter-Efficient Transfer Learning for NLP N. Houlsby, A. Giurgiu*, S. Jastrzbski*, B.

Parameter-Efficient Transfer Learning for NLP N. Houlsby, A. Giurgiu*, S. Jastrzbski*, B.

Parameter-Efficient Transfer Learning for NLP N. Houlsby, A. Giurgiu*, S. Jastrzbski*, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, S. Gelly Imagine doing Transfer Learning for NLP Ingredients: A large pretrained model

739 views • 15 slides
Applications of Computational Intelligence to Medicine and Health Manuel Graa Computational

Applications of Computational Intelligence to Medicine and Health Manuel Graa Computational

Applications of Computational Intelligence to Medicine and Health Manuel Graa Computational Intelligence Group UPV/EHU InMed 2013, Piraeus, Greece, 19 July 1 Contents Motivation and main lines Brain Image biomarkers Vessel Image

649 views • 48 slides
Modern Computational Statistics Lecture 20: Applications in Computational Biology Cheng Zhang

Modern Computational Statistics Lecture 20: Applications in Computational Biology Cheng Zhang

Modern Computational Statistics Lecture 20: Applications in Computational Biology Cheng Zhang School of Mathematical Sciences, Peking University December 09, 2019 Introduction 2/23 While modern statistical approaches have been quite

1.18k views • 54 slides
Exploring the Limits of Transfer Learning with a unified Text-to-Text Transformer Presented by -

Exploring the Limits of Transfer Learning with a unified Text-to-Text Transformer Presented by -

Exploring the Limits of Transfer Learning with a unified Text-to-Text Transformer Presented by - Vipul Rathore Elements and images borrowed from Raffel et al., 2019 Transfer Learning: Background Pre-train a model on a data-rich task

1.21k views • 52 slides
Radiative transfer models in the internet- accessible information-computational system

Radiative transfer models in the internet- accessible information-computational system

Radiative transfer models in the internet- accessible information-computational system Atmospheric radiation Firsov . . 1 , Chesnokova T.Yu. 2 , Voronina Yu.V. 2 , Kozodoeva E.V. 2 1-Volgograd State University, 100, Av.

323 views • 17 slides
W ITH the widespread use of hands-free electronic gad- are mapped to a multilingual set using a

W ITH the widespread use of hands-free electronic gad- are mapped to a multilingual set using a

1 Transfer learning for cross-lingual automatic speech recognition Amit Das Abstract In this study, two instance based transfer learning languages. With this, all language dependent transcriptions can phoneme modeling approaches are

408 views • 5 slides
Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

654 views • 44 slides
Big Transfer (BiT): General Visual Representation Learning Abhash Kumar Singh Harit Vishwakarma

Big Transfer (BiT): General Visual Representation Learning Abhash Kumar Singh Harit Vishwakarma

Big Transfer (BiT): General Visual Representation Learning Abhash Kumar Singh Harit Vishwakarma Short summary Transfer learning helps improve sample efficiency and simplifies hyperparameter tuning when training DNNs for vision. The authors

349 views • 11 slides
Learning and transfer (Chapter 3 from How People Learn) (1999) M. Suzanne Donovan, John D.

Learning and transfer (Chapter 3 from How People Learn) (1999) M. Suzanne Donovan, John D.

Learning and transfer (Chapter 3 from How People Learn) (1999) M. Suzanne Donovan, John D. Bransford, and James W. Pellegrino Introduction What is transfer of learning the ability to extend what has been learned in one context to new

661 views • 17 slides

More recommend