transfer learning and applications in computational
play

Transfer Learning and Applications in Computational Biology 1 - PowerPoint PPT Presentation

Nov 16, 2023 •264 likes •1.04k views

Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius Kloft, 1 , 3 , 4 Gunnar R atsch, Nico G ornitz, Gabriele Schweikert 1 Memorial Sloan-Kettering Cancer Center, NY, USA 2 Microsoft Research, Los

  1. Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius Kloft, 1 , 3 , 4 Gunnar R¨ atsch, Nico G¨ ornitz, Gabriele Schweikert 1 Memorial Sloan-Kettering Cancer Center, NY, USA 2 Microsoft Research, Los Angeles, USA 3 Courant Institute, NYU, New York, USA 4 Humbolt University, Berlin, Germany

  2. Memorial Sloan-Kettering Cancer Center Roadmap Motivation from computational biology TSS Donor Acceptor Donor Acceptor polyA/cleavage DNA TIS Stop Empirical comparison of domain adaptation algorithms Algorithms for hierarchical multi-task learning Algorithms for learning task relations Fast(er) Algorithms Discussion & Conclusion � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 2

  3. Memorial Sloan-Kettering Cancer Center A Core CompBio Problem: Gene Finding DNA genic intergenic pre-mRNA exon intron exon intron exon mRNA 5' UTR 3' UTR cap polyA Protein Given a piece of DNA sequence Predict gene products including intermediate processing steps Predict signals used during processing Predict the correct corresponding label sequence with labels � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 3

  4. Memorial Sloan-Kettering Cancer Center A Core CompBio Problem: Gene Finding TSS polyA/cleavage DNA Splice Splice Splice Splice pre-mRNA Donor Acceptor Donor Acceptor mRNA TIS Stop cap polyA Protein Given a piece of DNA sequence Predict gene products including intermediate processing steps Predict signals used during processing Predict the correct corresponding label sequence with labels � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 3

  5. Memorial Sloan-Kettering Cancer Center A Core CompBio Problem: Gene Finding TSS Donor Acceptor polyA/cleavage Donor Acceptor DNA TIS Stop pre-mRNA mRNA cap polyA Protein Given a piece of DNA sequence Predict gene products including intermediate processing steps Predict signals used during processing Predict the correct corresponding label sequence with labels � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 3

  6. Memorial Sloan-Kettering Cancer Center A Core CompBio Problem: Gene Finding TSS Donor Acceptor polyA/cleavage Donor Acceptor DNA TIS Stop pre-mRNA mRNA cap polyA Protein Given a piece of DNA sequence Predict gene products including intermediate processing steps Predict signals used during processing Predict the correct corresponding label sequence with labels � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 3

  7. Memorial Sloan-Kettering Cancer Center Example: Splice Site Recognition True Splice Sites CT...GTCGTA...GAAGCTAGGAGCGC...ACGCGT...GA 150 nucleotides window around dimer ≈ GCCAATATTTTTCTATTCAGGTGCAATCAATCACCCATCAT 1 1 ATTGAATGAACATATTCCAGGGTCTCCTTCCACCTCAACAA 1 AGCAACGAACTCCATTACAGCAAGGACATCGAAGTCGATCA 1 GCCAATTTTTGACCTTGCAGAATCAATCGTGCACGTTCGGA -1 CATCTGAAATTTCCCCCAAGTATAGCGGAAATAGACCGACG GAAATTTCCCCCAAGTATAGCGGAAATAGACCGACGAAATC -1 -1 CCCAAGTATAGCGGAAATAGACCGACGAAATCGCTCTCTCC -1 AATCGCTCTCTCCCTGGGAGCGATGCGAATGTCAAATTCGA -1 ACCAAAAAATCAATTTTTAGATTTTTCGAATTAATTTTTCG -1 TGCTTTGCATGTTTCTAAAGTTACAGCCGTTCAAAATTTAA GCATGTTTCTAAAGTTACAGCCGTTCAAAATTTAAAAACTC -1 -1 ACCAATACGCAATGACTGAGTCTGTAATTTCACATAGTAAT � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 4

  8. Memorial Sloan-Kettering Cancer Center Example: Splice Site Recognition Potential Splice Sites CT...GTCGTA...GAAGCTAGGAGCGC...ACGCGT...GA 150 nucleotides window around dimer ≈ GCCAATATTTTTCTATTCAGGTGCAATCAATCACCCATCAT 1 1 ATTGAATGAACATATTCCAGGGTCTCCTTCCACCTCAACAA 1 AGCAACGAACTCCATTACAGCAAGGACATCGAAGTCGATCA 1 GCCAATTTTTGACCTTGCAGAATCAATCGTGCACGTTCGGA -1 CATCTGAAATTTCCCCCAAGTATAGCGGAAATAGACCGACG GAAATTTCCCCCAAGTATAGCGGAAATAGACCGACGAAATC -1 -1 CCCAAGTATAGCGGAAATAGACCGACGAAATCGCTCTCTCC -1 AATCGCTCTCTCCCTGGGAGCGATGCGAATGTCAAATTCGA -1 ACCAAAAAATCAATTTTTAGATTTTTCGAATTAATTTTTCG -1 TGCTTTGCATGTTTCTAAAGTTACAGCCGTTCAAAATTTAA GCATGTTTCTAAAGTTACAGCCGTTCAAAATTTAAAAACTC -1 -1 ACCAATACGCAATGACTGAGTCTGTAATTTCACATAGTAAT . . . � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 4

  9. Memorial Sloan-Kettering Cancer Center Domain Adaptation for Genome Annotation Motivation: Increasing number of sequenced genomes Often newly sequenced genomes are poorly annotated However often relatives with good annotation exist Idea: Transfer knowledge between organisms Example: Splice site annotation in worm genomes ( ≈ 2010) Newly sequenced organism: C. briggsae ≈ 100 confirmed genes (590 splice site pairs) Well annotated relative: C. elegans ≈ 10.000 confirmed genens (36.782 splice site pairs) � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 5

  10. Memorial Sloan-Kettering Cancer Center Domain Adaptation for Genome Annotation Motivation: Increasing number of sequenced genomes Often newly sequenced genomes are poorly annotated However often relatives with good annotation exist Idea: Transfer knowledge between organisms Example: Splice site annotation in worm genomes ( ≈ 2010) Newly sequenced organism: C. briggsae ≈ 100 confirmed genes (590 splice site pairs) Well annotated relative: C. elegans ≈ 10.000 confirmed genens (36.782 splice site pairs) � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 5

  11. Memorial Sloan-Kettering Cancer Center The “Bioinformatics Way” of Transfer Learning 1 Homology-based annotation (a.k.a. “Comparative genomics”) Source Target Works for closely related species, does not require any labeled data from target organism. � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 6

  12. Memorial Sloan-Kettering Cancer Center The “Bioinformatics Way” of Transfer Learning 1 Homology-based annotation (a.k.a. “Comparative genomics”) Source Target ? Works for closely related species, does not require any labeled data from target organism. � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 6

  13. Memorial Sloan-Kettering Cancer Center Domain Adaptation by Learning vs. Homology [Schweikert et al., 2008; Widmer et al., 2010b] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 7

  14. Memorial Sloan-Kettering Cancer Center Domain Adaptation by Learning vs. Homology [Schweikert et al., 2008; Widmer et al., 2010b] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 7

  15. Memorial Sloan-Kettering Cancer Center Domain Adaptation by Learning vs. Homology [Schweikert et al., 2008; Widmer et al., 2010b] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 7

  16. Memorial Sloan-Kettering Cancer Center Domain Adaptation by Learning vs. Homology [Schweikert et al., 2008; Widmer et al., 2010b] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 7

  17. Memorial Sloan-Kettering Cancer Center Domain Adaptation Algorithms Overview [Schweikert et al., 2008] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 8

  18. Memorial Sloan-Kettering Cancer Center Large-Scale Empirical Comparison Varying distances Different data set sizes [MPI Developmental Biology and UCSC Genome Browser] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 9

  19. Memorial Sloan-Kettering Cancer Center Experimental Setup Source dataset size: always 100k examples Target dataset sizes: { 2500, 6500, 16000, 64000, 100000 } Simple kernel (WDK of degree 1 ⇒ under-fitting) Extensive model selection for each method Area under Precision/Recall curve for evaluation � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 10

  20. Memorial Sloan-Kettering Cancer Center Domain Adaptation Results Summary Considerable improvements possible Sophisticated domain adaptation methods needed on distantly related organisms Best overall performance has DualTask Most cost effective Convex/AdvancedConvex [Schweikert et al., 2008] � Gunnar R¨ c atsch ( cBio@MSKCC) Transfer Learning in Computational Biology NIPS MTL Workshop December 13, 2014 11

Recommend

Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius

Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius

Transfer Learning and Applications in Computational Biology 1 Christian Widmer, 1 , 2 Marius Kloft, 1 , 3 Gunnar R atsch, 2 Gabriele Schweikert Nico G ornitz, 1 Memorial Sloan-Kettering Cancer Center, NY, USA 2 Technical University of

1.22k views • 96 slides
Effective transfer learning for clinical applications Benjamin van der Burgh (LIACS) OVERVIEW

Effective transfer learning for clinical applications Benjamin van der Burgh (LIACS) OVERVIEW

Effective transfer learning for clinical applications Benjamin van der Burgh (LIACS) OVERVIEW 1. Transfer learning in NLP 2. Experiments on Dutch data 3. Well-being tracking using clinical journals 2 PROJECT BACKGROUND Physiotherapists

464 views • 25 slides
Industrial Transfer Learning Introduction to Industrial Transfer Learning Industrial Transfer

Industrial Transfer Learning Introduction to Industrial Transfer Learning Industrial Transfer

Industrial Transfer Learning Introduction to Industrial Transfer Learning Industrial Transfer Learning Motivation Machine Learning in Manufacturing Decision Support Automation Process Control Self-Optimization Predictive Quality Predictive

742 views • 24 slides
Introduction to Machine Learning Deep Learning Applications Barnabs Pczos Applications

Introduction to Machine Learning Deep Learning Applications Barnabs Pczos Applications

Introduction to Machine Learning Deep Learning Applications Barnabs Pczos Applications Image Classification (Alexnet, VGG, Resnet) on Cifar 10, Cifar 100, Mnist, Imagenet Art Neural style transfer on images and videos

1.19k views • 83 slides
Transfer Learning Eu Wern Teh What are we covering? Why transfer learning? Fine

Transfer Learning Eu Wern Teh What are we covering? Why transfer learning? Fine

Transfer Learning Eu Wern Teh What are we covering? Why transfer learning? Fine Tuning: how? why? Example Practise: Ants and Bees dataset Caltech 101 dataset Why Transfer Learning? Speedup Training

584 views • 24 slides
Improving Domain-specific Transfer Learning Applications for Image Recognition and Differential

Improving Domain-specific Transfer Learning Applications for Image Recognition and Differential

Improving Domain-specific Transfer Learning Applications for Image Recognition and Differential Equations M.Sc. Thesis in Computer Science and Engineering Candidates: Alessandro Saverio Paticchio, Tommaso Scarlatti Advisor : Prof. Marco

710 views • 42 slides
Transfer learning and domain adaptation Semi-supervised and transfer learning Myth : you cant

Transfer learning and domain adaptation Semi-supervised and transfer learning Myth : you cant

Day 2 Lecture 5 Transfer learning and domain adaptation Semi-supervised and transfer learning Myth : you cant do deep learning unless you have a million labelled examples for your problem. Reality You can learn useful representations

183 views • 15 slides
Heat Transfer Heat Transfer Introduction Practical occurrences, applications, factors

Heat Transfer Heat Transfer Introduction Practical occurrences, applications, factors

Heat Transfer Heat Transfer Introduction Practical occurrences, applications, factors affecting heat transfer Categories and modes of heat transfer Conduction In a slab and across a pipe Convection Free

516 views • 39 slides
Machine Learning HMM applications in computational biology Central dogma CCTGAGCCAACTATTGATGAA

Machine Learning HMM applications in computational biology Central dogma CCTGAGCCAACTATTGATGAA

10-701 Machine Learning HMM applications in computational biology Central dogma CCTGAGCCAACTATTGATGAA DNA transcription mRNA CCUGAGCCAACUAUUGAUGAA translation Protein PEPTIDE 2 Biological data is rapidly accumulating Transcription

703 views • 36 slides
Applications of Machine Learning in Computational Biology Narges Razavian New York University

Applications of Machine Learning in Computational Biology Narges Razavian New York University

Applications of Machine Learning in Computational Biology Narges Razavian New York University Slides thanks to James Galagan@Board Institute Su-In Lee@Univ of Washington Rainer Breitling@ Univ of Glasgow Christopher M. Bishop @ ECCV

804 views • 41 slides
1. Computational Fluid a. Computational Fluid Dynamics is in the domain of Computational Science

1. Computational Fluid a. Computational Fluid Dynamics is in the domain of Computational Science

1. Computational Fluid a. Computational Fluid Dynamics is in the domain of Computational Science b. Applications i. Computational Fluid Dynamics have many applications. Some applications include: 1. Automotive Aerodynamics 2. Designing HVAC

191 views • 3 slides
Meta-transfer Learning for Few-shot Learning Yaoyao Liu Tianjin University and NUS School of

Meta-transfer Learning for Few-shot Learning Yaoyao Liu Tianjin University and NUS School of

NUS-Tsinghua-Southampton Centre for Extreme Search Meta-transfer Learning for Few-shot Learning Yaoyao Liu Tianjin University and NUS School of Computing OUTLINE Research Background Methods Meta-transfer Learning Hard-task

455 views • 33 slides
Transfer Learning Approach for Botnet Detection based on Recurrent Variational Autoencoder

Transfer Learning Approach for Botnet Detection based on Recurrent Variational Autoencoder

Transfer Learning Approach for Botnet Detection based on Recurrent Variational Autoencoder Jeeyung Kim Scientific Data Management Research Group Computational Research Division Lawrence Berkeley National Laboratory 2020 SNTA, 06/02/2020 J.

423 views • 21 slides
Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization

413 views • 28 slides
Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and

Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and

Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and contextualized word embeddings Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Institute of Computational Perception

1.11k views • 51 slides
Parameter-Efficient Transfer Learning for NLP N. Houlsby, A. Giurgiu*, S. Jastrzbski*, B.

Parameter-Efficient Transfer Learning for NLP N. Houlsby, A. Giurgiu*, S. Jastrzbski*, B.

Parameter-Efficient Transfer Learning for NLP N. Houlsby, A. Giurgiu*, S. Jastrzbski*, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, S. Gelly Imagine doing Transfer Learning for NLP Ingredients: A large pretrained model

739 views • 15 slides
Applications of Computational Intelligence to Medicine and Health Manuel Graa Computational

Applications of Computational Intelligence to Medicine and Health Manuel Graa Computational

Applications of Computational Intelligence to Medicine and Health Manuel Graa Computational Intelligence Group UPV/EHU InMed 2013, Piraeus, Greece, 19 July 1 Contents Motivation and main lines Brain Image biomarkers Vessel Image

649 views • 48 slides
Modern Computational Statistics Lecture 20: Applications in Computational Biology Cheng Zhang

Modern Computational Statistics Lecture 20: Applications in Computational Biology Cheng Zhang

Modern Computational Statistics Lecture 20: Applications in Computational Biology Cheng Zhang School of Mathematical Sciences, Peking University December 09, 2019 Introduction 2/23 While modern statistical approaches have been quite

1.18k views • 54 slides
Exploring the Limits of Transfer Learning with a unified Text-to-Text Transformer Presented by -

Exploring the Limits of Transfer Learning with a unified Text-to-Text Transformer Presented by -

Exploring the Limits of Transfer Learning with a unified Text-to-Text Transformer Presented by - Vipul Rathore Elements and images borrowed from Raffel et al., 2019 Transfer Learning: Background Pre-train a model on a data-rich task

1.21k views • 52 slides
Radiative transfer models in the internet- accessible information-computational system

Radiative transfer models in the internet- accessible information-computational system

Radiative transfer models in the internet- accessible information-computational system Atmospheric radiation Firsov . . 1 , Chesnokova T.Yu. 2 , Voronina Yu.V. 2 , Kozodoeva E.V. 2 1-Volgograd State University, 100, Av.

323 views • 17 slides
W ITH the widespread use of hands-free electronic gad- are mapped to a multilingual set using a

W ITH the widespread use of hands-free electronic gad- are mapped to a multilingual set using a

1 Transfer learning for cross-lingual automatic speech recognition Amit Das Abstract In this study, two instance based transfer learning languages. With this, all language dependent transcriptions can phoneme modeling approaches are

408 views • 5 slides
Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

653 views • 44 slides
Big Transfer (BiT): General Visual Representation Learning Abhash Kumar Singh Harit Vishwakarma

Big Transfer (BiT): General Visual Representation Learning Abhash Kumar Singh Harit Vishwakarma

Big Transfer (BiT): General Visual Representation Learning Abhash Kumar Singh Harit Vishwakarma Short summary Transfer learning helps improve sample efficiency and simplifies hyperparameter tuning when training DNNs for vision. The authors

348 views • 11 slides
Learning and transfer (Chapter 3 from How People Learn) (1999) M. Suzanne Donovan, John D.

Learning and transfer (Chapter 3 from How People Learn) (1999) M. Suzanne Donovan, John D.

Learning and transfer (Chapter 3 from How People Learn) (1999) M. Suzanne Donovan, John D. Bransford, and James W. Pellegrino Introduction What is transfer of learning the ability to extend what has been learned in one context to new

661 views • 17 slides

More recommend