from transfac to hocomoco using cross validation and
play

From Transfac to HOCOMOCO: using cross-validation and human curation - PowerPoint PPT Presentation

Sep 30, 2022 •337 likes •721 views

From Transfac to HOCOMOCO: using cross-validation and human curation to take most from the high throughput data compiling a complete collection of transcription factor binding motifs Vsevolod J. Makeev Vavilov Institute of General Genetics,

  1. From Transfac to HOCOMOCO: using cross-validation and human curation to take most from the high throughput data compiling a complete collection of transcription factor binding motifs Vsevolod J. Makeev Vavilov Institute of General Genetics, Russian Academy of Sciences, Moscow March 8, 2018

  2. D.melanogaster enhancers We started to work with regulatory genomics in 1998 Dima Papatsenko studied Drosophila enhancers he was interested in TF binding sites

  3. Our first collection of TFBS A site verified by at least two Table 1. Comparison between the Refined and Consistent Maps methods from footprints, mutant, or highly conserved blocks Bicoid (34 sites), Caudal (15), Ftz (25), Hunchback (43), Knirps (47), Kruppel (21), and Tramtrak (7) Distribution of sites shown for the even-skipped strip 2 region. Most of the experimentally verified binding sites shown are shared between the two maps (hits, shown in red). Two known Bicoid sites Aligned with CLUSTALW and false-negatives in blue) are missing in the consistent map due to their low positional weight matrix score. In vitro binding assays support the suggestion of low affinity for these two Bicoid sites (Wilson et al. 1996). High-scoring matches (false-positives) to Bicoid, Kru ¨ppel, and Giant are shown in green. manually and cut the flanks

  4. Aligning footprints with genome mapping =1 п.н. 2008 Mapping footprints on the genome allows recovering up to 40 Usually it is enough to add only two letters Genome data may be very useful for interpretation in vitro results Ivan Kulakovskiy http://autosome.ru/dmmpmm/ DMMPMM collection

  5. TRANSFAC appears!

  6. Nice Sp1 model for studying CpG islands Sp1 JASPAR 2007 Sp1 Remapped and realigned (SELEX data) TRANSFAC 2008

  7. Chip-on-chip data Chip-on-chip yielded long regions (up to 20K) Wasn’t suitable for motif discovery But perhaps could be helped with in vitro data

  8. Integrative motif discovery: early ChIPmunk Subsampling on many sets of sequences then optimization on total set of weighted sequencies

  9. Chip-seq data

  10. ChIPmunk page Peak shape and motif shape prior (like double box) available at http://autosome.ru/ChIPMunk/

  11. TRANSFAC comes into view again ... and supplies us with a new version of SITE database (for free)

  12. Core workflow (2011), with Vlad Bajic from KAUST

  13. Discovery strategies usually agree!

  14. Human curation

  15. Some notes on PWMs PWM can be used to calculate a score for any sequence j + L − 1 Score [ j ] = � PWM [ j , s ( j )] j s ( j ) is the letter in the position j of the alignment of PWM with the sequence L is the PWM length

  16. PWM and the scoring threshold as a binary classifier Each pair ( PWM , threshold ) classifies any word as a motif hit (YES/NO)

  17. Fast exact calculation of motif P-vlaue Suppose there is a probability distribution upon the l -words Motif P -value is the sum of probabilities of all words scoring above the threshold In 2007 H´ el` en Touzet and Jean St´ efan Varr´ e designed nice precise algorithm

  18. Motfs can be compared as clussifiers i.e. pairs ( PWM, threshold ) One needs to set both thresholds ... but after that it is possible to calculate the percentage of common words recognized by both motfs and compare it with a larger set of words recognized by any of them Matrices of different origine (or even PWM and PCM) can be compared without additional normalization

  19. MacroApe to compare motifs We modified Touzet - Varr´ e algorithm to compare PMWs Available at http://opera.autosome.ru/macroape Can be used to extract motifs from various motif databases

  20. Measuring performance with AU ROC We can use theoretically calculated P-values for a false-positive rate This allows us to compare performance of different motifs on the same benchmark datasets

  21. Hocomoco database log 2011 first website published 2012, first publication, v.9, Nucleic acids research, database 2013 2015, second publication, v.10, Nucleic acids research, database, 2016 2017, third publication, v.11, Nucleic acids research, database 2018 http://hocomoco11.autosome.ru/ http://www.cbrc.kaust.edu.sa/hocomoco11

  22. Extension from HT-SELEX data (v.10) large number of HT-SELEX data and new ChIP-seq data allowed us to extend the core base only by benchmarking and curation

  23. Curation of extantion v.10 similar to known models (0.05 Jaccard similarity) consistent within a TF family, TFclass families are taken or at least with a clearly exhibited consensus (based on LOGO representation, manually assessed).

  24. Extension from GTRD ChIP-seq database Gather as many datasets as possible Motif discovery in all datasets Benchmarking and conservative filtering

  25. Machine dataset filtering v.11 Cross-validation based dataset filtering If known motif performs better than the genuine dataset motif the entire dataset is discarded

  26. Dinucleotide models

  27. Many motifs are very similar Figure: ETC family Difficulties for MARA style analysis. SwissRegulon contains small number of ”isolated” motifs

  28. Motif classes correspond to structural classes of TFs Adapted from TFclass database, Wingender et al., 2015

  29. http://www.cbrc.kaust.edu.sa/hocomoco11 http://hocomoco.autosome.ru

  30. v.11 Hocomoco statistics models for 453 mouse and 680 human transcription factors contains 1302 mononucleotide and 576 dinucleotide PWMs build from more than 3000 ChIP-seq tracks and four peak callers

  31. What one needs motifs for ? Mike Visser et al. Genome Research, 2012; 22:446-455

  32. No experimental location of TFBS method in vitro native or segment # segments comment in vivo synthetic length ChIP in vivo native 40 (exo) 150 - indirect 5000 50000 binding One-hybrid synthetic ∼ 30 20-50 in bacteria in vivo SELEX, RSS synthetic ∼ 20 20-50 saturation in vitro HT-SELEX synthetic ∼ 50 5000 saturation in vitro PBA synthetic ∼ 50 10000 overlapping in vitro Footprints either native ∼ 100 20 - 10000 indirect Table: Experimental methods of TF binding identification

  33. Limitations for using motifs to explain eQTLs Because many other processes (mostly chromatin related) contribute to the protein positioning at the genome From Levo and Segal, 2014, Nat Rev Genet

  34. who cite HOCOMOCO (References on 2016 paper, 63 total for Jan. 2018) Functional genomics (genome structure, annotation, etc) 15 Genetics: annotation of loci and rSNP 13 Systems biology (regulatory networks from DE data) 10 Algorithms and Machine learning assisted genome annotation 7 ”Stories” about particular promoters etc 7 DNA - protein interaction studies 6 TF studies - databases, structure of DNA recognition motifs etc 4 Genetic engineering - prediction of genemics manipulation 2 General Molecular biology (transctiption initiation etc) 1

  35. Autosome.RU software family + Hocomoco database

  36. Who contributed this? CB RAS: VIGG RAS: Julya Medvedeva Artem Kasianov ISB Ltd: Ivan Kulakovskiy Ruslan Shapirov Ilya Vorontsov Ivan Yevshin Seva Makeev Fedor Kolpakov KAUST: Skolkovo Tech: Haitham Ashoor Dima Papatsenko Wail Ba-alawi students Arturo Magana-Mora Alla Fedorova, MSU FBB Ulf Schaefer Eugen Rumynskiy, MIPT Vlad Bajic Nastya Soboleva, MIPT

  37. Thank you! Russian Fund of Basics Research Russian Scientific Fund Ministry of Science and Education of Russian Federation Biobase and personally Edgar Wingender and Alexander Kel RIKEN Fantom Project Ecole Polytechnique and personally Mireille Regnier

Recommend

Occams Razor Sampling Bias : generate the data carefuly Data Snooping : handle the data

Occams Razor Sampling Bias : generate the data carefuly Data Snooping : handle the data

recap: Validation and Cross Validation Validation Cross Validation D ( N ) D 1 D 2 D N D train Learning From Data D ( N K ) g 1 g 2 g N Lecture 14 ( x 1 , y 1 ) ( x 2 , y 2 ) ( x N , y N ) Three Learning Principles D

486 views • 15 slides
Introduction to Data Science: Classifier n 1 n 1 k k Suppose you want to compare two

Introduction to Data Science: Classifier n 1 n 1 k k Suppose you want to compare two

Classier evaluation Classier evaluation Leave-one-out Cross-Validation Leave-one-out Cross-Validation Leave-one-out Cross-Validation Classier evaluation Classier evaluation Leave-one-out Cross-Validation Resampled validation set

984 views • 51 slides
in Spark Using GPU Minsik Cho, Rajesh Bordawekar IBM TJW Research 1 Cross-Validation 101

in Spark Using GPU Minsik Cho, Rajesh Bordawekar IBM TJW Research 1 Cross-Validation 101

Accelerating Cross-Validation in Spark Using GPU Minsik Cho, Rajesh Bordawekar IBM TJW Research 1 Cross-Validation 101 [Wikipedia] Popular Model Validation Technique to avoid overfitting, for better generalization useful when not

430 views • 20 slides
Importance-Weighted Cross- Importance-Weighted Cross- Validation for Covariate Shift Validation

Importance-Weighted Cross- Importance-Weighted Cross- Validation for Covariate Shift Validation

Importance-Weighted Cross- Importance-Weighted Cross- Validation for Covariate Shift Validation for Covariate Shift (1) (2) Masashi Sugiyama , Benjamin Blankertz , (2,3) (2) Matthias Krauledat , Guido Dornhege , (3,2) Klaus-Robert

623 views • 26 slides
Stratified Cross-Validation in Multi-Label Classification Using Genetic Algorithms 7-8/02/2013

Stratified Cross-Validation in Multi-Label Classification Using Genetic Algorithms 7-8/02/2013

Stratified Cross-Validation in Multi-Label Classification Using Genetic Algorithms 7-8/02/2013 TIN2010-20900-C04 Albacete Index Introduction Multilabel Classification Cross-Validation and Stratified Cross-Validation Methods and

682 views • 43 slides
Progress to Date in A3: Method Transfer, Partial Validation and Cross validation A3: Method

Progress to Date in A3: Method Transfer, Partial Validation and Cross validation A3: Method

Progress to Date in A3: Method Transfer, Partial Validation and Cross validation A3: Method Transfer, partial and cross validation Team members: In scope Life cycle of a method after first full validation or relation Team

252 views • 10 slides
Cross-validation and the Bootstrap In the section we discuss two resampling methods:

Cross-validation and the Bootstrap In the section we discuss two resampling methods:

Cross-validation and the Bootstrap In the section we discuss two resampling methods: cross-validation and the bootstrap. These methods refit a model of interest to samples formed from the training set, in order to obtain additional

688 views • 44 slides
STAT 213 Cross-Validation (and Multifactor ANOVA?) Colin Reimer Dawson Oberlin College 12

STAT 213 Cross-Validation (and Multifactor ANOVA?) Colin Reimer Dawson Oberlin College 12

Outline Last Time Cross-Validation STAT 213 Cross-Validation (and Multifactor ANOVA?) Colin Reimer Dawson Oberlin College 12 April 2016 Outline Last Time Cross-Validation Outline Last Time Cross-Validation Outline Last Time

381 views • 25 slides
Criticality experiments and benchmarks for for validation of cross validation of cross sections:

Criticality experiments and benchmarks for for validation of cross validation of cross sections:

Criticality experiments and benchmarks Criticality experiments and benchmarks for for validation of cross validation of cross sections: the neptunium sections: the neptunium case case L.S.Leong, L.Tassan-Got, L.Audouin, C. Paradela,

464 views • 21 slides
CSC411 Tutorial #3 Cross-Validation and Decision Trees February 3, 2016 Boris Ivanovic*

CSC411 Tutorial #3 Cross-Validation and Decision Trees February 3, 2016 Boris Ivanovic*

CSC411 Tutorial #3 Cross-Validation and Decision Trees February 3, 2016 Boris Ivanovic* csc411ta@cs.toronto.edu *Based on the tutorial given by Erin Grant, Ziyu Zhang, and Ali Punjani in previous years. Outline for Today Cross-Validation

355 views • 24 slides
Data Mining II Model Validation Heiko Paulheim Why Model Validation? We have seen so far

Data Mining II Model Validation Heiko Paulheim Why Model Validation? We have seen so far

Data Mining II Model Validation Heiko Paulheim Why Model Validation? We have seen so far Various metrics (e.g., accuracy, F-measure, RMSE, ) Evaluation protocol setups Split Validation Cross Validation Special

1.04k views • 55 slides
15-388/688 - Practical Data Science: Nonlinear modeling, cross-validation, and regularization J.

15-388/688 - Practical Data Science: Nonlinear modeling, cross-validation, and regularization J.

15-388/688 - Practical Data Science: Nonlinear modeling, cross-validation, and regularization J. Zico Kolter Carnegie Mellon University Fall 2019 1 Outline Example: return to peak demand prediction Overfitting, generalization, and cross

903 views • 67 slides
Learning From Data Lecture 14 Three Learning Principles Occams Razor Sampling Bias Data

Learning From Data Lecture 14 Three Learning Principles Occams Razor Sampling Bias Data

Learning From Data Lecture 14 Three Learning Principles Occams Razor Sampling Bias Data Snooping M. Magdon-Ismail CSCI 4100/6100 recap: Validation and Cross Validation Validation Cross Validation D ( N ) D 1 D 2 D N D train D

1.01k views • 58 slides
Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection

Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection

Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection Cross Validation M. Magdon-Ismail CSCI 4100/6100 recap: Regularization Regularization combats the effects of noise by putting a leash on the

349 views • 31 slides
simulations Workshop on Bioinformatics of Gene Regulation on the occasion of 30 Years TRANSFAC

simulations Workshop on Bioinformatics of Gene Regulation on the occasion of 30 Years TRANSFAC

Biophysical modeling of transcription factor binding sites using large SELEX libraries and computational simulations Workshop on Bioinformatics of Gene Regulation on the occasion of 30 Years TRANSFAC Gttingen 7.-9. March 2018 Philipp Bucher

412 views • 28 slides
Approximate Cross-Validation and Dynamic Experiments for Policy Choice Maximilian Kasy

Approximate Cross-Validation and Dynamic Experiments for Policy Choice Maximilian Kasy

Approximate Cross-Validation and Dynamic Experiments for Policy Choice Approximate Cross-Validation and Dynamic Experiments for Policy Choice Maximilian Kasy Department of Economics, Harvard University April 23, 2018 1 / 23 Approximate

588 views • 23 slides
Holdout and Cross- -Validation Validation Holdout and Cross Methods Overfitting Avoidance

Holdout and Cross- -Validation Validation Holdout and Cross Methods Overfitting Avoidance

Holdout and Cross- -Validation Validation Holdout and Cross Methods Overfitting Avoidance Methods Overfitting Avoidance Decision Trees Decision Trees Reduce error pruning Reduce error pruning Cost Cost- -complexity pruning

699 views • 17 slides
Cross-validation and the Bootstrap In the section we discuss two resampling methods:

Cross-validation and the Bootstrap In the section we discuss two resampling methods:

Cross-validation and the Bootstrap In the section we discuss two resampling methods: cross-validation and the bootstrap. 1 / 44 Cross-validation and the Bootstrap In the section we discuss two resampling methods: cross-validation and the

1.13k views • 66 slides
Bayesian Cross Validation in Regular Asymptotic Theory Information Geometry and Its Applications

Bayesian Cross Validation in Regular Asymptotic Theory Information Geometry and Its Applications

Higher Order Analysis of Bayesian Cross Validation in Regular Asymptotic Theory Information Geometry and Its Applications IV In honor of Prof. Amaris 80 th birthday June 12-17, 2016, Liblice, Czech Republic Sumio Watanabe Tokyo Institute of

486 views • 30 slides
Gene Regulation 30 Years TRANSFAC Gttingen, March 07-09, 2018 Georg August University,

Gene Regulation 30 Years TRANSFAC Gttingen, March 07-09, 2018 Georg August University,

Bioinformatics of Gene Regulation 30 Years TRANSFAC Gttingen, March 07-09, 2018 Georg August University, Gttingen, Germany Wingender E. Compilation of transcription regulating proteins. Nucleic Acids Res. 1988 Mar 25;16(5):1879-902. PMID:

336 views • 11 slides
Cross-Validation Machine Learning 1 Model selection Very broadly: Choosing the best model using

Cross-Validation Machine Learning 1 Model selection Very broadly: Choosing the best model using

Cross-Validation Machine Learning 1 Model selection Very broadly: Choosing the best model using given data What makes a model Features Hyper-parameters that control the hypothesis space Example: depth of a decision tree, neural

328 views • 12 slides
Significance testing after cross-validation Joshua Loftus ( jloftus@turing.ac.uk ) (building from

Significance testing after cross-validation Joshua Loftus ( jloftus@turing.ac.uk ) (building from

Significance testing after cross-validation Joshua Loftus ( jloftus@turing.ac.uk ) (building from joint work with Jonathan Taylor) 9 December, 2016 Slides and markdown source at https://joftius.github.io/turing 1 / 20 Setting: regression model

590 views • 23 slides
Evaluate Deep Q-Learning for Sequential Targeted Marketing with 10-fold Cross Validation

Evaluate Deep Q-Learning for Sequential Targeted Marketing with 10-fold Cross Validation

Agent Actions Evaluate Deep Q-Learning for Sequential Targeted Marketing with 10-fold Cross Validation Rewards Observations Jian Wu Full Stack Engineer Marketing Environment The idea of this talk was developed at Samsung SDSRA AI Lab

208 views • 19 slides
5: Overtraining and Cross-validation Machine Learning and Real-world Data Helen Yannakoudakis 1

5: Overtraining and Cross-validation Machine Learning and Real-world Data Helen Yannakoudakis 1

5: Overtraining and Cross-validation Machine Learning and Real-world Data Helen Yannakoudakis 1 Computer Laboratory University of Cambridge Lent 2018 1 Based on slides created by Simone Teufel Last session: smoothing and significance testing

1.56k views • 15 slides

More recommend