est clustering
play

EST clustering Lorenzo Cerutti Swiss Institute of Bioinformatics - PowerPoint PPT Presentation

Feb 21, 2023 •18 likes •378 views

EST clustering Lorenzo Cerutti Swiss Institute of Bioinformatics EMBNet course, September 2002 EST clustering EMBNet 2002 Expressed sequence tags (ESTs) ESTs represent partial sequences of cDNA clones (average 360 bp). Single-pass reads

  1. EST clustering Lorenzo Cerutti Swiss Institute of Bioinformatics EMBNet course, September 2002

  2. EST clustering EMBNet 2002 Expressed sequence tags (ESTs) ESTs represent partial sequences of cDNA clones (average ∼ 360 bp). Single-pass reads from the 5’ and/or 3’ ends of cDNA clones. mRNA AAAAA Primer / Reverse transcriptase AAAAA 5’ staggered length cDNAs due to polymerase processivity } cDNAs Cloning and sequencing 3’ EST 5’ EST 1

  3. EST clustering EMBNet 2002 Interest for ESTs ESTs represent the most extensive available survey of the transcribed portion of genomes. ESTs are indispensable for gene structure prediction, gene discovery and genomic mapping. Characterization of splice variants and alternative polyadenilation. In silico differential display and gene expression studies (specific tissue expression, normal/disease states). SNP data mining. High-volume and high-throughput data production at low cost. There are 12,323,094 of EST entries in GenBank (dbEST) (August 16, 2002): • 4,550,451 entries of human ESTs; • 2,633,209 entries of mouse ESTs; • ... 2

  4. EST clustering EMBNet 2002 Low data quality of ESTs High error rates ( ∼ 1 / 100 ) because of the sequence reading single-pass. Sequence compression and frame-shift errors due to the sequence reading single-pass. A single EST represents only a partial gene sequence. Not a defined gene/protein product. Not curated in a highly annotated form. High redundancy in the data ⇒ huge number of sequences to analyze. 3

  5. EST clustering EMBNet 2002 Improving ESTs: Clustering, Assembling and Gene indices The value of ESTs is greatly enhanced by clustering and assembling. • solving redundancy can help to correct errors; • longer and better annotated sequences; • easier association to mRNAs and proteins; • detection of splice variants; • fewer sequences to analyze. Gene indices : All expressed sequences (as ESTs) concerning a single gene are grouped in a single index class, and each index class contains the information for only one gene. Different clustering/assembly procedures have been proposed with associated resulting databases (gene indices): • UniGene (http://www.ncbi.nlm.nih.gov/UniGene) • TIGR Gene Indices (http://www.tigr.org/tdb/tgi.shtml) • STACK (http://www.sambi.ac.za/Dbases.html) 4

  6. EST clustering EMBNet 2002 EST clustering pipeline Quality check Pre−processing Repeats/Vector mask Unigene Initial clustering Assembly TIGR Alignment processing STACK Cluster joining Alignments Consensi Expressed forms 5

  7. EST clustering EMBNet 2002 Pre-processing 6

  8. EST clustering EMBNet 2002 Data source The data sources for clustering can be in-house, proprietary, public database or a hybrid of this (chromatograms and/or sequence files). Each EST must have the following information: • A sequence ID (ex. sequence-run ID); • Location in respect of the poly A (3’ or 5’); • The CLONE ID from which the EST has been generated; • Organism; • Tissue and/or conditions; • The sequence. The EST can be stored in FASTA format: >T27784 EST16067 Human Endothelial cells Homo sapiens cDNA 5’ CCCCCGTCTCTTTAAAAATATATATATTTTAAATATACTTAAATATATATTTCTAATATC TTTAAATATATATATATATTTNAAAGACCAATTTATGGGAGANTTGCACACAGATGTGAA ATGAATGTAATCTAATAGANGCCTAATCAGCCCACCATGTTCTCCACTGAAAAATCCTCT TTCTTTGGGGTTTTTCTTTCTTTCTTTTTTGATTTTGCACTGGACGGTGACGTCAGCCAT GTACAGGATCCACAGGGGTGGTGTCAAATGCTATTGAAATTNTGTTGAATTGTATACTTT TTCACTTTTTGATAATTAACCATGTAAAAAATGAACGCTACTACTATAGTAGAATTGAT 7

  9. EST clustering EMBNet 2002 Pre-processing EST pre-processing consists in a number of essential steps to minimize the chance to cluster unrelated sequences. • Screening out low quality regions: ⊲ Low quality sequence readings are error prone. ⊲ Programs as Phred ( Ewig et al., 98 ) read chromatograms and assesses a quality value to each nucleotide. • Screening out contaminations. • Screening out vector sequences (vector clipping). • Screening out repeat sequences (repeats masking). • Screening out low complexity sequences. Dedicated software are available for these tasks: • RepeatMasker ( Smit and Green , http://ftp.genome.washington.edu/RM/RepeatMasker.html); • VecScreen (http://www.ncbi.nlm.nih.gov/VecScreen); • Lucy ( Chou and Holmes , 01); • ... 8

  10. EST clustering EMBNet 2002 Vector-clipping and contaminations Vector-clipping • Vector sequences can skew clustering even if a small vector fragment remains in each read. • Delete 5’ and 3’ regions corresponding to the vector used for cloning. • Detection of vector sequences is not a trivial task, because they normally lies in the low quality region of the sequence. • UniVec is a non-redundant vector database available from NCBI: http://www.ncbi.nlm.nih.gov/VecScreen/UniVec.html Contaminations • Find and delete: ⊲ bacterial DNA, yeast DNA, and other contaminations; ⊲ ... Standard pairwise alignment programs are used for the detection of vector and other contaminants (for example cross-match, BLASTN, FASTA). They are reasonably fast and accurate. 9

  11. EST clustering EMBNet 2002 Repeats masking Some repetitive elements found in the human genome: Length Copy number Fraction of the genome LINEs (long interspersed elements) 6-8 kb 850,000 21% SINEs (short interspersed elements) 100-300 bp 1,500,000 13% LTR (autonomous) 6-11 kb � 450,000 8% LTR (non-autonomous) 1.5-3 kb DNA transposons (autonomous) 2-3 kb � 300,000 3% DNA transposons (non-autonomous) 80-3000 bp SSRs (simple sequence repeats or microsatellite and minisatellites) 3% 10

  12. EST clustering EMBNet 2002 Repeats masking Repeated elements: • They represent a big part of the mammalian genome. • They are found in a number of genomes (plants, ...) • They induce errors in clustering and assembling. • They should be masked, not deleted, to avoid false sequence assembling. • ... but also interesting elements for evolutionary studies. • SSRs important for mapping of diseases. Tools to find repeats: • RepeatMasker has been developed to find repetitive elements and low-complexity sequences. RepeatMasker uses the cross-match program for the pairwise alignments (http://repeatmasker.genome.washington.edu/cgi-bin/RepeatMasker). • MaskerAid improves the speed of RepeatMasker by ∼ 30 folds using WU-BLAST instead of cross-match (http://sapiens.wustl.edu/maskeraid) • RepBase is a database of prototypic sequences representing repetitive DNA from different eukaryotic species.: http://www.girinst.org/Repbase Update.html. 11

  13. EST clustering EMBNet 2002 Low complexity masking Low complexity sequences contains an important bias in their nucleotide compositions (poly A tracts, AT repeats, etc.). Low complexity regions can provide an artifactual basis for cluster membership. Clustering strategies employing alignable similarity in their first pass are very sensitive to low complexity sequences. Some clustering strategies are insensitive to low complexity sequences, because they weight sequences in respect to their information content (ex. d2-cluster). Programs as DUST (NCBI) can be used to mask low complexity regions. 12

  14. EST clustering EMBNet 2002 Pre-processing s d a e r y t i l a u q g h n i g l i l h a c t c e CCCCCGTCTCTTTAAAAATATATATATTTTAAATATACTTAAATATATATTTCTAATATC e s l a TTTAAATATATATATATATTTNAAAGACCAATTTATGGGAGANTTGCACACAGATGTGAA e B S ATGAATGTAATCTAATAGANGCCTAATCAGCCCACCATGTTCTCCACTGAAAAATCCTCT TTCTTTGGGGTTTTTCTTTCTTTCTTTTTTGATTTTGCACTGGACGGTGACGTCAGCCAT GTACAGGATCCACAGGGGTGGTGTCAAATGCTATTGAAATTNTGTTGAATTGTATACTTT TTCACTTTTTGATAATTAACCATGTAAAAAATGAACGCTACTACTATAGTAGAATTGAT Vector clipping CCCCCGTCTCTTTAAAAATATATATATTTTAAATATACTTAAATATATATTTCTAATATC TTTAAATATATATATATATTTNAAAGACCAATTTATGGGAGANTTGCACACAGATGTGAA ATGAATGTAATCTAATAGANGCCTAATCAGCCCACCATGTTCTCCACTGAAAAATCCTCT TTCTTTGGGGTTTTTCTTTCTTTCTTTTTTGATTTTGCACTGGACGGTGACGTCAGCCAT GTACAGGATCCACAGGGGTGGTGTCAAATGCTATTGAAATTNTGTTGAATTGTATACTTT TTCACTTTTTGATAATTAACCATGTAAAAAATGXXXXXXXXXXXXXXXXXXXXXXXXXX Repeat/Low complexity masking CCCCCGTCTCTTTAAAANNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN NNNNNNNNNNNNNNNNNNNTTNAAAGACCAATTTATGGGAGANTTGCACACAGATGTGAA ATGAATGTAATCTAATAGANGCCTAATCAGCCCACCATGTTCTCCACTGAAAAATCCTCT TTCTTTGGGGTTTTTCTTTCTTTCTTTTTTGATTTTGCACTGGACGGTGACGTCAGCCAT GTACAGGATCCACAGGGGTGGTGTCAAATGCTATTGAAATTNTGTTGAATTGTATACTTT TTCACTTTTTGATAATTAACCATGTAAAAAATGXXXXXXXXXXXXXXXXXXXXXXXXXX Sequence ready for clustering CCCCCGTCTCTTTAAAANNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN NNNNNNNNNNNNNNNNNNNTTNAAAGACCAATTTATGGGAGANTTGCACACAGATGTGAA ATGAATGTAATCTAATAGANGCCTAATCAGCCCACCATGTTCTCCACTGAAAAATCCTCT TTCTTTGGGGTTTTTCTTTCTTTCTTTTTTGATTTTGCACTGGACGGTGACGTCAGCCAT GTACAGGATCCACAGGGGTGGTGTCAAATGCTATTGAAATTNTGTTGAATTGTATACTTT TTCACTTTTTGATAATTAACCATGTAAAAAATG 13

  15. EST clustering EMBNet 2002 Clustering 14

Recommend

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Many

768 views • 43 slides
Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find a finite set of representatives that provides good coverage of a complex, possibly infinite, high-dimensional space. Finding meaningful

396 views • 36 slides
Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides
Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering Mixture models Hierarchical clustering Dimensionality reduction Principal component analysis Multidimensional scaling Isomap

1.61k views • 17 slides
Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods 1 2 What is Clustering? What is Clustering? Typical

647 views • 18 slides
Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

LUDWIG- MAXIMILIANS- INSTITUTE DATABASE UNIVERSITT FOR SYSTEMS MNCHEN MNCHEN INFORMATICS INFORMATICS GROUP GROUP Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative Clustering, Multiview

513 views • 9 slides
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides
Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K- means clustering What is clustering?

1.32k views • 66 slides
PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin

PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin

PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin Motivation Example Clustering cannot be analyzed without specifying what it will be used for! be used for! Example Cluster then pack

293 views • 28 slides
Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means clustering Hierarchical clustering Incremental clustering Probability-based clustering Self-Organising Maps Classification vs. Clustering

1k views • 67 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management of web data Outline

607 views • 41 slides
Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean clustering, hierarchical clustering Adaptive learning (online learning) CL, FSCL, RPCL Gaussian Mixture Models (GMM)

805 views • 34 slides
Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document clustering Motivations Document representations Success criteria Clustering algorithms Partitional Hierarchical Ch. 16 What is

766 views • 61 slides
LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides
Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review

Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review

Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Clustering (unsupervised) vs.

526 views • 30 slides
Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review

Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review

Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Clustering (unsupervised) vs.

356 views • 31 slides
Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic

Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic

Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic Links Applications Algorithms SimRank Yin X., Han J. and Yu P. S., 2006, LinkClus: Efficient ReCoM Clustering via Heterogeneous

221 views • 8 slides
Clustering in Go May 2016 Wilfried Schobeiri MediaMath

Clustering in Go May 2016 Wilfried Schobeiri MediaMath

5/26/2016 Clustering in Go Clustering in Go May 2016 Wilfried Schobeiri MediaMath http://127.0.0.1:3999/clustering-in-go.slide#1 1/42 5/26/2016 Clustering in Go Who am I? Go enthusiast These days, mostly codes for fun Focused on

917 views • 42 slides
LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

DATA MINING LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

998 views • 78 slides
Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods Hierarchical Methods 1 2 What is Clustering?

617 views • 24 slides

More recommend