cutting the dendrogram through permutation tests
play

Cutting the dendrogram through permutation tests Dario Bruzzese - PowerPoint PPT Presentation

Feb 07, 2023 •23 likes •693 views

Cutting the dendrogram through permutation tests Dario Bruzzese Domenico Vistocco dbruzzes@unina.it vistocco@unicas.it Department of Department of Preventive Medical Sciences Economics U NIVERSITY OF N APLES ITALY U NIVERSITY OF C ASSINO

  1. Cutting the dendrogram through permutation tests Dario Bruzzese Domenico Vistocco dbruzzes@unina.it vistocco@unicas.it Department of Department of Preventive Medical Sciences Economics U NIVERSITY OF N APLES ITALY U NIVERSITY OF C ASSINO ITALY Dario Bruzzese, Domenico Vistocco () Compstat 2010 1 / 19

  2. La Carte Motivation 1 The stairstep-like permutation procedure 2 Notation The outline Some results 3 Real datasets Synthetic dataset ToDo List 4 Dario Bruzzese, Domenico Vistocco () Compstat 2010 2 / 19

  3. La Carte Motivation 1 The stairstep-like permutation procedure 2 Notation The outline Some results 3 Real datasets Synthetic dataset ToDo List 4 Dario Bruzzese, Domenico Vistocco () Compstat 2010 3 / 19

  4. Motivation Automatically determine the optimal cut-off level of a dendrogram Explore partitions different from those allowed by an horizontal cut Dario Bruzzese, Domenico Vistocco () Compstat 2010 4 / 19

  5. Motivation Automatically determine the optimal cut-off level of a dendrogram Explore partitions different from those allowed by an horizontal cut The rep1HighNoise dataset Yeung KY, Medvedovic M, Bumgarner KY: Clustering gene-expression data with repeated measurements. Genome Biology, 2003, 4:R34 n = 200 p = 20 Dario Bruzzese, Domenico Vistocco () Compstat 2010 4 / 19

  6. Motivation Automatically determine the optimal cut-off level of a dendrogram Explore partitions different from those allowed by an horizontal cut Horizontal cut k = 3 Dario Bruzzese, Domenico Vistocco () Compstat 2010 4 / 19

  7. Motivation Automatically determine the optimal cut-off level of a dendrogram Explore partitions different from those allowed by an horizontal cut An alternative cut k = 3 Dario Bruzzese, Domenico Vistocco () Compstat 2010 4 / 19

  8. La Carte Motivation 1 The stairstep-like permutation procedure 2 Notation The outline Some results 3 Real datasets Synthetic dataset ToDo List 4 Dario Bruzzese, Domenico Vistocco () Compstat 2010 5 / 19

  9. Notation Let: Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  10. Notation Let: n the number of objects to classify; � � �� �� � � Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  11. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  12. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) C 1 C 1 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  13. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) C 2 C 2 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  14. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) C 3 C 3 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  15. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R C k L and C k R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  16. Notation � � C 1 L ∪ C 1 h R Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R C k L and C k R C 1 C 1 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  17. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C 2 L ∪ C 2 h � � C k L ∪ C k R h the height necessary to merge R C k L and C k R C 2 C 2 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  18. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C 3 L ∪ C 3 � � C k L ∪ C k h h the height necessary to merge R R C k L and C k R C 3 C 3 L R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  19. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R C k L and C k R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  20. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k � � h the height necessary to merge C 1 h R L C k L and C k R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 1 L Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  21. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C 1 � � h C k L ∪ C k h the height necessary to merge R R C k L and C k R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 1 R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  22. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R � � C 2 C k L and C k h L R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 2 L Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  23. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R � � C 2 h C k L and C k R R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 2 R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  24. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge R C k L and C k � � C 3 h R L � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 3 L Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  25. Notation Let: n the number of objects to classify; C k L and C k R the two classes merged at level k (k=1,...,n-1) � � C k L ∪ C k h the height necessary to merge � � R C 3 h R C k L and C k R � � C k the height at which C k h j has been obtained j (j ∈ { L, R }) C 3 R Dario Bruzzese, Domenico Vistocco () Compstat 2010 6 / 19

  26. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  27. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset initialization: aggregationLevelsToVisit ← h ( C 1 L ∪ C 1 R ) permClusters ← [ ] i ← 1 Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  28. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset initialization: aggregationLevelsToVisit ← h ( C 1 L ∪ C 1 R ) permClusters ← [ ] i ← 1 repeat if C i L ≡ C i R then add C i L ∪ C i R to permClusters else add h ( C i L ) and h ( C i R ) to aggregationLevelsToVisit sort aggregationLevelsToVisit in descending order end Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  29. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset initialization: aggregationLevelsToVisit ← h ( C 1 L ∪ C 1 R ) permClusters ← [ ] i ← 1 repeat if C i L ≡ C i R then add C i L ∪ C i R to permClusters else add h ( C i L ) and h ( C i R ) to aggregationLevelsToVisit sort aggregationLevelsToVisit in descending order end remove the first element from aggregationLevelsToVisit i ← i+1 Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  30. The algorithm - Pseudo Code Input : A dataset and its related dendrogram Output : A partition of the dataset initialization: aggregationLevelsToVisit ← h ( C 1 L ∪ C 1 R ) permClusters ← [ ] i ← 1 repeat if C i L ≡ C i R then add C i L ∪ C i R to permClusters else add h ( C i L ) and h ( C i R ) to aggregationLevelsToVisit sort aggregationLevelsToVisit in descending order end remove the first element from aggregationLevelsToVisit i ← i+1 until aggregationLevelsToVisit is empty Dario Bruzzese, Domenico Vistocco () Compstat 2010 7 / 19

  31. The algorithm - The outline Initialization i ← 0 aggregationLevelsToVisit h ( C 1 L ∪ C 1 R ) permClusters Dario Bruzzese, Domenico Vistocco () Compstat 2010 8 / 19

  32. The algorithm - The outline � � C 1 L ∪ C 1 h R Iteration i ← 1 aggregationLevelsToVisit h ( C 1 L ∪ C 1 R ) permClusters Dario Bruzzese, Domenico Vistocco () Compstat 2010 8 / 19

  33. The algorithm - The outline Iteration i ← 1 aggregationLevelsToVisit h ( C 1 L ∪ C 1 R ) permClusters C 1 C 1 L R clusters to compare H 0 : C 1 L ≡ C 1 R �→ reject Dario Bruzzese, Domenico Vistocco () Compstat 2010 8 / 19

Recommend

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Domenico Vistocco

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Domenico Vistocco

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Domenico Vistocco dbruzzes@unina.it vistocco@unicas.it Department of

1.46k views • 102 slides
Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Umberto Giani

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Umberto Giani

Stairstep-like dendrogram cut: a permutation test approach Dario Bruzzese Umberto Giani Domenico Vistocco dbruzzes@unina.it ugiani@unina.it vistocco@unicas.it -

1.51k views • 84 slides
Nonparametric hypothesis tests and permutation tests 1.7 & 2.3. Probability Generating

Nonparametric hypothesis tests and permutation tests 1.7 & 2.3. Probability Generating

Nonparametric hypothesis tests and permutation tests 1.7 & 2.3. Probability Generating Functions 3.8.3. Wilcoxon Signed Rank Test 3.8.2. Mann-Whitney Test Prof. Tesler Math 283 Fall 2018 Prof. Tesler Wilcoxon and Mann-Whitney Tests

554 views • 36 slides
Lecture 21: Bootstrap and Permutation Tests The bootstrap Bootstrapping generally refers to

Lecture 21: Bootstrap and Permutation Tests The bootstrap Bootstrapping generally refers to

Mathematical Tools for Neuroscience (NEU 314) Princeton University, Spring 2016 Jonathan Pillow Lecture 21: Bootstrap and Permutation Tests The bootstrap Bootstrapping generally refers to statistical approach to quantifying uncertainty by

345 views • 3 slides
Permutation tests for coefficients of variation in general one-way ANOVA models Markus Pauly 1 and

Permutation tests for coefficients of variation in general one-way ANOVA models Markus Pauly 1 and

Permutation tests for coefficients of variation in general one-way ANOVA models Markus Pauly 1 and ukasz Smaga 2 1 Faculty of Statistics TU Dortmund University Dortmund, Germany 2 Faculty of Mathematics and Computer Science Adam Mickiewicz

360 views • 34 slides
Permutation tests Fabian Pedregosa October 3, 2017 Data Science Learn2Launch, UC Berkeley

Permutation tests Fabian Pedregosa October 3, 2017 Data Science Learn2Launch, UC Berkeley

Permutation tests Fabian Pedregosa October 3, 2017 Data Science Learn2Launch, UC Berkeley Announcements Next week is the first presentation! 1. 10 min presentation (by teams) + 5 min questions 2. At least: objective of the project,

531 views • 33 slides
On Synchronized Permutation tests in Two-way ANOVA Dario Basso 1 , Luigi Salmaso 1 , Fortunato

On Synchronized Permutation tests in Two-way ANOVA Dario Basso 1 , Luigi Salmaso 1 , Fortunato

On Synchronized Permutation tests in Two-way ANOVA Dario Basso 1 , Luigi Salmaso 1 , Fortunato Pesarin 2 1 : Department of Management and Engineering, University of Padova 2 : Department of Statistics, University of Padova dario@stat.unipd.it

717 views • 29 slides
Constraint-based Bayesian Network Learning with Permutation Tests Marco Scutari

Constraint-based Bayesian Network Learning with Permutation Tests Marco Scutari

Constraint-based Bayesian Network Learning with Permutation Tests Marco Scutari marco.scutari@stat.unipd.it Adriana Brogini brogini@stat.unipd.it Department of Statistical Sciences University of Padova June 15, 2010 Marco Scutari and

404 views • 15 slides
Lecture 4: Permutation Methods Applied Statistics 2014 1 / 21 Randomization Model Population

Lecture 4: Permutation Methods Applied Statistics 2014 1 / 21 Randomization Model Population

Randomization Model Population Model Rank Tests Assignment Lecture 4: Permutation Methods Applied Statistics 2014 1 / 21 Randomization Model Population Model Rank Tests Assignment Permutation Methods Non-parametric methods for testing

351 views • 23 slides
Dendrogram-based Algorithm for Dominated Graph Flooding Claude

Dendrogram-based Algorithm for Dominated Graph Flooding Claude

Dendrogram-based Algorithm for Dominated Graph Flooding Claude Tadonki Joint work with Fernand Meyer and Franois Irigoin C entre de R echerche en I nformatique - C entre de M orphologie M athmatique Mines

293 views • 18 slides
J , the all- 1 matrix. (The corresponding only G -invariant association scheme is the trivial A

J , the all- 1 matrix. (The corresponding only G -invariant association scheme is the trivial A

Association schemes and Permutation groups permutation groups Let G be a permutation group on . Then the characteristic functions of the orbits of G on Peter J Cameron G

44 views • 3 slides
The diameter of permutation groups permutation groups H. A. Helfgott February 2017 The

The diameter of permutation groups permutation groups H. A. Helfgott February 2017 The

The diameter of permutation groups H. A. Helfgott Introduction Diameter bounds New work on The diameter of permutation groups permutation groups H. A. Helfgott February 2017 The diameter of Cayley graphs permutation groups H. A.

751 views • 64 slides
Table Saw BEST PRACTICES 2018 Table Saw Purpose NCUs Table Saw Purposes Cutting

Table Saw BEST PRACTICES 2018 Table Saw Purpose NCUs Table Saw Purposes Cutting

Table Saw BEST PRACTICES 2018 Table Saw Purpose NCUs Table Saw Purposes Cutting Plywood Cutting aluminum sheeting Cutting Lumber Cutting high density foam Best Practice - Tooling Inspection Set-up Operation Storage

645 views • 15 slides
Metal Cutting (Machining) Metal cutting , commonly called machining , is the removal of unwanted

Metal Cutting (Machining) Metal cutting , commonly called machining , is the removal of unwanted

Metal Cutting (Machining) Metal cutting , commonly called machining , is the removal of unwanted portions from a block of material in the form of chips so as to obtain a finished product of desired size, shape, and finish. 1 Metal Cutting

633 views • 52 slides
Statistics on permutation tableaux Pawel Hitczenko Drexel University parts based on joint work

Statistics on permutation tableaux Pawel Hitczenko Drexel University parts based on joint work

Statistics on permutation tableaux Pawel Hitczenko Drexel University parts based on joint work with Sylvie Corteel (Paris-Sud) and parts with Svante Janson (Uppsala) LIPN, February 5, 2008 Permutation tableaux Permutation tableau T : a Ferrers

1.11k views • 50 slides
Growth in permutation groups and linear New work on algebraic groups permutation groups H. A.

Growth in permutation groups and linear New work on algebraic groups permutation groups H. A.

Growth in permutation groups and linear algebraic groups H. A. Helfgott Introduction Diameter bounds Growth in permutation groups and linear New work on algebraic groups permutation groups H. A. Helfgott September 2018 Growth in

697 views • 65 slides
ERP- PPS Quality Quality Additional equipment Disc cutting machine with 0,2 mm accuracy Vibro

ERP- PPS Quality Quality Additional equipment Disc cutting machine with 0,2 mm accuracy Vibro

ERP- PPS Quality Quality Additional equipment Disc cutting machine with 0,2 mm accuracy Vibro cleaning device Gas cutting equipment Zinser 1211 3 automatic band cutting machines Conventional piece- production Turning: diameter up to 450 mm.

312 views • 14 slides
Writing reliable end to end tests End to end browser tests They take a long time to run. Around

Writing reliable end to end tests End to end browser tests They take a long time to run. Around

Writing reliable end to end tests End to end browser tests They take a long time to run. Around 4-12 hours Long feedback cycles Tough to read or modify Flaky Not part of the development life cycle Unit tests are End to end important but they End

530 views • 38 slides
What is on the tests? What comes to mind when you think of this test? What is on the

What is on the tests? What comes to mind when you think of this test? What is on the

What is on the tests? What comes to mind when you think of this test? What is on the tests? Content What is on the tests? Content ELA Grammar, Writing skills, Vocabulary, Reading Comp What is on the tests? Content

565 views • 38 slides
Christchurch 20 th November 2012 Cutting Edge 2012 School of Addiction 2013 Cutting Edge 2013

Christchurch 20 th November 2012 Cutting Edge 2012 School of Addiction 2013 Cutting Edge 2013

Leadership Day Christchurch 20 th November 2012 Cutting Edge 2012 School of Addiction 2013 Cutting Edge 2013 Practitioner Registration Cutting Edge 2012 Relevant: 97.5% Enjoyable: 99% Recommend: 89% Cost reasonable 80%

255 views • 21 slides
MA/CSSE 473 Day 13 Permutation Generation MA/CSSE 473 Day 13 HW 6 due Monday , HW 7 next

MA/CSSE 473 Day 13 Permutation Generation MA/CSSE 473 Day 13 HW 6 due Monday , HW 7 next

MA/CSSE 473 Day 13 Permutation Generation MA/CSSE 473 Day 13 HW 6 due Monday , HW 7 next Thursday, Student Questions Tuesdays exam Permutation generation 1 Exam 1 If you want additional practice problems for Friday's exam:

260 views • 11 slides
Oligomorphic permutation groups: growth rates and algebras Peter J. Cameron

Oligomorphic permutation groups: growth rates and algebras Peter J. Cameron

Oligomorphic permutation groups: growth rates and algebras Peter J. Cameron p.j.cameron@qmul.ac.uk Gregynog Mathematics Colloquium22 May 2007 The definition Examples, 2 Consider the group S r acting on the disjoint union Let G be a permutation

304 views • 4 slides
Permutation-based Combinatorial Optimization Problems under the Microscope Josu Ceberio

Permutation-based Combinatorial Optimization Problems under the Microscope Josu Ceberio

Permutation-based Combinatorial Optimization Problems under the Microscope Josu Ceberio Intelligent Systems Group Department of Computer Science and Artificial Intelligence University of the Basque Country (UPV/EHU) 1 Permutation-based

1.3k views • 89 slides
Enumeration of Permutation Classes by Inflation of Independent Sets of Graphs mile Nadeau

Enumeration of Permutation Classes by Inflation of Independent Sets of Graphs mile Nadeau

Enumeration of Permutation Classes by Inflation of Independent Sets of Graphs mile Nadeau (based on joint work with Christian Bean and Henning Ulfarsson) Reykjavik University Permutations Patterns 2019 Staircase encoding For any permutation

969 views • 78 slides

More recommend