scaling newsum
play

Scaling NewSum Big data text Clustering and https://www.scify.org - PowerPoint PPT Presentation

Dec 24, 2022 •223 likes •460 views

Scaling NewSum Big data text Clustering and https://www.scify.org Summarization using N-Gram graphs Alexandros Tzoumas | a.tzoumas@scify.org WWW.FED4FIRE.EU Whats our product about? Scaling NewSum | SciFY.org 3 WWW.FED4FIRE.EU 4

  1. Scaling NewSum Big data text Clustering and https://www.scify.org Summarization using N-Gram graphs Alexandros Tzoumas | a.tzoumas@scify.org WWW.FED4FIRE.EU

  2. What’s our product about? Scaling NewSum | SciFY.org

  3. 3 WWW.FED4FIRE.EU

  4. 4 WWW.FED4FIRE.EU

  5. 5 WWW.FED4FIRE.EU

  6. Business Goals Scaling NewSum | SciFY.org

  7. Goals Business goals From a technical perspective - improve the quality of the Measure and evaluate: - the accuracy of candidate solutions our product offers clustering components, - allow NewSum technology to - the effectiveness (summary expand to new domains/markets quality) of alternative summarization components - the overall scalability of the system 7 WWW.FED4FIRE.EU

  8. Challenges Business challenges From a technical perspective - Expansion to new markets should take - Define a process for evaluating different domain specific characteristics into account clustering and summarization components as system parameters - Scale the algorithms to process thousand of - A product manager is not able to configure sources/articles the product-related settings appropriate for each domain, so a semi-supervised process would be invaluable 8 WWW.FED4FIRE.EU

  9. The Experiments Scaling NewSum | SciFY.org

  10. Setup Tengu testbed with the support of IMEC Cassandra - Hadoop - Spark 10 WWW.FED4FIRE.EU

  11. Experiments Experiment set 1 Results Goal: Measure effectiveness of NewSum’s candidate clustering implementations Related datasets: Selected the algorithm with higher Multiling (articles with clustering information) precision & recall 6GB database of news articles Methodology: Run clustering on 2 different clustering implementations and measure recall and precision. Automatic evaluation for MultiLing dataset Manual process for news articles dataset 11 WWW.FED4FIRE.EU

  12. Experiments Experiment set 2 Results Goal: Measure scalability Increased 5 times the speed of the Related dataset: clustering pipeline! 6GB database of news articles Methodology: Identified areas of improvement Run the clustering pipeline using as input a) the algorithm from experiment set 1 b) a variable number of articles. Measure speed 12 WWW.FED4FIRE.EU

  13. Experiments Experiment set 3 Results Goal: Measure effectiveness of NewSum’s candidate summarization implementations Implemented/Identified the process Related datasets: 6GB database of news articles for selecting the algorithm appropriate for each scenario Methodology: Run the summarization pipeline using as input a) configuration/parameter setting b) a number of clusters to be summarized. Recall and precision were measured through a manual process. Results: Implemented/Identified the process for selecting the algorithm appropriate for each scenario 13 WWW.FED4FIRE.EU

  14. Conclusions Scaling NewSum | SciFY.org

  15. What we achieved - Defined a process for evaluating clustering algorithms - Defined a process for evaluating summarization components - Increased 5 times the speed of the clustering pipeline! - Measured scalability and identified bottlenecks WWW.FED4FIRE.EU 15

  16. How Fed4Fire+ helped us Patron’s support was crucial to the success of the experiments 16 WWW.FED4FIRE.EU

  17. How Fed4Fire+ helped us Provided a quick way to start experimenting with big data without having to worry about the underlying technologies 17 WWW.FED4FIRE.EU

  18. How Fed4Fire+ helped us Funding allowed us allocate time to implement the algorithms and analyze next steps 18 WWW.FED4FIRE.EU

  19. Next Steps Scaling NewSum | SciFY.org

  20. Next steps Continue working on algorithm implementations Distributed N-gram graphs Improve clustering speed using blocking methodology Automate the set up of a pipeline in a cloud environment to be used in production. Release a domain specific product related to Blockchain news. 20 WWW.FED4FIRE.EU

  21. 21 WWW.FED4FIRE.EU

  22. www.scify.org WWW.FED4FIRE.EU This project has received funding from the European Union’s Horizon 2020 research and innovation programme, which is co-funded by the European Commission and the Swiss State Secretariat for Education, Research and Innovation, under grant agreement No 732638.

Recommend

An efficient reduced basis method for the stochastic Darcy flow model Craig Newsum University of

An efficient reduced basis method for the stochastic Darcy flow model Craig Newsum University of

An efficient reduced basis method for the stochastic Darcy flow model Craig Newsum University of Manchester craig.newsum@manchester.ac.uk July, 2017 Craig Newsum QUIET 2017 1 / 3 Goal: Efficient numerical methods for PDEs with uncertain data

713 views • 3 slides
Analysis of Scaling Algorithms for Matrix & Operator Scaling Contents Scaling Algorithms

Analysis of Scaling Algorithms for Matrix & Operator Scaling Contents Scaling Algorithms

Rafael Oliveira University of Toronto Analysis of Scaling Algorithms for Matrix & Operator Scaling Contents Scaling Algorithms Three Step Analysis Generalization One More Application of Scaling Non-Negative Matrices &

604 views • 15 slides
Scaling From simple models to rich strategies PPPLab Day, November 30th Scaling: recent

Scaling From simple models to rich strategies PPPLab Day, November 30th Scaling: recent

Scaling From simple models to rich strategies PPPLab Day, November 30th Scaling: recent publications Insight Series Tool Animation The rationale for scaling As the SDGs require transformational change, scaling can provide: Reaching more

609 views • 37 slides
ETI se Scaling for many organizations Hierarchy Recursive vs T.to eRaokon traverses the

ETI se Scaling for many organizations Hierarchy Recursive vs T.to eRaokon traverses the

Discovery Naming or IN 5 Why name 172.217 11.14 or google.com Indirection Security Isafety concerns How name iEE f google.com Et Challenge Scaling Scaling for many clients Caching D y D ETI se Scaling for many organizations Hierarchy

215 views • 4 slides
Outline Scaling Scalinga Plenitude of Power Laws Scaling-at-large Scaling-at-large

Outline Scaling Scalinga Plenitude of Power Laws Scaling-at-large Scaling-at-large

Scaling Outline Scaling Scalinga Plenitude of Power Laws Scaling-at-large Scaling-at-large Scaling-at-large Principles of Complex Systems Allometry Allometry Definitions Definitions Course 300, Fall, 2008 Examples Examples

568 views • 27 slides
So#ware Scaling Mo/va/on & Goals HW Configura/on & Scale Out So#ware Scaling

So#ware Scaling Mo/va/on & Goals HW Configura/on & Scale Out So#ware Scaling

So#ware Scaling Mo/va/on & Goals HW Configura/on & Scale Out So#ware Scaling Efforts System management Opera/ng system Programming environment PreAcceptance Work HW stabiliza/on & early scaling

390 views • 19 slides
UMBC A B M A L T F O U M B C I M Y O R T 1 (10/18/04) I E S R C E O V

UMBC A B M A L T F O U M B C I M Y O R T 1 (10/18/04) I E S R C E O V

Advanced VLSI Design Scaling CMPE 640 Scaling Technology scaling rate is approximately 13%/year, halving every 5 years. 100 10 m 1 0.1 1960 1970 1980 1990 2000 The size of the circuits also continues to increase. Besides increasing

362 views • 7 slides
Outline Scalinga Plenitude of Power Laws Scaling-at-large Scaling-at-large Principles of

Outline Scalinga Plenitude of Power Laws Scaling-at-large Scaling-at-large Principles of

Scaling Scaling Outline Scalinga Plenitude of Power Laws Scaling-at-large Scaling-at-large Principles of Complex Systems Allometry Allometry CSYS/MATH 300, Fall, 2010 Definitions Definitions Examples Examples History: Metabolism

250 views • 20 slides
Outline Introduction What are the cusp anomaly and generalized scaling function? Superstring in

Outline Introduction What are the cusp anomaly and generalized scaling function? Superstring in

Scaling and integrability from AdS 5 S 5 Riccardo Ricci Imperial College London DAMTP Cambridge 14th October Work in collaboration with S. Giombi, R.Roiban, A. Tseytlin and C.Vergu Scaling and integrability from AdS 5 S 5 Riccardo Ricci

893 views • 50 slides
Communication Avoiding Power Scaling Power Scaling Derivatives of Algorithmic Communication

Communication Avoiding Power Scaling Power Scaling Derivatives of Algorithmic Communication

Communication Avoiding Power Scaling Power Scaling Derivatives of Algorithmic Communication Complexity John D. Leidel, Yong Chen Parallel Programming Models & Systems for High End Computing (P2S2 2015) Sept 1, 2015 1 Overview Intro:

776 views • 26 slides
Using EBS with Auto Scaling Groups How to use the immense power of AWS Auto-Scaling Groups for a

Using EBS with Auto Scaling Groups How to use the immense power of AWS Auto-Scaling Groups for a

Using EBS with Auto Scaling Groups How to use the immense power of AWS Auto-Scaling Groups for a stateful Docker application. Background Background In a service-oriented world where requests can come from anywhere at any time, keeping a

1.04k views • 31 slides
Scaling and Universality in Probability Francesco Caravenna Universit` a degli Studi di

Scaling and Universality in Probability Francesco Caravenna Universit` a degli Studi di

Weak Convergence Brownian Motion A glimpse of SLE Scaling Limits with Disorder Scaling and Universality in Probability Francesco Caravenna Universit` a degli Studi di Milano-Bicocca Luxembourg June 14, 2016 Francesco Caravenna Scaling

1.94k views • 163 slides
Scaling and Universality in Probability Francesco Caravenna Universit` a degli Studi di

Scaling and Universality in Probability Francesco Caravenna Universit` a degli Studi di

Weak Convergence Brownian Motion A glimpse of SLE Scaling Limits with Disorder Scaling and Universality in Probability Francesco Caravenna Universit` a degli Studi di Milano-Bicocca Luxembourg June 14, 2016 Francesco Caravenna Scaling

745 views • 33 slides
Performance Scaling How is my parallel code performing and scaling? Performance metrics

Performance Scaling How is my parallel code performing and scaling? Performance metrics

Performance Scaling How is my parallel code performing and scaling? Performance metrics Measure the execution time T how do we quantify performance improvements? Speed up typically S(N,P) < P Parallel efficiency

377 views • 17 slides
Scaling for Ultrafast, G.FAST, FTTP, 5G and the Cloud Neil J. McRae Chief Architect BT 1

Scaling for Ultrafast, G.FAST, FTTP, 5G and the Cloud Neil J. McRae Chief Architect BT 1

Scaling for Ultrafast, G.FAST, FTTP, 5G and the Cloud Neil J. McRae Chief Architect BT 1 British Telecommunications plc 2018 Agenda BT Core network Traffic growth and trends Scaling the core new router introduction scaling

621 views • 21 slides
Scaling UP PB in County Durham Overview Finding our local PB grass roots! Scaling UP our

Scaling UP PB in County Durham Overview Finding our local PB grass roots! Scaling UP our

Scaling UP PB in County Durham Overview Finding our local PB grass roots! Scaling UP our approach County wide Sharing our experiences and learning From sceptic to supporter! Where to from here? Find out more Our local PB

335 views • 18 slides
Scaling-up SLA Monitoring in Scaling-up SLA Monitoring in Pervasive Environments Pervasive

Scaling-up SLA Monitoring in Scaling-up SLA Monitoring in Pervasive Environments Pervasive

Istituto di Scienza e Tecnologie dell'Informazione A. Faedo Software Engineering Laboratory Scaling-up SLA Monitoring in Scaling-up SLA Monitoring in Pervasive Environments Pervasive Environments Engineering of Software Services for

677 views • 22 slides
Chapter 11: Scaling and Round-off Noise Keshab K. Parhi Outline Introduction Scaling

Chapter 11: Scaling and Round-off Noise Keshab K. Parhi Outline Introduction Scaling

Chapter 11: Scaling and Round-off Noise Keshab K. Parhi Outline Introduction Scaling and Round-off Noise State Variable Description of Digital Filters Scaling and Round-off Noise Computation Round-off Noise Computation

577 views • 46 slides
Shaken but Not Stirred: an Example of Subject Classification using Multidimensional Scaling

Shaken but Not Stirred: an Example of Subject Classification using Multidimensional Scaling

PhUSE US Connect 2018 AB06 Shaken but Not Stirred: an Example of Subject Classification using Multidimensional Scaling /////////// 05Jun2018 / Manuel Sandoval / V1.0 Agenda Multidimensional Scaling Definition Dissimilarity Functions

515 views • 20 slides
Non-Intrusively Avoiding Scaling Problems in and out of MPI Collectives Hongbo Li , Zizhong Chen,

Non-Intrusively Avoiding Scaling Problems in and out of MPI Collectives Hongbo Li , Zizhong Chen,

Non-Intrusively Avoiding Scaling Problems in and out of MPI Collectives Hongbo Li , Zizhong Chen, Rajiv Gupta, and Min Xie May 21st, 2018 Outline Scaling Problem Avoidance Framework Evaluation Conclusion Outline Scaling Problem Avoidance

587 views • 44 slides
Geodesically Convex Optimization & Applications to Operator Scaling and Invariant Theory

Geodesically Convex Optimization & Applications to Operator Scaling and Invariant Theory

Zeyuan Allen-Zhu Ankit Garg Yuanzhi Li Rafael Oliveira Avi Wigderson Geodesically Convex Optimization & Applications to Operator Scaling and Invariant Theory Contents 2nd order methods for Matrix Scaling Geodesic Convexity

472 views • 23 slides
Scaling properties in multijet events towards high multiplicities Peter Schichtel

Scaling properties in multijet events towards high multiplicities Peter Schichtel

Scaling properties in multijet events towards high multiplicities Peter Schichtel Durham University @ Higgs plus Jets 2014 Content Introduction theoretical uncertainties in multi-jet observables jet scaling patterns Jet

451 views • 18 slides
Scaling Pinterest Marty Weiner Level 83 Interwebz Geek Evolution Scaling Pinterest Growth

Scaling Pinterest Marty Weiner Level 83 Interwebz Geek Evolution Scaling Pinterest Growth

Scaling Pinterest Marty Weiner Level 83 Interwebz Geek Evolution Scaling Pinterest Growth March 2010 Page views per day RackSpace 1 small Web Engine 1 small MySQL DB 1 Engineer + 2 Founders Mar 2010 Jan 2011 Jan 2012 May 2012

440 views • 41 slides
Using Best-Worst Using Best-Worst Scaling to measure all Scaling to measure all sorts of things

Using Best-Worst Using Best-Worst Scaling to measure all Scaling to measure all sorts of things

Using Best-Worst Using Best-Worst Scaling to measure all Scaling to measure all sorts of things sorts of things AIMWA Associate Fellows and Fellows Seminar OCTOBER 2010 OCTOBER 2010 1 Rating Scales are commonly used in management

523 views • 29 slides

More recommend