multilingual and noisy data challenges in large scale
play

Multilingual and Noisy Data Challenges in Large-Scale Book Scanning - PowerPoint PPT Presentation

May 04, 2023 •448 likes •1.52k views

Noisy text in Google Books OCR in Google Research Challenges Multilingual and Noisy Data Challenges in Large-Scale Book Scanning Ashok C. Popat Staff Research Scientist, Google, Inc. September 17, 2011 Ashok C. Popat September 17, 2011 1 /

  1. Noisy text in Google Books OCR in Google Research Challenges Methods compared against • OCR-confidence output, using several versions of a commercial engine • HEUR-T, HEUR-K: Heuristics by Taghva ‘01; Kulp ‘07 • Dictionaries: Extract vocabulary files from Web data. Use the most frequent N terms, where N ranges from 1K to 1M • Hard Dictionary (HDL, HDM): Penalize passage by C 1 for each OOV term; penalize by C 2 for each punctuation-or-symbol tokenized as a singleton. • Soft Dictionary (SDL, SDM): For each term in a passage, find edit distance to dictionary word (or C 2 for punctuation-or-symbol tokenized as a singleton.) Penalty for the passage is the total edit distance divided by the passage length in Unicode points Ashok C. Popat September 17, 2011 22 / 73

  2. Noisy text in Google Books OCR in Google Research Challenges Comparison among methods Table: All considered languages (approx 30) s 2 condition N 95% CI ¯ τ intra-rater 0.790 522 0.050 (0.770, 0.809) inter-rater 0.668 3056 0.087 (0.658, 0.679) OCR-conf 0.263 2610 0.216 (0.245, 0.280) HEUR-T 0.339 2610 0.146 (0.325, 0.354) HEUR-K 0.381 2610 0.149 (0.367, 0.396) SEQ 0.600 2610 0.090 (0.589, 0.612) SPA 0.665 2610 0.086 (0.654, 0.676) Ashok C. Popat September 17, 2011 23 / 73

  3. Noisy text in Google Books OCR in Google Research Challenges Comparison among methods (continued) Table: Eleven intersection languages s 2 condition N 95% CI ¯ τ intra-rater 0.803 291 0.052 (0.777, 0.829) inter-rater 0.665 1895 0.093 (0.651, 0.679) OCR-conf 0.251 1455 0.239 (0.226, 0.276) HEUR-T 0.375 1455 0.135 (0.356, 0.394) HEUR-K 0.428 1455 0.141 (0.408, 0.447) HDM1M 0.516 1455 0.111 (0.499, 0.533) SDM50K 0.586 1455 0.106 (0.570, 0.603) SEQ 0.607 1455 0.094 (0.592, 0.623) SPA 0.670 1455 0.087 (0.655, 0.686) Ashok C. Popat September 17, 2011 24 / 73

  4. Noisy text in Google Books OCR in Google Research Challenges Application to e-book readers • For a given paragraph in an e-book, is it better to render the text or swap in the image? Ashok C. Popat September 17, 2011 25 / 73

  5. Noisy text in Google Books OCR in Google Research Challenges Application to e-book readers • For a given paragraph in an e-book, is it better to render the text or swap in the image? Ashok C. Popat September 17, 2011 25 / 73

  6. Noisy text in Google Books OCR in Google Research Challenges Application to mobile device OCR • Can we select only the Good OCR text from a given image region? • Viterbi search: • Two states: garbage and clean • Scores computed as described, plus transition costs • Transitions discounted based on image distance between symbols • About 30 languages enabled; language not set in advance Ashok C. Popat September 17, 2011 26 / 73

  7. Noisy text in Google Books OCR in Google Research Challenges Example 1 Ashok C. Popat September 17, 2011 27 / 73

  8. Noisy text in Google Books OCR in Google Research Challenges OCR Engine A Ashok C. Popat September 17, 2011 28 / 73

  9. Noisy text in Google Books OCR in Google Research Challenges OCR Engine B Ashok C. Popat September 17, 2011 29 / 73

  10. Noisy text in Google Books OCR in Google Research Challenges Example 2 Ashok C. Popat September 17, 2011 30 / 73

  11. Noisy text in Google Books OCR in Google Research Challenges OCR Engine A Ashok C. Popat September 17, 2011 31 / 73

  12. Noisy text in Google Books OCR in Google Research Challenges OCR Engine B Ashok C. Popat September 17, 2011 32 / 73

  13. Noisy text in Google Books OCR in Google Research Challenges Example 3 Ashok C. Popat September 17, 2011 33 / 73

  14. Noisy text in Google Books OCR in Google Research Challenges OCR Engine A Ashok C. Popat September 17, 2011 34 / 73

  15. Noisy text in Google Books OCR in Google Research Challenges OCR Engine B Ashok C. Popat September 17, 2011 35 / 73

  16. Noisy text in Google Books OCR in Google Research Challenges Example 4 Ashok C. Popat September 17, 2011 36 / 73

  17. Noisy text in Google Books OCR in Google Research Challenges OCR Engine A Ashok C. Popat September 17, 2011 37 / 73

  18. Noisy text in Google Books OCR in Google Research Challenges OCR Engine B Ashok C. Popat September 17, 2011 38 / 73

  19. Noisy text in Google Books OCR in Google Research Challenges Summary • Pan-lingual detector of noisy text • Spatial and sequential versions • Works well for most of the approx. 30 languages considered • Works well relative to several plausible alternatives • Application in books and beyond Ashok C. Popat September 17, 2011 39 / 73

  20. Noisy text in Google Books OCR in Google Research Challenges . . . which brings us to OCR • Joint work with. . . Eugene Ie, Mike Jahr, Dmitriy Genzel, Franz Och, Andrew Senior, Nemanja Spasojevic, Frank Tang, Remco Teunen, others Ashok C. Popat September 17, 2011 40 / 73

  21. Noisy text in Google Books OCR in Google Research Challenges OCR in Google Research • Organize the world’s information and make it universally accessible and useful • OCR still unavailable for some important languages • Take advantage of latest technologies • Massive amounts of data available • Goal: Best-in-the-world OCR for all scripts and languages Ashok C. Popat September 17, 2011 41 / 73

  22. Noisy text in Google Books OCR in Google Research Challenges A non-trivial task. . . Ashok C. Popat September 17, 2011 42 / 73

  23. Noisy text in Google Books OCR in Google Research Challenges A non-trivial task. . . Ashok C. Popat September 17, 2011 42 / 73

  24. Noisy text in Google Books OCR in Google Research Challenges A non-trivial task. . . Ashok C. Popat September 17, 2011 42 / 73

  25. Noisy text in Google Books OCR in Google Research Challenges A non-trivial task. . . Ashok C. Popat September 17, 2011 42 / 73

  26. Noisy text in Google Books OCR in Google Research Challenges A non-trivial task. . . Ashok C. Popat September 17, 2011 42 / 73

  27. Noisy text in Google Books OCR in Google Research Challenges A non-trivial task. . . Ashok C. Popat September 17, 2011 42 / 73

  28. Noisy text in Google Books OCR in Google Research Challenges Approach • Entirely from scratch Ashok C. Popat September 17, 2011 43 / 73

  29. Noisy text in Google Books OCR in Google Research Challenges Approach • Entirely from scratch • Multiple models and features Ashok C. Popat September 17, 2011 43 / 73

  30. Noisy text in Google Books OCR in Google Research Challenges Approach • Entirely from scratch • Multiple models and features • MERT-optimized log-linear combination Ashok C. Popat September 17, 2011 43 / 73

  31. Noisy text in Google Books OCR in Google Research Challenges Approach • Entirely from scratch • Multiple models and features • MERT-optimized log-linear combination • Latest algorithms, e.g., from speech Ashok C. Popat September 17, 2011 43 / 73

  32. Noisy text in Google Books OCR in Google Research Challenges Approach • Entirely from scratch • Multiple models and features • MERT-optimized log-linear combination • Latest algorithms, e.g., from speech • Data-driven based on massive amounts of data Ashok C. Popat September 17, 2011 43 / 73

  33. Noisy text in Google Books OCR in Google Research Challenges Early results on book images • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 44 / 73

  34. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 45 / 73

  35. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 46 / 73

  36. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 47 / 73

  37. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 48 / 73

  38. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 49 / 73

  39. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 50 / 73

  40. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 51 / 73

  41. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 52 / 73

  42. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 53 / 73

  43. Noisy text in Google Books OCR in Google Research Challenges Early results on book images (cont.) • Image • Transcription (Ref = human annotator; Rec = Google Research) Ashok C. Popat September 17, 2011 54 / 73

  44. Noisy text in Google Books OCR in Google Research Challenges Good progress so far. . . Ashok C. Popat September 17, 2011 55 / 73

  45. Noisy text in Google Books OCR in Google Research Challenges . . . but by no means done Ashok C. Popat September 17, 2011 56 / 73

  46. Noisy text in Google Books OCR in Google Research Challenges Example: Thai Ashok C. Popat September 17, 2011 57 / 73

  47. Noisy text in Google Books OCR in Google Research Challenges Example: Thai Ashok C. Popat September 17, 2011 58 / 73

  48. Noisy text in Google Books OCR in Google Research Challenges Bootstrapping a basic Thai-capable system • Steps 1 Download 25Mb of Thai text from Wikisource 2 Generate synthetic training data from text 3 Split data into training and dev set 4 Train LM from training set 5 Train optical models from training set 6 Tune system on dev set (MERT) 7 Run on images from Google books • Entire process: ∼ 12 hours! • Crippled system: small LM, small optical models, few fonts, no real dev set, no ground-truth test set Ashok C. Popat September 17, 2011 59 / 73

  49. Noisy text in Google Books OCR in Google Research Challenges Current topics of interest • Synthetic training data • Unsupervised / discriminative training • Discriminative feature extraction • More languages Ashok C. Popat September 17, 2011 60 / 73

  50. Noisy text in Google Books OCR in Google Research Challenges Challenges in Google Books Ashok C. Popat September 17, 2011 61 / 73

  51. Noisy text in Google Books OCR in Google Research Challenges Joint work with. . . Dar-Shyang Lee, Jeff Breidenbach, Stavan Parikh, Viresh Ratnakar, Ray Smith, Ranjith Unnikrishnan, others Ashok C. Popat September 17, 2011 62 / 73

  52. Noisy text in Google Books OCR in Google Research Challenges Challenges: Multiple scripts/languages on a page Ashok C. Popat September 17, 2011 63 / 73

  53. Noisy text in Google Books OCR in Google Research Challenges Challenges: per-word script and language variation Ashok C. Popat September 17, 2011 64 / 73

  54. Noisy text in Google Books OCR in Google Research Challenges Challenges: Geometric and graylevel distortions Ashok C. Popat September 17, 2011 65 / 73

  55. Noisy text in Google Books OCR in Google Research Challenges Other challenges • Multiple languages in same or similar script • Arabic-Farsi, Marathi-Hindi-Nepali • Bad initial OCR can become a virtue Ashok C. Popat September 17, 2011 66 / 73

  56. Noisy text in Google Books OCR in Google Research Challenges Other challenges • Multiple languages in same or similar script • Arabic-Farsi, Marathi-Hindi-Nepali • Bad initial OCR can become a virtue • The same language in multiple scripts • Chinese, Japanese, Azarbaijani, Mongolian, Punjabi, Hindi, Serbian, Pali Ashok C. Popat September 17, 2011 66 / 73

  57. Noisy text in Google Books OCR in Google Research Challenges Other challenges • Multiple languages in same or similar script • Arabic-Farsi, Marathi-Hindi-Nepali • Bad initial OCR can become a virtue • The same language in multiple scripts • Chinese, Japanese, Azarbaijani, Mongolian, Punjabi, Hindi, Serbian, Pali • Archaic and reformed orthographies • Fraktur, Imperial Russian, 18th century English Ashok C. Popat September 17, 2011 66 / 73

  58. Noisy text in Google Books OCR in Google Research Challenges Other challenges • Multiple languages in same or similar script • Arabic-Farsi, Marathi-Hindi-Nepali • Bad initial OCR can become a virtue • The same language in multiple scripts • Chinese, Japanese, Azarbaijani, Mongolian, Punjabi, Hindi, Serbian, Pali • Archaic and reformed orthographies • Fraktur, Imperial Russian, 18th century English • Dark matter: what scripts and languages are actually present? Ashok C. Popat September 17, 2011 66 / 73

  59. Noisy text in Google Books OCR in Google Research Challenges More challenge examples Ashok C. Popat September 17, 2011 67 / 73

  60. Noisy text in Google Books OCR in Google Research Challenges More challenge examples Ashok C. Popat September 17, 2011 67 / 73

  61. Noisy text in Google Books OCR in Google Research Challenges More challenge examples (cont.) Ashok C. Popat September 17, 2011 68 / 73

  62. Noisy text in Google Books OCR in Google Research Challenges More challenge examples (cont.) Ashok C. Popat September 17, 2011 68 / 73

  63. Noisy text in Google Books OCR in Google Research Challenges More challenge examples (cont.) Ashok C. Popat September 17, 2011 69 / 73

Recommend

Ethics in Techniques for large-scale data Graham J.L. Kemp TECHNIQUES FOR LARGE-SCALE DATA

Ethics in Techniques for large-scale data Graham J.L. Kemp TECHNIQUES FOR LARGE-SCALE DATA

Ethics in Techniques for large-scale data Graham J.L. Kemp TECHNIQUES FOR LARGE-SCALE DATA Masters level course: Chalmers MPALG and MPSOF programmes (DAT345) GU Applied Data Science programme (DIT871) Learning outcomes include:

305 views • 18 slides
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research Wang

VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research Wang

VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research Wang et.al. Qi Huang Outline 1. Motivation 2. VATEX Dataset Overview 3. Multilingual Video Captioning 4. Video-guided Machine Translation 5.

522 views • 18 slides
Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, Data-Centric

Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, Data-Centric

Overcoming the Top Four Challenges to Real-Time Performance in Large-Scale, Data-Centric Applications Tom Lubinski Founder and CEO SL Corporation 17 November 2011 Disclaimers I am not Mike Lee No Mariachi hat No Facial hair

599 views • 42 slides
Creating Large-Scale Multilingual Cognate Tables Winston Wu and David Yarowsky Center for

Creating Large-Scale Multilingual Cognate Tables Winston Wu and David Yarowsky Center for

Creating Large-Scale Multilingual Cognate Tables Winston Wu and David Yarowsky Center for Language and Speech Processing Johns Hopkins University http://educationviews.org/wp-content/uploads/2013/06/world-bread-cognates-panis.jpg Cognates and

409 views • 18 slides
Cosmic Calibration Katrin Heitmann Statistical Challenges for Large-Scale Structure in the Era of

Cosmic Calibration Katrin Heitmann Statistical Challenges for Large-Scale Structure in the Era of

Cosmic Calibration Katrin Heitmann Statistical Challenges for Large-Scale Structure in the Era of LSST Oxford , April 18, 2018 Cosmological Constraints Large Scale Simulations Cosmic Emulation Cosmic Emulators Large Synoptic Survey Telescope

944 views • 32 slides
Challenges for Large-scale Data Processing Eiko Yoneki University of Cambridge Computer

Challenges for Large-scale Data Processing Eiko Yoneki University of Cambridge Computer

Challenges for Large-scale Data Processing Eiko Yoneki University of Cambridge Computer Laboratory 2010s: Big Data Why Big Data now? Increase of Storage Capacity Increase of Processing Capacity Availability of Data Hardware

510 views • 47 slides
Challenges for Large-scale Data Processing Eiko Yoneki University of Cambridge Computer

Challenges for Large-scale Data Processing Eiko Yoneki University of Cambridge Computer

Challenges for Large-scale Data Processing Eiko Yoneki University of Cambridge Computer Laboratory 2010s: Big Data Why Big Data now? Increase of Storage Capacity Increase of Processing Capacity Availability of Data Hardware

536 views • 11 slides
A large-scale chemical data integration system Gaia Paolini Pfizer Confidential 1 Large-Scale

A large-scale chemical data integration system Gaia Paolini Pfizer Confidential 1 Large-Scale

A large-scale chemical data integration system Gaia Paolini Pfizer Confidential 1 Large-Scale Chemical Data Integration Summary Current situation Business case Aims The design process Functionality Applications

261 views • 25 slides
Internet-scale Experimentation The challenges of large-scale networked system experimentation and

Internet-scale Experimentation The challenges of large-scale networked system experimentation and

Internet-scale Experimentation The challenges of large-scale networked system experimentation and measurements MIT Tech Review on The Loon Project The state of affairs An ever growing Internet ~3 billion people 15 billion devices

1.21k views • 107 slides
PageRank and recommenders on very large scale A Big Data perspective through Stratosphere

PageRank and recommenders on very large scale A Big Data perspective through Stratosphere

PageRank and recommenders on very large scale PageRank and recommenders on very large scale A Big Data perspective through Stratosphere Mrton Balassi Data Mining and Search Group 1 1 Computer and Automation Research Institute of the Hungarian

719 views • 68 slides
Meeting the Challenges of Ultra- -Large Large- -Scale Scale Meeting the Challenges of Ultra

Meeting the Challenges of Ultra- -Large Large- -Scale Scale Meeting the Challenges of Ultra

Meeting the Challenges of Ultra- -Large Large- -Scale Scale Meeting the Challenges of Ultra Distributed Real- -time & Embedded time & Embedded Distributed Real (DRE) Systems (DRE) Systems Wednesday, May 30, 2007, ISORC, , ISORC,

730 views • 49 slides
Large-Scale Click- stream and transaction log mining in practice Uwe Mayer, Nish Parikh, Gyanit

Large-Scale Click- stream and transaction log mining in practice Uwe Mayer, Nish Parikh, Gyanit

Large-Scale Click- stream and transaction log mining in practice Uwe Mayer, Nish Parikh, Gyanit Singh October 6-9, 2013. BIG DATA SCIENCE Best Practices Key Ideas Big Data Sets Big Data Properties Challenges in working

777 views • 73 slides
Meeting the Challenges of Ultra- -Large Large- -Scale Scale Meeting the Challenges of Ultra

Meeting the Challenges of Ultra- -Large Large- -Scale Scale Meeting the Challenges of Ultra

Meeting the Challenges of Ultra- -Large Large- -Scale Scale Meeting the Challenges of Ultra Distributed Real- -time & Embedded Systems time & Embedded Systems Distributed Real with QoS- -enabled Middleware & enabled

682 views • 50 slides
FINANCING LARGE SCALE SOLAR Large Scale Solar Conference - Sydney Gloria Chan Director, Large

FINANCING LARGE SCALE SOLAR Large Scale Solar Conference - Sydney Gloria Chan Director, Large

FINANCING LARGE SCALE SOLAR Large Scale Solar Conference - Sydney Gloria Chan Director, Large Scale Solar Lead April 2017 CONTENTS 1. Introduction to CEFC 2. Investment trends 3. The future of large scale solar 4. Pathway to sustainable

664 views • 21 slides
Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of

Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of

CS Fundamental Missing Data Recovery Theory Challenges Algorithms FPC Total Variation Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of Computational and Applied Mathematics Rice University,

391 views • 27 slides
Large-scale Data Processing and Optimisation Eiko Yoneki University of Cambridge Computer

Large-scale Data Processing and Optimisation Eiko Yoneki University of Cambridge Computer

Large-scale Data Processing and Optimisation Eiko Yoneki University of Cambridge Computer Laboratory Massive Data: Scale-Up vs Scale-Out Popular solution for massive data processing scale and build distribution, combine theoretically

204 views • 18 slides
Meeting the Challenges of Ultra- -Large Large- - Meeting the Challenges of Ultra Scale Systems

Meeting the Challenges of Ultra- -Large Large- - Meeting the Challenges of Ultra Scale Systems

Meeting the Challenges of Ultra- -Large Large- - Meeting the Challenges of Ultra Scale Systems via Model- -Driven Driven Scale Systems via Model Engineering Engineering February 2, 2007 February 2, 2007 Dr. Douglas C. Schmidt

479 views • 47 slides
Weak Supervised Learning on Ray using Snorkel British Sentiment Analysis with LSTM Using Noisy

Weak Supervised Learning on Ray using Snorkel British Sentiment Analysis with LSTM Using Noisy

Weak Supervised Learning on Ray using Snorkel British Sentiment Analysis with LSTM Using Noisy Crowd Sami (sa894) - R244 Large-Scale Data Processing and Optimisation Snorkel generating training label Generating training labels with weak-

219 views • 8 slides
Efficient Large-Scale Graph Processing on Hybrid CPU and GPU Systems A. Gharaibeh, E.

Efficient Large-Scale Graph Processing on Hybrid CPU and GPU Systems A. Gharaibeh, E.

Efficient Large-Scale Graph Processing on Hybrid CPU and GPU Systems A. Gharaibeh, E. Santos-Neto, L. Costa, M. Ripeanu. IEEE TPC, 2014 Sami (sa894) - R244: Large-scale data processing and optimization Efficient Large-Scale Graph Processing on

381 views • 25 slides
Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets?

Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets?

Learning with Large Datasets L eon Bottou NEC Laboratories America Why Large-scale Datasets? Data Mining Gain competitive advantages by analyzing data that describes the life of our computerized society. Artificial Intelligence

878 views • 60 slides
Challenges of Migrating ABS Surveys to Blaise Web On a Large Scale and Short Timeframe Helen

Challenges of Migrating ABS Surveys to Blaise Web On a Large Scale and Short Timeframe Helen

Challenges of Migrating ABS Surveys to Blaise Web On a Large Scale and Short Timeframe Helen Robson Director of Corporate eCollect Australian Bureau of Statistics Outline About the ABS Key drivers for change The journey

309 views • 15 slides
An NDN Testbed for Large-scale Scientific Data Huhnkuk Lim Korea Institute of Science &

An NDN Testbed for Large-scale Scientific Data Huhnkuk Lim Korea Institute of Science &

An NDN Testbed for Large-scale Scientific Data Huhnkuk Lim Korea Institute of Science & Technology Information (KISTI) NDNComm 2015 Sep. 28, 2015 Motivations on NDN for Large-scale Scientific Application As the data volumes and

226 views • 11 slides
Implementation of large-scale online mathematics examinations: successes, challenges and

Implementation of large-scale online mathematics examinations: successes, challenges and

Implementation of large-scale online mathematics examinations: successes, challenges and thoughts for the future Martyn Parker and Lynne Bailey, University of Warwick Outline Operating context Assessment design Implementation Successes,

1.07k views • 72 slides
Large-Scale Data Engineering Data streams and low latency processing event.cwi.nl/lsde2015 DATA

Large-Scale Data Engineering Data streams and low latency processing event.cwi.nl/lsde2015 DATA

Large-Scale Data Engineering Data streams and low latency processing event.cwi.nl/lsde2015 DATA STREAM BASICS event.cwi.nl/lsde2015 What is a data stream? Large data volume, likely structured, arriving at a very high rate Potentially

498 views • 38 slides

More recommend