the details training and valida1ng big models on big data
play

The details: training and valida1ng big models on big data - PowerPoint PPT Presentation

Feb 15, 2024 •173 likes •682 views

The details: training and valida1ng big models on big data David Mimno Princeton, Computer Science 1 Saturday, November 3, 12 2 Saturday, November 3, 12 George Dyson, Turings

  1. The ¡details: ¡training ¡and ¡ valida1ng ¡big ¡models ¡on ¡big ¡data David ¡Mimno Princeton, ¡Computer ¡Science 1 Saturday, November 3, 12

  2. 2 Saturday, November 3, 12

  3. George ¡Dyson, ¡Turing’s ¡Cathedral • “The ¡reac*on ¡of ¡most ¡meteorologists ¡ towards ¡computer-­‑assisted ¡forecas*ng ¡ paralleled ¡that ¡of ¡the ¡Ins*tute ¡ mathema*cians ¡towards ¡computer-­‑assisted ¡ mathema*cs: ¡skep*cism ¡that ¡a ¡machine ¡ could ¡improve ¡upon ¡what ¡they ¡were ¡doing ¡ with ¡brains ¡alone.” 3 Saturday, November 3, 12

  4. George ¡Dyson, ¡Turing’s ¡Cathedral • “The ¡reac*on ¡of ¡most ¡meteorologists ¡ towards ¡ computer-­‑assisted ¡forecas*ng ¡ paralleled ¡that ¡of ¡the ¡Ins*tute ¡ mathema*cians ¡towards ¡computer-­‑assisted ¡ mathema*cs: ¡skep*cism ¡that ¡a ¡machine ¡ could ¡improve ¡upon ¡what ¡they ¡were ¡doing ¡ with ¡brains ¡alone.” 4 Saturday, November 3, 12

  5. Outline • Training ¡topic ¡models • Modeling ¡choices • Diagnos*cs 5 Saturday, November 3, 12

  6. Outline • Training ¡topic ¡models • Modeling ¡choices • Diagnos*cs 6 Saturday, November 3, 12

  7. Library-­‑scale ¡topic ¡models Input: ¡1.2M ¡pre-­‑1922 ¡books Output: ¡2000 ¡“topics” (33 ¡billion ¡non-­‑stopwords) (distribu*ons ¡over ¡words) Random ¡examples, ¡each ¡row ¡is ¡a ¡topic 7 Saturday, November 3, 12

  8. Latent ¡Dirichlet ¡Alloca1on α Book-­‑topic ¡propor*ons θ η z w β N d D K 8 Saturday, November 3, 12

  9. An ¡example ¡document Etruscan trade price temple market 9 Saturday, November 3, 12

  10. Assign ¡topics z 3 2 1 3 1 Etruscan trade price temple market w 10 Saturday, November 3, 12

  11. Assign ¡topics z 3 2 1 3 1 Etruscan trade price temple market w 3 2 1 3 1 3 2 1 3 1 3 2 1 3 1 3 2 1 3 1 Etrusca trade price temple market 3 2 1 3 1 3 2 1 3 1 3 2 1 3 1 Etrusca trade price temple market 3 2 1 3 1 Etrusca trade price temple market n 3 2 1 3 1 3 2 1 3 1 Etrusca trade price temple market 3 2 1 3 1 Etrusca trade price temple market n 3 2 1 3 1 3 2 1 3 1 Etrusca trade price temple market n Etrusca trade price temple market 3 2 1 3 1 Etrusca trade price temple market n 3 2 1 3 1 3 2 1 3 1 Etrusca trade price temple market n Etrusca trade price temple market 3 2 1 3 1 n Etrusca trade price temple market n 3 2 1 3 1 3 2 1 3 1 Etrusca trade price temple market n Etrusca trade price temple market 3 2 1 3 1 n Etrusca trade price temple market n 3 2 1 3 1 3 2 1 3 1 Etrusca trade price temple market n Etrusca trade price temple market 3 2 1 3 1 n Etrusca trade price temple market n 3 2 1 3 1 2 3 2 1 1 Etrusca trade price temple market n Etrusca trade price temple market 3 2 1 3 1 n Etrusca trade price temple market n 3 2 1 3 1 Etrusca trade price temple market n Etrusca trade price temple market 2 3 2 1 1 n Etrusca trade price temple market n 3 2 1 3 1 Etrusca trade price temple market n trade temple ship trade market n Etrusca trade price temple market n 2 3 2 1 1 Etrusca trade price temple market n n ship temple ship trade market Etrusca trade price temple market n n Italy temple ship trade market n 11 Saturday, November 3, 12

  12. Global ¡sta1s1cs 3 2 1 3 1 Etruscan trade price temple market 1 2 3 Etruscan 1 0 35 Total ¡ market 50 0 1 counts ¡ price 42 1 0 from ¡ all ¡ temple 0 0 20 docs trade 10 8 1 ... 12 Saturday, November 3, 12

  13. Algorithm • Ini*alize ¡topic ¡assignments ¡randomly • For ¡each ¡itera*on: – For ¡each ¡document: • For ¡each ¡word: – Resample ¡topic ¡for ¡word, ¡given ¡all ¡other ¡words ¡and ¡their ¡ current ¡topic ¡assignments • Produce ¡reports 13 Saturday, November 3, 12

  14. Algorithm • Ini*alize ¡topic ¡assignments ¡randomly • For ¡each ¡itera*on: – For ¡each ¡document: • For ¡each ¡word: – Resample ¡topic ¡for ¡word, ¡given ¡all ¡other ¡words ¡and ¡their ¡ current ¡topic ¡assignments • Produce ¡reports 14 Saturday, November 3, 12

  15. Sample ¡topic ¡for ¡“trade” 3 2 1 3 1 Etruscan trade price temple market 1 2 3 Etruscan 1 0 35 market 50 0 1 price 42 1 0 temple 0 0 20 trade 10 8 1 ... 15 Saturday, November 3, 12

  16. Remove ¡current ¡assignment 3 2 1 3 1 Etruscan trade price temple market 1 2 3 Etruscan 1 0 35 market 50 0 1 price 42 1 0 temple 0 0 20 trade 10 8 1 ... 16 Saturday, November 3, 12

  17. Remove ¡current ¡assignment 3 ? 1 3 1 Etruscan trade price temple market 1 2 3 Etruscan 1 0 35 market 50 0 1 price 42 1 0 temple 0 0 20 trade 10 7 1 ... 17 Saturday, November 3, 12

  18. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market 18 Saturday, November 3, 12

  19. Which ¡topics ¡occur ¡in ¡this ¡doc? 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 19 Saturday, November 3, 12

  20. Which ¡topics ¡like ¡the ¡word ¡“trade”? 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 1 2 3 trade 10 7 1 20 Saturday, November 3, 12

  21. Which ¡topics ¡like ¡the ¡word ¡“trade”? 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 21 Saturday, November 3, 12

  22. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 22 Saturday, November 3, 12

  23. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 23 Saturday, November 3, 12

  24. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market 1 2 3 Etruscan 1 0 35 market 50 0 1 price 42 1 0 temple 0 0 20 trade 10 7 1 ... 24 Saturday, November 3, 12

  25. Pick ¡a ¡topic ¡for ¡“trade” 3 1 1 3 1 Etruscan trade price temple market 1 2 3 Etruscan 1 0 35 market 50 0 1 price 42 1 0 temple 0 0 20 trade 11 7 1 ... 25 Saturday, November 3, 12

  26. Increase ¡counts ¡for ¡1 ¡and ¡“trade”|1 3 1 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 26 Saturday, November 3, 12

  27. Varia1onal ¡inference Etruscan trade price temple market Topic 1 Topic 2 Topic 3 27 Saturday, November 3, 12

  28. Outline • Training ¡topic ¡models • Modeling ¡choices • Diagnos*cs 28 Saturday, November 3, 12

  29. Things ¡people ¡didn’t ¡know ¡they ¡ had ¡to ¡think ¡about • What ¡is ¡a ¡ document ? • Which ¡ words ¡ are ¡interes*ng? • What ¡is ¡a ¡word, ¡anyway? • Knobs: – Number ¡of ¡topics – Hyper-­‑parameters 29 Saturday, November 3, 12

  30. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 30 Saturday, November 3, 12

  31. Which ¡topics ¡like ¡the ¡word ¡“trade”? Topic 1 31 Saturday, November 3, 12

  32. Which ¡topics ¡like ¡the ¡word ¡“trade”? Topic 1 α price market 32 Saturday, November 3, 12

  33. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 33 Saturday, November 3, 12

  34. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 34 Saturday, November 3, 12

  35. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 35 Saturday, November 3, 12

  36. Pick ¡a ¡topic ¡for ¡“trade” 3 ? 1 3 1 Etruscan trade price temple market Topic 1 Topic 2 Topic 3 36 Saturday, November 3, 12

  37. Hyper-­‑parameters: ¡learn ¡or ¡fix? Pros Cons Fixed ¡ All ¡topics ¡ Duplicate ¡ similar ¡size, ¡ topics, ¡frequent ¡ quality words ¡repeated Learned Some ¡topics ¡ Small ¡topics ¡ big, ¡others ¡ may ¡be ¡low ¡ small quality 37 Saturday, November 3, 12

  38. Outline • Training ¡topic ¡models • Modeling ¡choices • Diagnos8cs 38 Saturday, November 3, 12

  39. What ¡makes ¡topics ¡bad? • Random , ¡unrelated ¡words • One ¡or ¡two ¡“ intruder ” ¡words • Boring, ¡ overly ¡general ¡words • Two ¡or ¡more ¡good ¡topics ¡combined, ¡ some*mes ¡with ¡a ¡general ¡word ¡in ¡common ¡ ( chimaeras ) 39 Saturday, November 3, 12

Recommend

Devils, Details, and Data: Measurement Models and Analysis Strategies for Novel Technology-Based

Devils, Details, and Data: Measurement Models and Analysis Strategies for Novel Technology-Based

Devils, Details, and Data: Measurement Models and Analysis Strategies for Novel Technology-Based Clinical Outcome Assessments ISCT SCTM 2018 Au Autumn Mee eeting Robert M Bilder, UCLA Michael E. Tennenbaum Family Professor Psychiatry &

992 views • 31 slides
Updates on data details & running a hydrologic model on tree rings Data details The

Updates on data details & running a hydrologic model on tree rings Data details The

Steph McAfee University of Nevada, Reno Nevada State Climate Office Updates on data details & running a hydrologic model on tree rings Data details The number and location of stations changed over time. 1900 1920 1940

312 views • 20 slides
Coresets for Data-efficient Training of Machine Learning Models Baharan Mirzasoleiman*, Jeff

Coresets for Data-efficient Training of Machine Learning Models Baharan Mirzasoleiman*, Jeff

Coresets for Data-efficient Training of Machine Learning Models Baharan Mirzasoleiman*, Jeff Bilmes**, Jure Leskovec* * * ** Machine Learning Becomes Mainstream Personalized medicine Robotics Finance Autonomous cars 2 Data is the Fuel

704 views • 22 slides
Make sure we can query black box algorithms

Make sure we can query black box algorithms

Make sure we can query black box algorithms http://www.bloomberg.com/graphics/2016-amazon-same-day/ Auditing Black Box Models Training vs Testing No access to training Training data or algorithm data Test data Auditing Black Box

323 views • 16 slides
MMM Training Solutions Presentation Skills Training Contact Details Pramila.mathew@ mmmts.com

MMM Training Solutions Presentation Skills Training Contact Details Pramila.mathew@ mmmts.com

MMM Training Solutions Presentation Skills Training Contact Details Pramila.mathew@ mmmts.com Email: Pramila Mathew, Phone: +91-98409 88449 Contact: Overview: Excellent presentation skills give you a platform to demonstrate your

209 views • 5 slides
Learning Better Object Models using Video Data Patrick Li, Inmar Givoni, Brendan Frey Motivation

Learning Better Object Models using Video Data Patrick Li, Inmar Givoni, Brendan Frey Motivation

Learning Better Object Models using Video Data Patrick Li, Inmar Givoni, Brendan Frey Motivation Training on a collection of static monocular images is unnatural. Labelled Training Images are hard to get. And the lack of is becoming a problem.

567 views • 18 slides
Session 04 Model selection An example: Cars 93 data Details (~data) from 93 makes of car

Session 04 Model selection An example: Cars 93 data Details (~data) from 93 makes of car

Session 04 Model selection An example: Cars 93 data Details (~data) from 93 makes of car released in the USA in 1993. Variable names largely self-explanatory. Problem: build a prediction equation for the fuel economy from the other

563 views • 21 slides
TRAINING THE BASIC LAWS / RULES OF THE SPORT AT EVERY LEVEL. Theory The same but details

TRAINING THE BASIC LAWS / RULES OF THE SPORT AT EVERY LEVEL. Theory The same but details

TRAINING THE BASIC LAWS / RULES OF THE SPORT AT EVERY LEVEL. Theory The same but details may differ based on level Interpretation Need guidance / From experience Testing Levels adequate for the students success Has

497 views • 15 slides
INSTRUMENTATION AND ANALYTICAL SALES, SERVICE & TRAINING COMPANY DETAILS BUSINESS MODEL

INSTRUMENTATION AND ANALYTICAL SALES, SERVICE & TRAINING COMPANY DETAILS BUSINESS MODEL

INDUSTRIAL AUTOMATION & PROCESS CONTROL SOLUTIONS INSTRUMENTATION AND ANALYTICAL SALES, SERVICE & TRAINING COMPANY DETAILS BUSINESS MODEL Established December 10, 2001 Represent No More Than 10 Complimentary Product Lines

246 views • 13 slides
CoT: Cooperative Training for Generative Modeling of Discrete Data

CoT: Cooperative Training for Generative Modeling of Discrete Data

CoT: Cooperative Training for Generative Modeling of Discrete Data https://github.com/desire2020/CoT Sidi Lu, Lantao Yu, Siyuan Feng, Yaoming Zhu, Weinan Zhang, and Yong Yu Shanghai Jiao Tong University Autoregressive Models Autoregressive

399 views • 16 slides
Models and The Unified Modeling Language (UML) What is a Model: Maps Models are abstractions of

Models and The Unified Modeling Language (UML) What is a Model: Maps Models are abstractions of

Models and The Unified Modeling Language (UML) What is a Model: Maps Models are abstractions of the real situation to give a clearer view of some aspect. Maps are models of some region terrain and geography. Suppress details (buildings, etc.)

293 views • 11 slides
Color orado A o ALTO P Project Data Training Data Training Objectives Understand the

Color orado A o ALTO P Project Data Training Data Training Objectives Understand the

Color orado A o ALTO P Project Data Training Data Training Objectives Understand the alarming statistics/background Discuss the population of interest and timeframe Review naming report conventions; drugs monitored in the program;

204 views • 17 slides
Language Modeling Professor Marie Roch for details on N-gram models, see chapter 4: Jurafsky, D.,

Language Modeling Professor Marie Roch for details on N-gram models, see chapter 4: Jurafsky, D.,

Language Modeling Professor Marie Roch for details on N-gram models, see chapter 4: Jurafsky, D., and Martin, J. H. (2009). Speech and Language Processing (Pearson Prentice Hall, Upper Saddle River, NJ) Narrowing search with a language model

505 views • 17 slides
Grant Reporting Training Know & Follow The Details Of YOUR Grant Project Narrative

Grant Reporting Training Know & Follow The Details Of YOUR Grant Project Narrative

Fund For Veterans Assistance Grant Reporting Training Know & Follow The Details Of YOUR Grant Project Narrative Budget (Approved) Budget Narrative 2 Know & Follow The Rules Of FVA Grants Request for Applications

657 views • 48 slides
Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research

Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research

Day 4 Lecture 1 Generative models and adversarial training Kevin McGuinness kevin.mcguinness@dcu.ie Research Fellow Insight Centre for Data Analytics Dublin City University What is a generative model? A model P(X; ) that we can draw

416 views • 20 slides
TraininG towards a society of data-saVvy inforMation prOfessionals to enable open leadership

TraininG towards a society of data-saVvy inforMation prOfessionals to enable open leadership

TraininG towards a society of data-saVvy inforMation prOfessionals to enable open leadership INnovation Performance Comparison of Ad-hoc Retrieval Models over Full-text vs. Titles of Documents Ahmed Saleh , Tilman Beck, Lukas Galke, Ansgar Scherp

549 views • 27 slides
Data Lake to AI on GPUs CPUs can no longer handle the growing data demands of data science

Data Lake to AI on GPUs CPUs can no longer handle the growing data demands of data science

Data Lake to AI on GPUs CPUs can no longer handle the growing data demands of data science workloads Slow Process Suboptimal Infrastructure Preparing data and training models Hundreds to tens of thousands of CPU can take days or even weeks.

308 views • 17 slides
Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum

Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum

0. Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum & T. Mitchell, 1998 1. Problem: Learning to classify data (ex: web pages) when the description of each example can be partitioned in 2

560 views • 9 slides
Mini-Series: Own your CV Episode 2 Name, Contacts, Profile ,Education Name and Contact Details

Mini-Series: Own your CV Episode 2 Name, Contacts, Profile ,Education Name and Contact Details

Mini-Series: Own your CV Episode 2 Name, Contacts, Profile ,Education Name and Contact Details Sections of a CV we will be Personal Profile focusing on Education and Training NAME AND CONTACT DETAILS Name and Contact Details Do not

417 views • 12 slides
Industrial Training October 2016 Session - Oral Presentation Timetable Dates: 10, 11 & 12

Industrial Training October 2016 Session - Oral Presentation Timetable Dates: 10, 11 & 12

Universiti Tunku Abdul Rahman (UTAR) Faculty of Science Industrial Training October 2016 Session - Oral Presentation Timetable Dates: 10, 11 & 12 January 2017 (16 January 2017 - Special Session) Student Details Company Details Oral

434 views • 21 slides
CSE 291D/234 Data Systems for Machine Learning Arun Kumar Topic 1: Classical ML Training at

CSE 291D/234 Data Systems for Machine Learning Arun Kumar Topic 1: Classical ML Training at

CSE 291D/234 Data Systems for Machine Learning Arun Kumar Topic 1: Classical ML Training at Scale Chapters 2, 5, and 6 of MLSys book 1 Academic ML 101 Classical ML Generalized Linear Models (GLMs); from statistics Bayesian Networks ;

1.22k views • 93 slides
Assistants in Italy - Training Day www.britishcouncil.org Team Italy - Contact details Simon

Assistants in Italy - Training Day www.britishcouncil.org Team Italy - Contact details Simon

LANGUAGE ASSISTANTS PROGRAMME MONDAY 18 NOVEMBER & TUESDAY 19 NOVEMBER English Language Assistants in Italy - Training Day www.britishcouncil.org Team Italy - Contact details Simon Graham Consultant

591 views • 31 slides
CAM DATA REPORTS TRAINING Oregon Department of Human Services Printable supplemental training

CAM DATA REPORTS TRAINING Oregon Department of Human Services Printable supplemental training

CAM Data Reports Training Presentation 1/30/2019 CAM DATA REPORTS TRAINING Oregon Department of Human Services Printable supplemental training materials, including a Workbook and Job aid, are referenced throughout this session to help your

640 views • 47 slides
Introduction to Data Science: Logistic 0 1 1 according to a data fit criterion. account

Introduction to Data Science: Logistic 0 1 1 according to a data fit criterion. account

Linear models for classication Classier evaluation Linear models for classication Linear models for classication Linear models for classication Linear models for classication Linear models for classication Linear models

619 views • 34 slides

More recommend