classiefier using machine learning to paint a picture of
play

CLASSIEFIER: USING MACHINE LEARNING TO PAINT A PICTURE OF SOCIAL - PowerPoint PPT Presentation

Apr 29, 2023 •517 likes •904 views

Dr. Paola Oliva-Altamirano, Innovation Lab, Our Community, May 2019 CLASSIEFIER: USING MACHINE LEARNING TO PAINT A PICTURE OF SOCIAL TRENDS A foreigner Who am I? From Honduras to the US to Australia From Galaxies to Taxonomies Dr. Paola

  1. Dr. Paola Oliva-Altamirano, Innovation Lab, Our Community, May 2019 CLASSIEFIER: USING MACHINE LEARNING TO PAINT A PICTURE OF SOCIAL TRENDS

  2. A foreigner Who am I? From Honduras to the US to Australia From Galaxies to Taxonomies • Dr. Paola Oliva-Altamirano, Innovation Lab, Our Community, May 2019 Our Community - Innovation Lab 2

  3. Outline: Introducing Our community’s data initiatives • Background: CLASSIE a social dictionary • How did we scope CLASSIEfier ? • How did CLASSIEfier evolve as a project? • • Data science for social good concept Results and conclusions • Our Community - Innovation Lab 3

  4. Is a social enterprise and B Corp that provides advice, connections, training and easy-to-use tech tools for community-builders. Training and networking Grants database Donation Platform Software for grants applications Our Community - Innovation Lab 4

  5. Our Community - Innovation Lab 5

  6. From CLASSIE to CLASSIEfier

  7. Main objective – Classification of grants In 2016, OC introduced Australia lacked a CLASSIE unified taxonomy to CLASSIE opens the door The classification classify subjects, to standard system for Australian beneficiaries and classification social sector initiatives organization types and entities Our Community - Innovation Lab 7

  8. • Subjects CLASSIE Populations • A social sector dictionary • Organisation type Where is the money going? and How is the Australian social sector working? Our Community - Innovation Lab 8

  9. Hierarchical Classification – e.g. Subjects Level 1 Sport and Social Sciences recreation 17 categories Interdisciplinary Community Level 2 Anthropology Sport studies recreation 132 categories Biological Archeology Camps Ethnic studies Asian studies Parks Outdoor sport Paralympics Level 3 anthropology 492 categories Indigenous Mountain and Hiking and Level 4 studies rock climbing walking 243 categories Our Community - Innovation Lab 9

  10. Now we have the dictionary – How do we apply it? How do we ensure that users are • Questions choosing the correct category ? How do we classify historical data ? • 800,000 grant applications since 2010 Our Community - Innovation Lab 10

  11. CLASSIEfier is a tool that will automatically classify grants Our Community - Innovation Lab 11

  12. How did we scope CLASSIEfier?

  13. Source: “One model to rule them all” by Christoph Molnar

  14. CLASSIEfier – Two different models To give automatic suggestions to grant applicants 1. To classify historical data 2. Seems like you are applying for: q Sports and recreation q Art and culture q Community and development Our Community - Innovation Lab 14

  15. CLASSIEfier: How does it work? 15 Our Community - Innovation Lab

  16. How did CLASSIEfier evolve?

  17. CLASSIEfier – The Algorithm What do we have? 800,000 4,000 grant applications grant applications labeled by users since CLASSIE went live How do we generate more labels? At least 2000 applications per category Our Community - Innovation Lab 17

  18. CLASSIEfier – The Algorithm Keyword matching = the process of searching for ‘Literal’ First phase: matches (e.g. “hospital”) in a given piece of text (e.g. a grant description) to identify groups or subjects (e.g. health sector). a simple keyword matching to Example: extract more labels This project will raise awareness and empower deaf deaf people by providing key mental health information in their primary language (Australian Sign Language Sign Language ). People with hearing impediment People with hearing impediment . Stages: For example: • Identify keywords for CLASSIE “orphans” is a confusing category. • Extract applications that exhibit a strong match “wildlife welfare” is a straight forward • Score the classification done by Users category We found that: • Keyword matching accuracy differs from one category to another. • On average is around 80% Our Community - Innovation Lab 18

  19. CLASSIEfier – The Algorithm Training dataset: Second phase: 128,000 Training the Machine Learning model grant applications Classified by keyword matching DIFFICULTY #1: Multilabel DIFFICULTY #2: Hierarchy DIFFICULTY #3: Number of labels per category Our Community - Innovation Lab 19

  20. Example: A grant application that is aimed at helping teenagers teenagers with autism autism . Multilabels and Hierarchy Beneficiaries: • “Children and youth” at level 1 • “Adolescents” at level 2 And also, • “People with disabilities” at level 1 • “People with intellectual disabilities” at level 2 Our Community - Innovation Lab 20

  21. DIFFICULTY #3: Number of labels per category Categories such as Confucius, North American people , Nomadic • people among others will have less than 100 grant applications. 20X less Than the 2000 minimum required Niche classification or “ black holes ” Our Community - Innovation Lab 21

  22. How do we solve it? – Separate training Reads the application Classification Level 1 – Machine learning Information and Sports and recreation communications Classification Level 2: Classification Level 2: We have enough we do not have enough labels we use another labels we use ML model keyword matching Classification Level 3: Classification Level 3: Keyword matching Keyword matching Our Community - Innovation Lab 22

  23. CLASSIEfier – The Algorithm Third phase: Model interpretation: scoring and checking for biases Stages: Choose the best model – k- nearest neighbours (k-nn) • Choose the best parameters • Choose the best scoring • Our Community - Innovation Lab 23

  24. Scoring Recall: !" !"#$% &'(&)*+&,' ,- .&//&'0 1*213/ Precision: !" !"#$" &'(&)*+&,' ,- 2*( 453(&)+&,'/ Our Community - Innovation Lab 24

  25. Scoring Based on the fact that each application has several categories Recall: How many categories got picked per application 0 None 1 <45% 2 >45% 3 Perfect match Precision: How many categories are wrong per application 0 All 1 >55% 2 <55% 3 None – Perfect match CLASSIEfier ~4-5 0 6 Useless Model Perfect Model!! Our Community - Innovation Lab 25

  26. Misclassifications and black holes will cause to underfund minorities that are already overlooked Our Community - Innovation Lab 26

  27. “ The best minds of my generation are thinking about how to make people click ads ,” he says. “That sucks.” -- Jeff Hammerbacher (Cloudera and Facebook data leader) The Data Science for Social Good Movement

  28. Algorithmic bias • This will happen if you feed in the algorithm with data that is already biased or with insufficient data - The algorithm will predict biased classifications. Algorithms are mirrors • Sport people Our Community - Innovation Lab 28

  29. Know your Model! xkdc.com/1838/ Our Community - Innovation Lab 29

  30. SHAP (SHapley Additive exPlanations) AI Fairness 360 WEAT tests proposed in Caliskan et al. 2017 Our Community - Innovation Lab 30

  31. Document everything! – this is how we tackle biases Choose transparency Our Community - Innovation Lab 31

  32. Results and conclusions Church Religion Model = Religion Christian Reality – A fete in a Catholic school It is not feasible to classify human natural languages with 100% accuracy Our Community - Innovation Lab 32

  33. Results and conclusions Out 200 applications classified by Users we found that: Church Religion Christian 63% 18% 19% right wrong Half right CLASSIEfier works similar to humans , not better not worse. ~ 70-80% accuracy • Our Community - Innovation Lab 33

  34. Results and conclusions Church Religion Christian Approved Declined Grant applications Grant applications 85% accuracy 75% accuracy The model is also discriminating between good and bad applications • Our Community - Innovation Lab 34

  35. Results and conclusions Church Seems like you are applying Religion for: Christian q Sports and recreation q Art and culture q Community and development CLASSIEfier is now feeding back into CLASSIE Our Community - Innovation Lab 35

  36. CLASSIEfier – More than just an algorithm Writing and testing the Production – back and front Data preprocessing algorithm end product Maintenance Our Community - Innovation Lab 36

  37. Linkedin: paola-oliva-altamirano Email: paolao@ourcommunity.com.au Innovation lab: https://www.ourcommunity.com.au/innovationlab DO YOU WANT TO LEARN MORE?

Recommend

PAC Learning Matt Gormley Lecture 14 Oct. 17, 2018 1 ML Big Picture Learning Paradigms:

PAC Learning Matt Gormley Lecture 14 Oct. 17, 2018 1 ML Big Picture Learning Paradigms:

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University PAC Learning Matt Gormley Lecture 14 Oct. 17, 2018 1 ML Big Picture Learning Paradigms: Problem Formulation:

283 views • 17 slides
Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning:

Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning:

Linear Models Machine Learning 1 Checkpoint: The bigger picture Supervised learning: instances, concepts, and hypotheses Specific learners Learning Hypothesis/ Labeled algorithm Model h Decision trees data New example

519 views • 41 slides
Online Learning Machine Learning 1 Some slides based on lectures from Dan Roth, Avrim Blum and

Online Learning Machine Learning 1 Some slides based on lectures from Dan Roth, Avrim Blum and

Online Learning Machine Learning 1 Some slides based on lectures from Dan Roth, Avrim Blum and others Big picture 2 Big picture Last lecture: Linear models 3 Big picture Linear models How good is a learning algorithm? 4 Big picture

968 views • 65 slides
PAC Learning + Midterm Review Matt Gormley Lecture 15 March 7, 2018 1 ML Big Picture

PAC Learning + Midterm Review Matt Gormley Lecture 15 March 7, 2018 1 ML Big Picture

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University PAC Learning + Midterm Review Matt Gormley Lecture 15 March 7, 2018 1 ML Big Picture Learning Paradigms: Problem

692 views • 55 slides
Support Vector Machines Machine Learning 1 Big picture Linear models 2 Big picture Linear

Support Vector Machines Machine Learning 1 Big picture Linear models 2 Big picture Linear

Support Vector Machines Machine Learning 1 Big picture Linear models 2 Big picture Linear models How good is a learning algorithm? 3 Big picture Linear models Perceptron, Winnow Online learning How good is a learning algorithm?

992 views • 73 slides
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 14, 2015 Today: Readings: The Big Picture Decision trees, overfiting Overfitting Mitchell, Chapter 3 Review: probability

841 views • 65 slides
Big Picture Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University March 2

Big Picture Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University March 2

Big Picture Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University March 2 nd , 2005 What you have learned thus far Learning is function approximation Point estimation Regression Nave Bayes Logistic

598 views • 21 slides
Peter Thorpe Peter Thorpe We would like you to be able to paint a space themed picture in the

Peter Thorpe Peter Thorpe We would like you to be able to paint a space themed picture in the

Peter Thorpe Peter Thorpe We would like you to be able to paint a space themed picture in the style of the famous artist Peter Thorpe. Have ever heard of Peter Thorpe? Lets learn a little more about him... What is abstract art? A painting

531 views • 11 slides
An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning http://www.cs.iastate.edu/~cs573x/bbsilab.html Machine Learning Software Preparing Data Building Classifiers Interpreting Results Machine Learning Software

496 views • 26 slides
Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Checkpoint: The bigger picture Supervised learning: instances, concepts, and

895 views • 42 slides
Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by which computers can be trained through observation, rather than being explicitly programmed. Basically, a program is given input and adjusts its

559 views • 17 slides
Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: Readings: What is machine learning? The Discipline of ML Decision tree learning Mitchell, Chapter 3

872 views • 29 slides
Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning Pipeline in Classification Tasks A set of hand-designed audio features are selected

401 views • 26 slides
CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine

1/22/19 CS 335 Machine Learning What is Machine Learning? Dan Sheldon Spring 2019 What is Machine Learning? A Simple Task: Recognize Obama How do you program a computer to Recognize faces? Recommend movies? Decide which web

582 views • 5 slides
Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their

Machine Learning Machine Learning: algorithms that use experience to improve their performance We use machine learning in situations where it is very challenging (or impossible) to define the rules by hand: e.g. face detection

1.04k views • 28 slides
Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine

Introduction to Machine Learning COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Introduction to Machine Learning 1 / 12 Outline 1 Nearest Neighbor Prediction 2 Complexity Considerations 3 The Voronoi Diagram 4 Overfitting

565 views • 12 slides
1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

What is Learning? Herbert Simon: Learning is any process by which a system improves performance from CS 391L: Machine Learning experience. Introduction What is the task? Classification Problem solving / planning /

603 views • 6 slides
MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING

JACOPO DI IORIO, MATTEO FONTANA, DANIELE OCCHIUTO, CLAUDIA VOLPETTI MACHINE LEARNING, STATISTICAL LEARNING AND PARALLEL COMPUTING INTRODUCTION VS MACHINE LEARNING STATISTICAL LEARNING Separate evolutions, but with shared methodologies MERGE

565 views • 55 slides
Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine

Apache PredictionIO End-to-End Machine Learning Server with Apache Spark What is Machine Learning? What is Machine Learning? Examples of machine learning applications: Facial recognition software E-mail spam detectors Automated

302 views • 17 slides
Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning

Machine Learning 11 AI Slides (6e) c Lin Zuoquan@PKU 1998-2020 11 1 11 Machine Learning 11.1 Learning agents 11.2 Inductive learning 11.3 Deep learning 11.4 Statistical learning 11.5 Reinforcement learning 11.6 Transfer learning

1.7k views • 159 slides
Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Deep Learning: Intro Juhan Nam Review of Traditional Machine Learning The traditional machine learning pipeline Temporal Frame-level Unsupervised Classifier Summary

613 views • 27 slides
Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine

Machine Learning for Auto Optimization What is Machine Learning? Definition: Machine learning refers to any system where the performance of a machine in performing a task improves by gaining more experience in performing that task .

394 views • 27 slides
MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How

MACHINE LEARNING 2012 MACHINE LEARNING MACHINE LEARNING kernels 1 MACHINE LEARNING 2012 MACHINE LEARNING Kernels: Intuition How to separate the red class from the grey class? x 2 360 r x 1 Polar coordinates Data

1.04k views • 44 slides
BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

photo:@rewardyfahmi // Unsplash BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest Neighbor Classifier Aykut Erdem // Hacettepe University // Fall 2019 When Do We Use Machine Learning? ML is

1.33k views • 110 slides

More recommend