Better Language Models and Their Implications (GPT-2) Joey Lim - PowerPoint PPT Presentation

Feb 26, 2024 •402 likes •573 views

Better Language Models and Their Implications (GPT-2) Joey Lim List of Artificial Intelligence in Films A collection of 113 films from 1927-2019 that features some form of Artificial intelligence. This list is further divided into two

Better Language Models and Their Implications (GPT-2) Joey Lim
List of Artificial Intelligence in Films A collection of 113 films from 1927-2019 that features some form of Artificial intelligence. This list is further divided into two categories: Computer or Program, Robot. Of 113 films, 21 films featured both a Computer/Program and a Robot 1 , while the vast majority of the films featured Robots instead of Computer/Programed intelligence. 1. The red box indicates the appearance of both Computer/Program and Robots.
Notable Examples “HAL-9000” from 2001: A Space Odyssey (1968) “Samantha” from Her (2013)
HAL-9000 HAL-9000, short for Heuristically Programmed Algorithmic Computer, is an artificial intelligence that controls the operation of the Discovery One spaceship. HAL is most notable for being able to interact with the astronauts with its own seeming consciousness. HAL is able to not only comprehend speech and context but also exemplifies high intelligence in conditional cognition.
Samantha
1968 vs. 2013 - 2001... first depicted a stylized form of speech, which was beyond the imagination of what people at the time thought “Machines” could do. This added a human element to the otherwise monotonous HAL. - Samantha portrayed the ‘human’ capacity of artificial intelligence one step further by not only further stylizing colloquial speech, but also with speech that resembles human intonation. (Which has been already brought to consumer levels with Siri, Alexa, and etc.)
How GPT-2 Operates
Release Strategy (Staged Release) -While the initial announcement in February 2019, limited the model to a 345M Parameter version, as of November 2019, Open AI has released the full 1.5B Parameter version of GPT-2. -Over the 9 months of beta trials, and further research into release strategy, the 1.5B Parameter version was deemed to be ready for the public, despite the numerous factors that are still unresolved or widely discussed.
Discussion 1. General thoughts or ideas? 2. How does ‘artificially’ created text pose a different threat to the idea of “Truth” than images created/altered by mechanisms like deep fakes? 3. Does the GPT-2 pressure our unique ability as humans? And can this current state of technology be compared similarly to the industrial revolution devaluing human labor? 4. Could the commodification of this technology create greater social inequality or havoc? (ie. Price difference in 345M Parameter vs. 1.5B parameter)

Recommend

Models of Language Evolution models thereof its evolution language Models of Language Evolution

Models of Language Evolution models thereof its evolution language Models of Language Evolution ? What is language? never start with a dictionary definition!! the language of Google search query completion A language is a dialect with

1.13k views • 30 slides

4 Language Models 2: Log-linear Language Models This chapter will discuss another set of language

4 Language Models 2: Log-linear Language Models This chapter will discuss another set of language models: log-linear language models [9, 4], which take a very di ff erent approach than the count-based n -grams described above. 11 4.1 Model

501 views • 8 slides

Chapter 7 Language models Statistical Machine Translation Language models Language models

Chapter 7 Language models Statistical Machine Translation Language models Language models answer the question: How likely is a string of English words good English? Help with reordering p lm (the house is small) > p lm (small the is

1.06k views • 51 slides

Sequence-to-sequence Models and Attention Graham Neubig Preliminaries: Language Models

Sequence-to-sequence Models and Attention Graham Neubig Preliminaries: Language Models Language Models Language models are generative models of text s ~ P(x) The Malfoys! said Hermione. Harry was watching him. He looked like

1.18k views • 37 slides

CSEP 517 Natural Language Processing Language Models Luke Zettlemoyer Slides adapted from Dan

CSEP 517 Natural Language Processing Language Models Luke Zettlemoyer Slides adapted from Dan Klein, Michael Collins, Yejin Choi, Dan Jurafsky Overview The language modeling problem N-gram language models Evaluation: perplexity

530 views • 50 slides

Introduction to Information Retrieval http://informationretrieval.org IIR 12: Language Models for

Statistical language models Statistical language models in IR Discussion Introduction to Information Retrieval http://informationretrieval.org IIR 12: Language Models for IR Hinrich Sch utze Institute for Natural Language Processing,

1.69k views • 141 slides

3 Language Models 1: n -gram Language Models While the final goal of a probabilistic machine

3 Language Models 1: n -gram Language Models While the final goal of a probabilistic machine translation system is to create a model of the target sentence E given the source sentence F , P ( E | F ), in this chapter we will take a step back, and

458 views • 8 slides

3 Language Models 1: n -gram Language Models While the final goal of a statistical machine

3 Language Models 1: n -gram Language Models While the final goal of a statistical machine translation system is to create a model of the target sentence E given the source sentence F , P ( E | F ), in this chapter we will take a step back, and

106 views • 8 slides

Maxent Models (III), & Neural Language Models CMSC 473/673 UMBC September 25 th , 2017 Some

Maxent Models (III), & Neural Language Models CMSC 473/673 UMBC September 25 th , 2017 Some slides adapted from 3SLP Recap from last time Maximum Entropy Models a more general language model argmax ) ()

1.46k views • 66 slides

CSE 490 Natural Language Processing Spring 2016 Language Models Yejin Choi Slides adapted from

CSE 490 Natural Language Processing Spring 2016 Language Models Yejin Choi Slides adapted from Dan Klein, Michael Collins, Luke Zettlemoyer, Dan Jurafsky Overview The language modeling problem N-gram language models Evaluation:

991 views • 68 slides

CSE 447/547 Natural Language Processing Winter 2020 Language Models Yejin Choi Slides adapted

CSE 447/547 Natural Language Processing Winter 2020 Language Models Yejin Choi Slides adapted from Dan Klein, Michael Collins, Luke Zettlemoyer, Dan Jurafsky Overview The language modeling problem N-gram language models Evaluation:

1.14k views • 65 slides

Language Models Dan Klein, John DeNero UC Berkeley Language Models Language Models Acoustic

Language Models Dan Klein, John DeNero UC Berkeley Language Models Language Models Acoustic Confusions the station signs are in deep in english -14732 the stations signs are in deep in english -14735 the station signs are in deep into

1.44k views • 86 slides

Language Models Language Models Dan Klein, John DeNero UC Berkeley Language Models Acoustic

Language Models Language Models Dan Klein, John DeNero UC Berkeley Language Models Acoustic Confusions the station signs are in deep in english -14732 the stations signs are in deep in english -14735 the station signs are in deep into

757 views • 22 slides

Testable Implications of General Equilibrium Models: An Integer Programming Approach Laurens

Introduction Testable implications on the eq manifold Computational complexity IP Application Conclusion Testable Implications of General Equilibrium Models: An Integer Programming Approach Laurens Cherchye Thomas Demuynck Bram De Rock

905 views • 75 slides

Introduction So far: Point-wise classification (geometric models) Whats next: Structured

University of Oslo : Department of Informatics INF4820: Algorithms for Artificial Intelligence and Natural Language Processing Probabilities and Language Models Stephan Oepen & Milen Kouylekov Language Technology Group (LTG) October 15,

556 views • 27 slides

Computational Models of Language Learning Jelle Zuidema Institute for Logic, Language and

Computational Models of Language Learning Jelle Zuidema Institute for Logic, Language and Computation, U. of Amsterdam MSc Brain & Cognitive Science, Artificial Intelligence, Logic MoL Guest Lecture Computational Models of Language

945 views • 58 slides

III.4 Statistical Language Models 1. Basics of Statistical Language Models 2. Query-Likelihood

III.4 Statistical Language Models 1. Basics of Statistical Language Models 2. Query-Likelihood Approaches 3. Smoothing Methods 4. Divergence Approaches 5. Extensions Based on MRS Chapter 12 and

459 views • 45 slides

Language models Chapter 3 in Martin/Jurafsky Probabilistic Language Models Goal: assign a

10/17/19 Language models Chapter 3 in Martin/Jurafsky Probabilistic Language Models Goal: assign a probability to a sentence Machine Translation: Why? P(hi high h winds tonite) > P(la large winds tonite) Spell Correction

321 views • 11 slides

Implications Reading: EC 1.5 Peter J. Haas INFO 150 Fall Semester 2019 Lecture 4 1/ 19

Implications Reading: EC 1.5 Peter J. Haas INFO 150 Fall Semester 2019 Lecture 4 1/ 19 Implications Definition and Examples The Logic of Implications Negating Implications Contrapositives, Converses, and Inverses The Language of

251 views • 23 slides

Understanding Socio-emotional Needs of English Language Learners Families: Implications for

Understanding Socio-emotional Needs of English Language Learners Families: Implications for Parent Engagement Elizabeth M. Vera Loyola University Chicago Chicagoland Partnership for English Language Learners Diversity in Schools The

350 views • 32 slides

N-grams & Language ID If N-gram models represent language models, can we use N-gram

N-grams & Language ID If N-gram models represent language models, can we use N-gram models for N-grams Language Identification? For example, can we use it to differentiate between text in German, text in English, text in

381 views • 3 slides

Language Models: Evaluation & Neural Models CMSC 470 Marine Carpuat Slides credit: Jurasky

Language Models: Evaluation & Neural Models CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin Language Models What you should know What is a language model A probability model that assigns probabilities to sequences of

449 views • 25 slides

Proficiency attainment of K-16 language students: Implications for language programs Boston

Proficiency attainment of K-16 language students: Implications for language programs Boston University November 11, 2019 Fernando Rubio, University of Utah ACTFL levels of proficiency ACTFL levels of proficiency NOVICE Communicate minimally

1.15k views • 76 slides

Language Models Philipp Koehn 8 September 2020 Philipp Koehn Machine Translation: Language

Language Models Philipp Koehn 8 September 2020 Philipp Koehn Machine Translation: Language Models 8 September 2020 Language models 1 Language models answer the question: How likely is a string of English words good English? Help with

2.09k views • 44 slides