recent advances in automated fact checking
play

Recent Advances in Automated Fact Checking Immanuel Trummer - PowerPoint PPT Presentation

Jul 13, 2023 •538 likes •1.5k views

Recent Advances in Automated Fact Checking Immanuel Trummer Cornell University Automation & Fact Checking Automation & Fact Checking Lorem Ipsum ... Identifying Lorem Ipsum ... Check-Worthy Claims

  1. Recent Advances 
 in Automated Fact Checking Immanuel Trummer 
 Cornell University

  2. Automation & 
 Fact Checking

  3. Automation & 
 Fact Checking Lorem 
 Ipsum ... Identifying 
 Lorem 
 Ipsum ... Check-Worthy 
 Claims Lorem 
 Ipsum ... Lorem 
 Ipsum ... Claim Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ...

  4. Automation & 
 Fact Checking Lorem 
 Lorem 
 Ipsum ... Ipsum ... Lorem 
 Identifying 
 Ipsum ... Lorem 
 Ipsum ... Check-Worthy 
 Claims Lorem 
 Lorem 
 Ipsum ... Lorem Ipsum ... Lorem 
 Ipsum ... Ipsum ... Claim Check Matching 
 Lorem 
 Claims to 
 Ipsum ... Lorem 
 Ipsum ... Checks Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Lorem 
 Ipsum ... Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ...

  5. Automation & 
 Fact Checking Lorem 
 Lorem 
 Ipsum ... Ipsum ... Lorem 
 Identifying 
 Ipsum ... Lorem 
 Ipsum ... Check-Worthy 
 Claims Lorem 
 Verification Lorem 
 Ipsum ... Lorem Ipsum ... Lorem 
 Ipsum ... Ipsum ... Claim Check Matching 
 Lorem 
 Claims to 
 Ipsum ... Lorem 
 Ipsum ... Checks Talk Focus Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Lorem 
 Ipsum ... Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ...

  6. Data-Driven Fact Checking Claim Data Formula Verified/Refuted

  7. Data-Driven Fact Checking Claim Data Formula (aka. SQL Query ) Verified/Refuted

  8. Data-Driven Fact Checking Claim Data Formula Verified/Refuted

  9. Data-Driven Fact Checking (In Natural Language) Claim Data Formula Verified/Refuted

  10. Data-Driven Fact Checking (In Natural Language) Claim ? a t a d h c i h W Data Formula Verified/Refuted

  11. Data-Driven Fact Checking (In Natural Language) Claim ? a ? l a u t m a d r o h f c h i h c W i h W Data Formula Verified/Refuted

  12. International Energy Agency

  13. International Energy Agency Paris-based intergovernmental organization Established in 1974, 30 member countries Mission: serve statistics on energy sector

  14. International Energy Agency Paris-based intergovernmental organization Established in 1974, 30 member countries Mission: serve statistics on energy sector

  16. Claims Marked Up

  17. Claims Marked Up Hundreds of pages, Thousands of claims ...

  18. Claims Marked Up Verification Takes Weeks! Hundreds of pages, Thousands of claims ...

  19. Fact Checking @ IEA Claim Data Formula Verified/Refuted

  20. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Verified/Refuted

  21. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Electricity/Global/2017 
 Electricity/Global/2016 Verified/Refuted

  22. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Electricity/Global/2017 
 D17/D16=1.03 Electricity/Global/2016 Verified/Refuted

  23. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Electricity/Global/2017 
 D17/D16=1.03 Electricity/Global/2016 Verified/Refuted

  24. Fact Checking @ IEA In 2017, global electricity 
 demand grew by 3% ... Claim Data Formula Electricity/Global/2017 
 D17/D16=1.03 Electricity/Global/2016 Verified/Refuted

  25. The "Infodemic"

  26. Fighting the Infodemic

  27. Fighting the Infodemic

  28. Fighting the Infodemic

  29. Fighting the Infodemic France has 
 more cases than US Claim Data Formula CDC/Confirmed/France F>U CDC/Confirmed/US Verified/Refuted

  30. Fighting the Infodemic France has 
 more cases than US Claim Data Formula CDC/Confirmed/France F>U CDC/Confirmed/US Verified/Refuted

  31. Fighting the Infodemic France has 
 more cases than US Claim Data Formula CDC/Confirmed/France F>U CDC/Confirmed/US Verified/Refuted

  32. Other Use Cases • Data journalism • Business reports • Scientific papers • ...

  33. (Demo) Verifying text summaries of relational data sets. 
 SIGMOD 2019 
 S. Jo, I. Trummer, W. Yu, X. Wang, C. Yu, D. Liu, N. Mehta.

  34. Challenges

  35. Challenges Text-Data Inconsistency ... ... American ...

  36. Challenges Multi-Claim Sentences Text-Data Inconsistency ... ... American ...

  37. Challenges Context Multi-Claim Sentences Text-Data Inconsistency ... ... American ...

  38. Fully Automated Checking Claim Translation Formula Evaluation

  39. Fully Automated Checking Claim May go Wrong! Translation Formula Evaluation

  40. Semi-Automated Checking Claim Translation Probability Formula Evaluation

  41. Analyze Data Structure country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  42. Analyze Data Structure country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  43. Analyze Data Structure Germany Beer Spirit Keywords Serving Country USA ... country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  44. Analyze Data Structure United States Synonyms ... America ... U.S. Germany Beer Spirit Keywords Serving Country USA ... country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  45. Analyze Data Structure Matches United States Synonyms ... America ... U.S. Germany Beer Spirit Keywords Serving Country USA ... country beer_servings spirit_servings ... ... ... ... ... Germany 346 117 ... ... ... ... ... USA 249 158 ... ... ... ... ...

  46. Analyze Sentence Structure

  47. Analyze Sentence Structure

  48. Analyze Sentence Structure

  49. Analyze Sentence Structure

  50. Consider Text Structure Claim Sentence

  51. Consider Text Structure Paragraph Claim Sentence

  52. Consider Text Structure Section Paragraph Claim Sentence

  53. Consider Text Structure Chapter Section Paragraph Claim Sentence

  54. Consider Text Structure Chapter Section Paragraph Claim Sentence Integrate Surrounding Keywords

  55. Understand the Author Translate Text

  56. Understand the Author Translate Text Claim 
 Translation 
 Hypothesis Infer Topic

  57. Understand the Author Translate Text Document 
 Claim 
 Topic 
 Translation 
 Hypothesis Hypothesis Infer Topic

  58. (Demo) Verifying text summaries of relational data sets. 
 SIGMOD 2019 
 S. Jo, I. Trummer, W. Yu, X. Wang, C. Yu, D. Liu, N. Mehta.

  59. System Overview Lorem 
 Ipsum ... Data Analysis Text Analysis Topic Analysis

  60. Automated Accuracy 100 Correctness Chance 75 50 25 0 1 2 3 4 5 6 7 8 9 10 Nr. Proposed Formulas

  61. Automated Accuracy (Billions of possible formulas) 100 Correctness Chance 75 50 25 0 1 2 3 4 5 6 7 8 9 10 Nr. Proposed Formulas

  62. Automated Accuracy (Billions of possible formulas) 100 Correctness Chance 75 50 Want Auto-Suggestions 25 0 1 2 3 4 5 6 7 8 9 10 Nr. Proposed Formulas

  63. Automated Accuracy (Billions of possible formulas) 100 Need Human Feedback Correctness Chance 75 50 Want Auto-Suggestions 25 0 1 2 3 4 5 6 7 8 9 10 Nr. Proposed Formulas

  64. User Study Results With Tool Without Tool 1.6 Claims per Minute 1.2 6x Speedup! 0.8 0.4 0 Verification Method

  65. Mistakes Discovered 11% of claims were incorrect 7% average error of claim value Analyzed 50 articles from major data journalism venues

  66. Scaling It Up Lorem 
 Ipsum ... AggChecker

  67. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... AggChecker

  68. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... AggChecker

  69. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... AggChecker

  70. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... Scrutinizer

  71. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ...

  72. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ...

  73. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Ipsum ... Learning Data Mapper

  74. Scaling It Up Lorem 
 Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... Lorem 
 Ipsum ... ! k s A Ipsum ... Learning Data Mapper n r a e L

Recommend

Cure My FEVER: Building, Breaking, and Fixing Models for Fact-Checking Christopher Hidey Tuhin

Cure My FEVER: Building, Breaking, and Fixing Models for Fact-Checking Christopher Hidey Tuhin

Cure My FEVER: Building, Breaking, and Fixing Models for Fact-Checking Christopher Hidey Tuhin Chakrabarty Tariq Alhindi Siddharth Varia Kriste Krstovski Mona Diab Smaranda Muresan Automated Fact-checking and Related Tasks Source

382 views • 36 slides
Automated Reasoning LTL Model Checking Alan Bundy Automated Reasoning LTL Model Checking

Automated Reasoning LTL Model Checking Alan Bundy Automated Reasoning LTL Model Checking

Automated Reasoning LTL Model Checking Alan Bundy Automated Reasoning LTL Model Checking Lecture 9, page 1 Introduction So far we have looked at theorem proving Powerful, especially where good sets of rewrite rules or decision procedures

639 views • 26 slides
Recent Advances in Causality Checking Florian Leitner-Fischer University of

Recent Advances in Causality Checking Florian Leitner-Fischer University of

Recent Advances in Causality Checking Florian Leitner-Fischer University of Konstanz Department of Computer and Information Science Chair for Software Engineering software software engineering engineering Joint work

819 views • 26 slides
Recent Advances in Photonic Recent Advances in Photonic effect employing IP- based distributed

Recent Advances in Photonic Recent Advances in Photonic effect employing IP- based distributed

Development of Photonic Network Optical burst switching Next Generation Enhancement of Networks Dynamic (Adaptive) Optical Path Control statistical >10Tb/s multiplexing Photonic MPLS Network Recent Advances in Photonic Recent Advances

762 views • 8 slides
Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances in Biomolecular NMR Protonless NMR for the characterization of Unfolded proteins, Large protein assemblies, Paramagnetic systems I n cell

1.23k views • 70 slides
Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances in Biomolecular NMR Protonless NMR for the characterization of Unfolded proteins, Large protein assemblies, Paramagnetic systems I n cell

635 views • 42 slides
The Mobius Program Verification Environment Recent Advances in Extended Static Checking Joe

The Mobius Program Verification Environment Recent Advances in Extended Static Checking Joe

The Mobius Program Verification Environment Recent Advances in Extended Static Checking Joe Kiniry, University College Dublin Joe Kiniry, University College Dublin http://mobius.inria.fr/ http://mobius.inria.fr/ Contract n IST 015905

534 views • 21 slides
Automated Reasoning Model Checking with Spin (III) Alan Bundy Automated Reasoning Model

Automated Reasoning Model Checking with Spin (III) Alan Bundy Automated Reasoning Model

Automated Reasoning Model Checking with Spin (III) Alan Bundy Automated Reasoning Model Checking with Spin (III) Lecture 12, page 1 Outline of Verification Process Objective: form one big automata, which Spin can use for verification.

1.01k views • 25 slides
Automated Reasoning Model Checking with SPIN (II) Alan Bundy Automated Reasoning SPIN (II)

Automated Reasoning Model Checking with SPIN (II) Alan Bundy Automated Reasoning SPIN (II)

Automated Reasoning Model Checking with SPIN (II) Alan Bundy Automated Reasoning SPIN (II) page 1 Verifying Global Properties Assertions can be used to verify a property locally locally For example, place assert(MemReturned) at the

441 views • 25 slides
Automated Reasoning Model Checking with SPIN Alan Bundy Automated Reasoning SPIN Lecture 10,

Automated Reasoning Model Checking with SPIN Alan Bundy Automated Reasoning SPIN Lecture 10,

Automated Reasoning Model Checking with SPIN Alan Bundy Automated Reasoning SPIN Lecture 10, page 1 Model Checking in Practice System behaviour: what is possible in our model? We will use the language Promela to specify behaviour

341 views • 18 slides
Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me...

Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me...

Automated Program Repair Opportunities, Challenges, Advances Chris Timperley 1 About me... Research Interests: Postdoc Carnegie Mellon University, USA Automated Program Repair w/ Claire Le Goues Fault Localisation GI for

676 views • 39 slides
Confl flict and Development: Recent Advances and Future Agendas Professor Patricia Justino

Confl flict and Development: Recent Advances and Future Agendas Professor Patricia Justino

Confl flict and Development: Recent Advances and Future Agendas Professor Patricia Justino Institute of Development Studies, UK Where we are Three main advances in recent research on conflict and violence: shift from state-level to more

323 views • 9 slides
Recent Advances Aly Khawaja Outline STAR- CCM+: a complete simulation workflow Emphasis on

Recent Advances Aly Khawaja Outline STAR- CCM+: a complete simulation workflow Emphasis on

Meshing in STAR-CCM+: Recent Advances Aly Khawaja Outline STAR- CCM+: a complete simulation workflow Emphasis on pre-processing technology Recent advances in surface preparation and meshing Continue to enhance flexibility, control,

1.13k views • 30 slides
A Review of Fact-Checking, Fake News Detection and Argumentation Tariq Alhindi March 02, 2020

A Review of Fact-Checking, Fake News Detection and Argumentation Tariq Alhindi March 02, 2020

A Review of Fact-Checking, Fake News Detection and Argumentation Tariq Alhindi March 02, 2020 Outline 1. Introduction 2. Fact-Checking 3. Fake News Detection 4. Argumentation Outline 1. Introduction 2. Fact-Checking a. What

766 views • 72 slides
Ele lectio ions approach are you ready? Fact-checking for ed educators and fu future

Ele lectio ions approach are you ready? Fact-checking for ed educators and fu future

Ele lectio ions approach are you ready? Fact-checking for ed educators and fu future voters Mikko Salo, Kari Kivinen, Pirjo Sallinen & Valentina Uitto Brussels, 13.11.2018 Fight disinformation with media literacy event organized

298 views • 19 slides
Fact Checking Problem: Estimated that people will see more fake than real news by 2022

Fact Checking Problem: Estimated that people will see more fake than real news by 2022

Fact Checking Problem: Estimated that people will see more fake than real news by 2022 Data: Google Fact Check research dataset Input: Claim + Supporting evidence Output: True / Partly True / Not True Supporting Evidence

150 views • 3 slides
Fact-Checking 100% Renewable Cla laims Is Is Georgetown Really 100% Renewable and, , Even if

Fact-Checking 100% Renewable Cla laims Is Is Georgetown Really 100% Renewable and, , Even if

Fact-Checking 100% Renewable Cla laims Is Is Georgetown Really 100% Renewable and, , Even if if it it Were, Is Is it it Good Policy? Charles McConnell Laura Schepis Executive Director, Executive Director, Rice University Energy and

283 views • 15 slides
Recent advances in Mandelbrot martingales theory Julien Barral, Universit e Paris Nord

Recent advances in Mandelbrot martingales theory Julien Barral, Universit e Paris Nord

Recent advances in Mandelbrot martingales theory Julien Barral, Universit e Paris Nord Advances in Fractals and Related Topics, CUHK, September 2012 J. Barral Recent advances in Mandelbrot martingales theory Mandelbrot martingales Let = {

736 views • 54 slides
Recent Advances in Adaptive Sampling and Reconstruction for Monte

Recent Advances in Adaptive Sampling and Reconstruction for Monte

Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering M. Zwicker W. Jarosz J. Lehtinen B. Moon R. Ramamoorthi Univ. of Bern Disney Research Aalto

543 views • 13 slides
Recent Advances in Adap.ve Sampling and Reconstruc.on for Monte

Recent Advances in Adap.ve Sampling and Reconstruc.on for Monte

Recent Advances in Adap.ve Sampling and Reconstruc.on for Monte Carlo Rendering Deriva've Analysis Wojciech Jarosz wjarosz@disneyresearch.com Thursday, July 16, 15 Path tracing -

1.36k views • 99 slides
Recent Advances in Two-loop Superstrings Eric DHoker Institut des Hautes Etudes Scientifiques,

Recent Advances in Two-loop Superstrings Eric DHoker Institut des Hautes Etudes Scientifiques,

Recent Advances in Two-loop Superstrings Eric DHoker Institut des Hautes Etudes Scientifiques, 2014 May 6 Eric DHoker Recent Advances in Two-loop Superstrings Outline 1. Overview of two-loop superstring methods, including global issues;

627 views • 31 slides
An Interpretable Joint Graphical Model for Fact-Checking from Crowds An T. Nguyen 1 Aditya

An Interpretable Joint Graphical Model for Fact-Checking from Crowds An T. Nguyen 1 Aditya

An Interpretable Joint Graphical Model for Fact-Checking from Crowds An T. Nguyen 1 Aditya Kharosekar 1 Matthew Lease 1 Byron C. Wallace 2 1 University of Texas at Austin 2 Northeastern University 1 Problems Given a claim: Facebook Shut Down an

780 views • 41 slides
Advances in Automated Program Repair and a Call to Arms Westley Weimer University of Virginia

Advances in Automated Program Repair and a Call to Arms Westley Weimer University of Virginia

Advances in Automated Program Repair and a Call to Arms Westley Weimer University of Virginia Andreas Zeller, SSBSE Keynote 2011 Westley Weimer 2 For The Next Hour Automated Program Repair Historical Context Mistakes

1.16k views • 87 slides
Recent Advances in Adversarial Machine Learning Nicholas Carlini Google Research Recent

Recent Advances in Adversarial Machine Learning Nicholas Carlini Google Research Recent

Recent Advances in Adversarial Machine Learning Nicholas Carlini Google Research Recent Advances in Adversarial (Examples in) Machine Learning Nicholas Carlini Google Research The Year is 2014 Someone tells you they have a new algorithm to

1.52k views • 139 slides

More recommend