wit ith the generalized benford s s law
play

wit ith the Generalized Benford's 's Law Arnaud Soulet, Arnaud - PowerPoint PPT Presentation

Jun 01, 2023 β€’235 likes β€’530 views

Representativeness of Knowledge Bases wit ith the Generalized Benford's 's Law Arnaud Soulet, Arnaud Giacometti, Batrice Markhoff and Fabian M. Suchanek University of Tours Telecom ParisTech Reliability of f queries on Knowledge Bases

  1. Representativeness of Knowledge Bases wit ith the Generalized Benford's 's Law Arnaud Soulet, Arnaud Giacometti, BΓ©atrice Markhoff and Fabian M. Suchanek University of Tours Telecom ParisTech

  2. Reliability of f queries on Knowledge Bases statistical query [Auer et al., 2007] How many cities are small (<1k inhabitants) in France/Yemen? 2 ISWC 2018 - Monterey, CA

  3. Reliability of f queries on Knowledge Bases statistical query [Auer et al., 2007] Does Yemen really not have any small cities? 3 ISWC 2018 - Monterey, CA

  4. Reliability of f queries on Knowledge Bases statistical crowdsourcing query Voluntary bias [Callahan and Herring, 2011;Wagner et al., 2015] 4 ISWC 2018 - Monterey, CA

  5. Reliability of f queries on Knowledge Bases statistical crowdsourcing query Voluntary bias [Callahan and Herring, 2011;Wagner et al., 2015] We do not know the KB biases, but the statistics can give us a hint? 5 ISWC 2018 - Monterey, CA

  6. Missing facts Yemeni Sanaa Aden Taiz city: [Darari et al., 2016; Population: Galarraga et al., 2017; Lajus and Suchanek, 2018; missing 1,937,451 760,923 615,222 Razniewski et al., 2015; Razniewski et al., 2016] Several methods for estimating the completeness for facts 6 ISWC 2018 - Monterey, CA

  7. Missing facts β‰  Missing entities + + missing facts Yemeni Haid al- Sanaa Aden Taiz missing city: Jazil Population: missing missing 1,937,451 760,923 615,222 few Missing facts due to missing entities are ignored! 7 ISWC 2018 - Monterey, CA

  8. Completeness = #present facts / / (#present facts + + #mis issin ing facts) Assuming that 𝒧 βˆ— is an ideal KB (= correct + complete): 𝒧 βˆ— 𝒧 βˆ— 𝒧 1 𝒧 2 Small cities Small cities Big cities Big cities What is the best KB between 𝓛 𝟐 and 𝓛 πŸ‘ for statistical queries? 8 ISWC 2018 - Monterey, CA

  9. Completeness β‰  Representativeness Assuming that 𝒧 βˆ— is an ideal KB (= correct + complete): 𝒧 βˆ— 𝒧 βˆ— 𝒧 1 𝒧 2 Small cities Small cities Big cities Big cities More complete, less representative Less complete, more representative  οƒΌ Representativeness is more important than completeness for statistics! 9 ISWC 2018 - Monterey, CA

  10. Representativeness of f Knowledge Bases A KB 𝒧 is representative of 𝒧 βˆ— iff the distribution remains the same for all uniform-sampling invariant measures. 𝒧 … <1k inhab. β‰₯1k inhab. ∼ ∼ ∼ ∼ 𝒧 βˆ— … <1k inhab. β‰₯1k inhab . 10 ISWC 2018 - Monterey, CA

  11. Representativeness of f Knowledge Bases A KB 𝒧 is representative of 𝒧 βˆ— iff the distribution remains the same for all uniform-sampling invariant measures. 𝒧 … <1k inhab. β‰₯1k inhab. ∼ ∼ ∼ ∼ The ideal knowledge base 𝒧 βˆ— is unknown! 𝒧 βˆ— … <1k inhab. β‰₯1k inhab . Challenge: How to estimate the representativeness? 11 ISWC 2018 - Monterey, CA

  12. Example: population of f capitals Abidjan Bangkok Conakry Kingston Mogadishu Santiago Abuja Beijing Dakar Kinshasa Montevideo Seoul Accra Belgrade Damascus Kuala Lumpur Nairobi Sofia Addis Ababa Berlin Dhaka Lilongwe Niamey Taipei Algiers Bogota Doha Lima Ouagadougou Tashkent Amman Brasilia Erbil London Paris Tbilisi Ankara Brazzaville Freetown Luanda Phnom Penh Tegucigalpa Antananarivo Bucharest Havana Lusaka Prague Tokyo Ashgabat Budapest Islamabad Madrid Pyongyang Tripoli Bahawalpur Buenos Aires Jakarta Managua Quito Tunis Baku Cairo Kabul Maputo Riyadh Ulaanbaatar Bamako Caracas Khartoum Mexico City Sana'a Vienna 12 ISWC 2018 - Monterey, CA

  13. Example: population of f capitals 4 707 404 8 280 925 1 660 973 1 041 084 1 750 000 6 158 080 1 235 880 21 700 000 1 146 053 10 125 000 1 305 082 9 971 111 2 291 352 1 166 763 1 711 000 1 768 000 3 138 369 1 260 120 3 384 569 3 610 156 6 970 105 1 077 116 1 302 910 2 704 974 3 415 811 7 878 783 1 351 000 8 852 000 1 626 950 2 309 600 4 007 526 2 556 149 1 025 000 8 673 713 2 229 621 1 118 035 4 587 558 1 827 000 1 050 301 2 825 311 1 501 725 1 157 509 1 613 375 1 883 425 2 106 146 1 742 979 1 267 449 13 617 445 1 031 992 1 759 407 1 900 000 3 141 991 2 581 076 1 126 000 1 052 000 2 890 151 9 607 787 2 205 676 2 671 191 1 056 247 2 122 300 10 230 350 3 678 034 1 766 184 7 125 180 1 372 000 1 809 106 3 273 863 5 185 000 8 918 653 1 937 451 1 852 997 13 ISWC 2018 - Monterey, CA

  14. Example: population of f capitals 4 707 404 8 280 925 1 660 973 1 041 084 1 750 000 6 158 080 1 235 880 2 1 700 000 1 146 053 1 0 125 000 1 305 082 9 971 111 2 291 352 1 166 763 1 711 000 1 768 000 3 138 369 1 260 120 3 384 569 3 610 156 6 970 105 1 077 116 1 302 910 2 704 974 3 415 811 7 878 783 1 351 000 8 852 000 1 626 950 2 309 600 4 007 526 2 556 149 1 025 000 8 673 713 2 229 621 1 118 035 4 587 558 1 827 000 1 050 301 2 825 311 1 501 725 1 157 509 1 613 375 1 883 425 2 106 146 1 742 979 1 267 449 1 3 617 445 1 031 992 1 759 407 1 900 000 3 141 991 2 581 076 1 126 000 1 052 000 2 890 151 9 607 787 2 205 676 2 671 191 1 056 247 2 122 300 1 0 230 350 3 678 034 1 766 184 7 125 180 1 372 000 1 809 106 3 273 863 5 185 000 8 918 653 1 937 451 1 852 997 What is the distribution of the first significant digit of capital inhabitants? 14 ISWC 2018 - Monterey, CA

  15. Benford’s law Population of cities 0.30 0.20 Benford’s law 0.10 0.00 1 2 3 4 5 6 7 8 9 15 ISWC 2018 - Monterey, CA

  16. Benford’s law Population of cities Discharge of rivers Length of rivers 0.30 0.30 0.30 0.20 0.20 0.20 0.10 0.10 0.10 0.00 0.00 0.00 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 16 ISWC 2018 - Monterey, CA

  17. Benford’s law Population of cities Discharge of rivers Length of rivers 0.30 0.30 0.30 0.20 0.20 0.20 0.10 0.10 0.10 0.00 0.00 0.00 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 𝑄 𝑔𝑗𝑠𝑑𝑒 𝑒𝑗𝑕𝑗𝑒 π‘Œ = 𝑒 = log 1 + 1 𝑒 [Newcomb, 1881;Benford, 1938] 17 ISWC 2018 - Monterey, CA

  18. The Generalized Benford’s Law Population of cities Discharge of rivers Length of rivers 0.30 0.30 0.30 Ξ± β†’ 0 Ξ± β†’ 0 Ξ± β†’ 0 0.20 0.20 0.20 0.10 0.10 0.10 0.00 0.00 0.00 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 𝑄 𝑔𝑗𝑠𝑑𝑒 𝑒𝑗𝑕𝑗𝑒 π‘Œ = 𝑒 = 1 + 𝑒 𝛽 βˆ’ 𝑒 𝛽 10 𝛽 βˆ’ 1 [HΓΌrlimann, 2014] 18 ISWC 2018 - Monterey, CA

  19. The Generalized Benford’s Law Population of cities Discharge of rivers Length of rivers 0.30 0.30 0.30 Ξ± β†’ 0 Ξ± β†’ 0 Ξ± β†’ 0 0.20 0.20 0.20 0.10 0.10 0.10 0.00 0.00 0.00 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 0.75 0.75 0.75 Actors per movie Persons per birth place Out-degree of wikipedia pages Ξ± =-0.155 Ξ± =-0.149 0.50 0.50 0.50 Ξ± =-0.486 0.25 0.25 0.25 0.00 0.00 0.00 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 19 ISWC 2018 - Monterey, CA

  20. Key idea of f our method representativeness = compliance with the Generalized Benford’s Law #𝒒𝒔𝒇𝒕𝒇𝒐𝒖_π’ˆπ’ƒπ’…π’–π’• = #𝒒𝒔𝒇𝒕𝒇𝒐𝒖_π’ˆπ’ƒπ’…π’–π’•+#𝒏𝒋𝒕𝒕𝒋𝒐𝒉_π’ˆπ’ƒπ’…π’–π’•_π’ˆπ’‘π’”_π’…π’‘π’π’’π’Žπ’‹π’ƒπ’π’…π’‡ Population in France Population in Yemen DBpedia 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Representativeness = 97% Representativeness = 79% 20 ISWC 2018 - Monterey, CA

  21. Our method in superv rvised context xt facts of 𝑠 on 𝒧 distribution of the fsd facts of 𝑠 on 𝒧 βˆ— Benford’s law Using the known distribution of the first significant digit 21 ISWC 2018 - Monterey, CA

  22. Our method in superv rvised context xt facts of 𝑠 on 𝒧 distribution of the fsd 200 Population in Yemen 150 100 50 0 1 2 3 4 5 6 7 8 9 facts of 𝑠 on 𝒧 βˆ— 378 present facts 101 missing facts Benford’s law Representativeness: 101 πŸ’πŸ–πŸ— = πŸ’πŸ–πŸ— + 𝟐𝟏𝟐 = 79% 378 Computing the minimum number of facts for retrieving Benford’s law 22 ISWC 2018 - Monterey, CA

  23. Our method in unsuperv rvised context xt facts of 𝑠 on 𝒧 distribution of the fsd facts of 𝑠 on 𝒧 βˆ— GBL with Ξ± =0.12 ideal distribution is unknown! Learning the parameter Ξ± of the Generalized Benford’s Law 23 ISWC 2018 - Monterey, CA

  24. Our method in unsuperv rvised context xt facts of 𝑠 on 𝒧 distribution of the fsd 200 Population in Yemen 150 100 50 0 1 2 3 4 5 6 7 8 9 facts of 𝑠 on 𝒧 βˆ— 378 present facts 78 missing facts GBL with Ξ± =0.12 Representativeness: ideal distribution is 78 unknown! πŸ’πŸ–πŸ— = πŸ’πŸ–πŸ— + πŸ–πŸ— = 82% 378 Computing the minimum number of facts for retrieving Benford’s law 24 ISWC 2018 - Monterey, CA

  25. Experimental study  Evaluation protocol 1. Take a correct and complete relation as gold standard 2. Degrade the completeness by discarding facts 3. Approximate the representativeness  Gold standard: population in French cities according to govt statistics  Degradation:  Most-populated: remove the least populated cities  Least-populated: remove the most populated cities  Random: remove cities randomly 25 ISWC 2018 - Monterey, CA

Recommend

The amusing and excellent law of Benfords law Benford References Principles of Complex

The amusing and excellent law of Benfords law Benford References Principles of Complex

Benfords law The amusing and excellent law of Benfords law Benford References Principles of Complex Systems Course 300, Fall, 2008 Prof. Peter Dodds Department of Mathematics &amp; Statistics University of Vermont Licensed under the

337 views β€’ 8 slides
The amusing and excellent law of Benfords law Benford References Principles of Complex

The amusing and excellent law of Benfords law Benford References Principles of Complex

Benfords law The amusing and excellent law of Benfords law Benford References Principles of Complex Systems Course 300, Fall, 2008 Prof. Peter Dodds Department of Mathematics &amp; Statistics University of Vermont Licensed under the

549 views β€’ 17 slides
Suppor&amp;ng Social Music-Making Steve Benford Mixed Reality

Suppor&amp;ng Social Music-Making Steve Benford Mixed Reality

Suppor&amp;ng Social Music-Making Steve Benford Mixed Reality Laboratory The University of NoAngham Desert Rain Rider Spoke Breathless A

492 views β€’ 16 slides
The Classification of Generalized Riemann Derivatives Stefan Catoiu DePaul University, Chicago

The Classification of Generalized Riemann Derivatives Stefan Catoiu DePaul University, Chicago

Definition and basic properties Generalized vs. Ordinary Differentiation The Classification of Real Generalized Riemann Derivatives The Classification of Complex Generalized Riemann Derivatives The Classification of Generalized Riemann

768 views β€’ 48 slides
Generalized Contagion Generalized Model of Contagion Principles of Complex Systems References

Generalized Contagion Generalized Model of Contagion Principles of Complex Systems References

Generalized Contagion Generalized Contagion Generalized Model of Contagion Principles of Complex Systems References Course 300, Fall, 2008 Prof. Peter Dodds Department of Mathematics &amp; Statistics University of Vermont Licensed under

466 views β€’ 21 slides
Digit analysis using Benford's Law Bart Baesens Professor Data Science at KU Leuven DataCamp

Digit analysis using Benford's Law Bart Baesens Professor Data Science at KU Leuven DataCamp

DataCamp Fraud Detection in R FRAUD DETECTION IN R Digit analysis using Benford's Law Bart Baesens Professor Data Science at KU Leuven DataCamp Fraud Detection in R Introduction Take a newspaper at a random page and write down the first or

696 views β€’ 50 slides
This reduces to a generalized eigenvalue problem, i.e. to finding generalized eigenvectors of

This reduces to a generalized eigenvalue problem, i.e. to finding generalized eigenvectors of

This reduces to a generalized eigenvalue problem, i.e. to finding generalized eigenvectors of the following form, with the lowest eigenvalues: Ly Dy This technique is called Laplacian Eigenmaps since the matrix L is

464 views β€’ 27 slides
Generalized MPLS Signaling draft-ietf-mpls-generalized-signaling-05.txt

Generalized MPLS Signaling draft-ietf-mpls-generalized-signaling-05.txt

Generalized MPLS Signaling draft-ietf-mpls-generalized-signaling-05.txt draft-ietf-mpls-generalized-cr-ldp-04.txt draft-ietf-mpls-generalized-rsvp-te-04.txt Authors Authors Peter Ashwood-Smith (Nortel Networks Corp.) Ayan Banerjee

386 views β€’ 7 slides
Generalized Single Degree of Freedom Systems PVD, Generalized Parameters, Rayleigh Quotjent

Generalized Single Degree of Freedom Systems PVD, Generalized Parameters, Rayleigh Quotjent

Generalized Single Degree of Freedom Systems PVD, Generalized Parameters, Rayleigh Quotjent Giacomo Boffj htup://intranet.dica.polimi.it/people/boffjgiacomo Dipartjmento di Ingegneria Civile Ambientale e Territoriale Politecnico di Milano

1.33k views β€’ 70 slides
Generalized diffeomorphisms acting on generalized metrics Roberto Rubio Research seminar on

Generalized diffeomorphisms acting on generalized metrics Roberto Rubio Research seminar on

Generalized diffeomorphisms acting on generalized metrics Roberto Rubio Research seminar on differential geometry 4th May 2020 Joint work with Carl Tipler The Lie group of automorphisms of a Courant algebroid and the moduli space of

611 views β€’ 31 slides
This reduces to a generalized eigenvalue problem, i.e. to finding generalized eigenvectors of

This reduces to a generalized eigenvalue problem, i.e. to finding generalized eigenvectors of

This reduces to a generalized eigenvalue problem, i.e. to finding generalized eigenvectors of the following form, with the lowest eigenvalues: Ly Dy This technique is called Laplacian Eigenmaps since the matrix L is called

324 views β€’ 20 slides
Generalized Nonlinear Models gnm : a Package for Generalized Nonlinear Models Same form as

Generalized Nonlinear Models gnm : a Package for Generalized Nonlinear Models Same form as

Generalized Nonlinear Models gnm : a Package for Generalized Nonlinear Models Same form as generalized linear models: g ( E ( Y )) = g ( ) = ( x ; ) Var ( Y ) = f ( ) Heather Turner and David Firth except ( x ; ) can be

543 views β€’ 4 slides
Generalized Additive Models September 10, 2019 Generalized Additive Models September 10, 2019 1

Generalized Additive Models September 10, 2019 Generalized Additive Models September 10, 2019 1

Generalized Additive Models September 10, 2019 Generalized Additive Models September 10, 2019 1 / 43 Motto My nature is to be linear, and when Im not, I feel really proud of myself. Cynthia Weil a songwriter Generalized Additive

786 views β€’ 39 slides
Generalized Fermat principle and Zermelo navigation: a link between Lorentzian and Generalized

Generalized Fermat principle and Zermelo navigation: a link between Lorentzian and Generalized

Overview Introduction Finslerian and spacetime viewpoints Background: Riemann-Finsler, Lorentz Applications: Fermat and Zermelo Fermat vs Zermelo Generalized Fermat principle and Zermelo navigation: a link between Lorentzian and Generalized

1.04k views β€’ 103 slides
Generalized Additive Models David L Miller Overview What is a GAM? What is smoothing? How do

Generalized Additive Models David L Miller Overview What is a GAM? What is smoothing? How do

Generalized Additive Models David L Miller Overview What is a GAM? What is smoothing? How do GAMs work? ( Roughly ) Fitting and plotting simple models What is a GAM? Generalized Additive Models Generalized: many response distributions

977 views β€’ 68 slides
Generalized Order-Value Optimization Jos e Mario Mart nez www.ime.unicamp.br/ martinez

Generalized Order-Value Optimization Jos e Mario Mart nez www.ime.unicamp.br/ martinez

Generalized Order-Value Optimization Generalized Order-Value Optimization Jos e Mario Mart nez www.ime.unicamp.br/ martinez Department of Applied Mathematics, University of Campinas, Brazil August 2, 2011 Generalized Order-Value

704 views β€’ 38 slides
Semifinite Generalized Quadrangles G. Eric Moorhouse Department of Mathematics University of

Semifinite Generalized Quadrangles G. Eric Moorhouse Department of Mathematics University of

Semifinite Generalized Quadrangles G. Eric Moorhouse Department of Mathematics University of Wyoming RMAC Seminar10 October 2014 G. Eric Moorhouse Semifinite Generalized Quadrangles Generalized Quadrangles A generalized quadrangle (GQ) is

636 views β€’ 49 slides
Domination in generalized Domination in generalized Petersen graphs Petersen graphs Advisor:

Domination in generalized Domination in generalized Petersen graphs Petersen graphs Advisor:

Domination in generalized Domination in generalized Petersen graphs Petersen graphs Advisor: Sheng-Chyang Liaw Student: Yuan-Shin Li Introduction A vertex subset S of a graph G is a dominating set g p g if each vertex in V ( G ) S is

853 views β€’ 50 slides
The Generalized Theories of Evolution Why it is the Theory of Evolution that is Constantly

The Generalized Theories of Evolution Why it is the Theory of Evolution that is Constantly

The Generalized Theories of Evolution Why it is the Theory of Evolution that is Constantly Generalized? Ozan Altan Altinok The Generalized Theories of Evolution Outline - Introduction: Too quick generalizations? r: - Evolution, theories,

336 views β€’ 22 slides
Generalized Effective Reducibility Merlin Carl Europa-Universit at Flensburg Generalized

Generalized Effective Reducibility Merlin Carl Europa-Universit at Flensburg Generalized

Generalized Effective Reducibility Merlin Carl Europa-Universit at Flensburg Generalized Effective Reducibility Recursion Theory in Mathematics By the CTT , we can get negative results on computability and show that certain problems are not

1.63k views β€’ 149 slides
Generalized Buckley-Leverett system Wladimir Neves UFRJUniversidade Federal do Rio de Janeiro

Generalized Buckley-Leverett system Wladimir Neves UFRJUniversidade Federal do Rio de Janeiro

Introduction Non-linear porous-media theory Muskat problem Remarks on Muskat problem Generalized Buckley-Leverett system Statement of the Buckley-Leverett problem Existence of generalized solution Generalized Buckley-Leverett system Wladimir

613 views β€’ 42 slides
Homology of generalized generalized graph homology generalizing to configuration spaces

Homology of generalized generalized graph homology generalizing to configuration spaces

homology of configuration spaces Cooper the chromatic polynomial Homology of generalized generalized graph homology generalizing to configuration spaces simplicial complexes categorification Andrew Cooper joint work with R. Sazdanovi

680 views β€’ 52 slides
Generalized Weyl algebras and their global dimension V. V. Bavula 1 Generalized Weyl algebras

Generalized Weyl algebras and their global dimension V. V. Bavula 1 Generalized Weyl algebras

Generalized Weyl algebras and their global dimension V. V. Bavula 1 Generalized Weyl algebras Definition of the generalized Weyl algebras . Let D be a ring, = ( 1 , ..., n ) a set of commuting automorphisms of D , ( i j = j i

295 views β€’ 11 slides
Class 23: Generalized coordinates and Class 23: Generalized coordinates and constrains Euler

Class 23: Generalized coordinates and Class 23: Generalized coordinates and constrains Euler

Class 23: Generalized coordinates and Class 23: Generalized coordinates and constrains Euler Lagrange Equation Special case 1 d f f - f f 0 0 dx y' y If f(y, y, x) does not depend on y, (i.e. f(y,

143 views β€’ 10 slides

More recommend