estimating sizes of social networks via biased sampling
play

Estimating Sizes of Social Networks via Biased Sampling Liran - PowerPoint PPT Presentation

Dec 05, 2022 •203 likes •771 views

Estimating Sizes of Social Networks via Biased Sampling Liran Katzir, Edo Liberty, and Oren Somekh Yahoo! Labs, Haifa, Israel International World Wide Web Conference, 28th March - 1st April 2011, Hyderabad, India Yahoo! Labs: WWW2011 1 /

  1. Estimating Sizes of Social Networks via Biased Sampling Liran Katzir, Edo Liberty, and Oren Somekh Yahoo! Labs, Haifa, Israel International World Wide Web Conference, 28th March - 1st April 2011, Hyderabad, India Yahoo! Labs: WWW’2011 1 / 20

  2. Social Network size estimation Goal: Obtaining estimates for sizes of (sub)populations in social network. Why: Advertisement - estimate of market share. Business development - merger/acquisition or asset valuation. Yahoo! Labs: WWW’2011 2 / 20

  3. The Problem Difficulties: Social network have become pretty big: Facebook (650,000,000) Qzone (200,000,000) Twitter (175,000,000) ... No public API for population size queries. What is the total number of registered users? What is the number of registered (self-declared) 20–30 year olds living in New-York? Even if a public API is provided an independent estimate is needed. Exhaustive crawl is time/space/communication intensive and violates “politeness”. Yahoo! Labs: WWW’2011 3 / 20

  4. Population size estimation Population sizes can be estimated efficiently using the “birthday paradox”. The “birthday paradox”: Given r uniform samples from a set of n elements, the expected number of collisions is r ( r − 1) . 2 n A collision is a pair of identical samples. Example: Samples: X = ( d , b , b , a , b , e ). Total 3 collisions, ( x 2 , x 3 ), ( x 2 , x 5 ), and ( x 3 , x 5 ). Yahoo! Labs: WWW’2011 4 / 20

  5. Population size estimation Using the birthday paradox inversely: When observing C collisions the pouplation can be estimated by ⇒ n ≃ r 2 2 C If r = const · √ n this gives a rather good estimator. Similar to mark-and-recapture which counts collisions between two sample sets (but is essentially equivalent). Newer version of mark-and-recapture also handles non-uniform but a-priory known distributions [Chao, 1987]. Social network size estimation [Ye and Wu, 2010] Alas, we cannot sample users uniformly from most social networks... Yahoo! Labs: WWW’2011 5 / 20

  6. Uniform distribution on graphs Social networks can be viewed as an undirected graph which we can traverse using their public APIs. Special random walks can generate close to uniform sampling: 1 Bipartite Query-Web page graph [Bharat and Broder, 1998] [Bar-Yossef and Gurevich, 2007]. 2 Social network [Gjoka et al, 2010]. Uses only r = const √ n samples, but obtaining each sample might be hard. Yahoo! Labs: WWW’2011 6 / 20

  7. Graph size estimation It is possible to estimate the size of some graphs directly. 1 Estimate the size of a tree [Knuth, 1974]. 2 Estimate the size of a directed acyclic graph [Pitt, 1987]. We give an estimator for the size of undirected graphs (and sub graphs) which: 1 Counts collisions but uses the graph’s stationary distribution. (does not require a uniform sample) 2 Requires asymptotically less than √ n samples to converge. 3 Obtains samples efficiently. (provable small number of random walk steps.) Yahoo! Labs: WWW’2011 7 / 20

  8. Assumptions The graph can be traversed from nodes to neighboring nodes. We can perform a random walk the graph: start at any node In each step, proceed to one of the neighbors uniformly at random. Yahoo! Labs: WWW’2011 8 / 20

  9. Facts about random walks This random walk yields the stationary distribution. 1 The probability to get the i ’th node is d i D . 2 d i – i ’th node’s degree. 3 D = � n i =1 d i . taking a few steps/several walks ensures independence between two consecutive samples. Yahoo! Labs: WWW’2011 9 / 20

  10. Algorithm Outline 1 Sample r users using random walk. 2 C – the number of collisions. 3 Ψ 1 – the sum of the sampled nodes’ degrees. 4 Ψ − 1 – the sum of the inverse sampled nodes’ degrees. The estimated number of nodes: n = Ψ 1 Ψ − 1 ˆ . 2 C Yahoo! Labs: WWW’2011 10 / 20

  11. Example Sampling process: Sampled Nodes: Sampled Node Degree: C: Ψ 1 : Ψ − 1 : n : ˆ Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  12. Example Sampling process: Sampled Nodes: Sampled Node Degree: C: Ψ 1 : Ψ − 1 : n : ˆ Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  13. Example Sampling process: Sampled Nodes: Sampled Node Degree: C: Ψ 1 : Ψ − 1 : n : ˆ Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  14. Example Sampling process: Sampled Nodes: Sampled Node Degree: C: Ψ 1 : Ψ − 1 : n : ˆ Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  15. Example Sampling process: Sampled Nodes: Sampled Node Degree: C: Ψ 1 : Ψ − 1 : n : ˆ Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  16. Example Sampling process: Sampled Nodes: Sampled Node Degree: C: Ψ 1 : Ψ − 1 : n : ˆ Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  17. Example Sampling process: Sampled Nodes: d Sampled Node Degree: 3 C: 0 Ψ 1 : 3 Ψ − 1 : 1/3 n : ˆ – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  18. Example Sampling process: Sampled Nodes: d Sampled Node Degree: 3 C: 0 Ψ 1 : 3 Ψ − 1 : 1/3 n : ˆ – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  19. Example Sampling process: Sampled Nodes: d Sampled Node Degree: 3 C: 0 Ψ 1 : 3 Ψ − 1 : 1/3 n : ˆ – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  20. Example Sampling process: Sampled Nodes: d Sampled Node Degree: 3 C: 0 Ψ 1 : 3 Ψ − 1 : 1/3 n : ˆ – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  21. Example Sampling process: Sampled Nodes: d Sampled Node Degree: 3 C: 0 Ψ 1 : 3 Ψ − 1 : 1/3 n : ˆ – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  22. Example Sampling process: Sampled Nodes: d Sampled Node Degree: 3 C: 0 Ψ 1 : 3 Ψ − 1 : 1/3 n : ˆ – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  23. Example Sampling process: Sampled Nodes: d f Sampled Node Degree: 3 2 C: 0 0 Ψ 1 : 3 5 Ψ − 1 : 1/3 5/6 n : ˆ – – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  24. Example Sampling process: Sampled Nodes: d f Sampled Node Degree: 3 2 C: 0 0 Ψ 1 : 3 5 Ψ − 1 : 1/3 5/6 n : ˆ – – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  25. Example Sampling process: Sampled Nodes: d f Sampled Node Degree: 3 2 C: 0 0 Ψ 1 : 3 5 Ψ − 1 : 1/3 5/6 n : ˆ – – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  26. Example Sampling process: Sampled Nodes: d f Sampled Node Degree: 3 2 C: 0 0 Ψ 1 : 3 5 Ψ − 1 : 1/3 5/6 n : ˆ – – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  27. Example Sampling process: Sampled Nodes: d f Sampled Node Degree: 3 2 C: 0 0 Ψ 1 : 3 5 Ψ − 1 : 1/3 5/6 n : ˆ – – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  28. Example Sampling process: Sampled Nodes: d f Sampled Node Degree: 3 2 C: 0 0 Ψ 1 : 3 5 Ψ − 1 : 1/3 5/6 n : ˆ – – Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  29. Example Sampling process: Sampled Nodes: d f f Sampled Node Degree: 3 2 2 C: 0 0 1 Ψ 1 : 3 5 7 Ψ − 1 : 1/3 5/6 16/12 n : ˆ – – 4 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  30. Example Sampling process: Sampled Nodes: d f f Sampled Node Degree: 3 2 2 C: 0 0 1 Ψ 1 : 3 5 7 Ψ − 1 : 1/3 5/6 16/12 n : ˆ – – 4 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  31. Example Sampling process: Sampled Nodes: d f f Sampled Node Degree: 3 2 2 C: 0 0 1 Ψ 1 : 3 5 7 Ψ − 1 : 1/3 5/6 16/12 n : ˆ – – 4 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  32. Example Sampling process: Sampled Nodes: d f f Sampled Node Degree: 3 2 2 C: 0 0 1 Ψ 1 : 3 5 7 Ψ − 1 : 1/3 5/6 16/12 n : ˆ – – 4 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  33. Example Sampling process: Sampled Nodes: d f f Sampled Node Degree: 3 2 2 C: 0 0 1 Ψ 1 : 3 5 7 Ψ − 1 : 1/3 5/6 16/12 n : ˆ – – 4 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  34. Example Sampling process: Sampled Nodes: d f f Sampled Node Degree: 3 2 2 C: 0 0 1 Ψ 1 : 3 5 7 Ψ − 1 : 1/3 5/6 16/12 n : ˆ – – 4 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  35. Example Sampling process: Sampled Nodes: d f f c Sampled Node Degree: 3 2 2 4 C: 0 0 1 1 Ψ 1 : 3 5 7 11 Ψ − 1 : 1/3 5/6 16/12 19/12 n : ˆ – – 4 8 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  36. Example Sampling process: Sampled Nodes: d f f c Sampled Node Degree: 3 2 2 4 C: 0 0 1 1 Ψ 1 : 3 5 7 11 Ψ − 1 : 1/3 5/6 16/12 19/12 n : ˆ – – 4 8 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  37. Example Sampling process: Sampled Nodes: d f f c Sampled Node Degree: 3 2 2 4 C: 0 0 1 1 Ψ 1 : 3 5 7 11 Ψ − 1 : 1/3 5/6 16/12 19/12 n : ˆ – – 4 8 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

  38. Example Sampling process: Sampled Nodes: d f f c Sampled Node Degree: 3 2 2 4 C: 0 0 1 1 Ψ 1 : 3 5 7 11 Ψ − 1 : 1/3 5/6 16/12 19/12 n : ˆ – – 4 8 Input social network graph: Yahoo! Labs: WWW’2011 11 / 20

Recommend

Estimating Properties of Social Networks via Random Walk considering Private Nodes Kazuki

Estimating Properties of Social Networks via Random Walk considering Private Nodes Kazuki

KDD 2020 Research track Estimating Properties of Social Networks via Random Walk considering Private Nodes Kazuki Nakajima Kazuyuki Shudo Tokyo Institute of Technology Graph Sampling on Social Networks Purpose: Understand global social

443 views • 21 slides
Bayesian Inference of a Finite Population Mean Under Length-Biased Sampling Zhiqing Xu, Balgobin

Bayesian Inference of a Finite Population Mean Under Length-Biased Sampling Zhiqing Xu, Balgobin

Bayesian Inference of a Finite Population Mean Under Length-Biased Sampling Zhiqing Xu, Balgobin Nandram and Binod Manandhar Estimating regrowth in a quarry 2 sets of three transects We count n from a finite population N Sampling

84 views • 7 slides
The Monte Carlo Method Estimating through sampling (estimating , p -value, integrals,...)

The Monte Carlo Method Estimating through sampling (estimating , p -value, integrals,...)

The Monte Carlo Method Estimating through sampling (estimating , p -value, integrals,...) The main difficulty - sampling sparse events The general sampling to counting reduction The Markov Chain Monte Carlo (MCMC) method -

1.39k views • 73 slides
Lecture 7 Barna Saha AT&T-Labs Research September 26, 2013 Outline Sampling Estimating F k

Lecture 7 Barna Saha AT&T-Labs Research September 26, 2013 Outline Sampling Estimating F k

Lecture 7 Barna Saha AT&T-Labs Research September 26, 2013 Outline Sampling Estimating F k [AMS96] Reservoir Sampling Priority Sampling Estimating F k Suppose, you know m , the stream length Sample a index p uniformly and

348 views • 15 slides
Sampling in networks Argimiro Arratia & R. Ferrer-i-Cancho Universitat Polit` ecnica de

Sampling in networks Argimiro Arratia & R. Ferrer-i-Cancho Universitat Polit` ecnica de

Sampling strategies Biases of sampling strategies Sampling in networks Argimiro Arratia & R. Ferrer-i-Cancho Universitat Polit` ecnica de Catalunya Complex and Social Networks (20 20 -202 1 ) Master in Innovation and Research in Informatics

1.18k views • 31 slides
Estimating and Sampling Graphs with Multidimensional Random Walks Group 2: Mingyan Zhao,

Estimating and Sampling Graphs with Multidimensional Random Walks Group 2: Mingyan Zhao,

Estimating and Sampling Graphs with Multidimensional Random Walks Group 2: Mingyan Zhao, Chengle Zhang, Biao Yin, Yuchen Liu Motivation Complex Network Social Network Biological Network eference: www.forbe.com imdevsoftware.wordpress.com

899 views • 34 slides
Sampling Representative Users from Large Social Networks Jie Tang, Chenhui Zhang Tsinghua

Sampling Representative Users from Large Social Networks Jie Tang, Chenhui Zhang Tsinghua

Sampling Representative Users from Large Social Networks Jie Tang, Chenhui Zhang Tsinghua University Keke Cai, Li Zhang, Zhong Su IBM, China Research Lab Download Code&Data here: http://aminer.org/repuser 1 Sampling Representative Users

409 views • 22 slides
Rejection Sampling Schemes for Extracting Uniform Distribution from Biased PUFs Rei Ueno , Kohei

Rejection Sampling Schemes for Extracting Uniform Distribution from Biased PUFs Rei Ueno , Kohei

Rejection Sampling Schemes for Extracting Uniform Distribution from Biased PUFs Rei Ueno , Kohei Kazumori, and Naofumi Homma Tohoku University Background PUF circuit One PUF cell response (Latch PUF) Example of PUF signal Set wafer

588 views • 16 slides
Finding Associations Similarity via Biased and Computing Pair Sampling General framework 2/35

Finding Associations Similarity via Biased and Computing Pair Sampling General framework 2/35

IEEE International Conference on Data Mining, Miami, Florida,USA 06-09/12/2009 Finding Associations Similarity via Biased and Computing Pair Sampling General framework 2/35 Introduction Running example: co-starring actors Part I Part

714 views • 44 slides
On Biased Reservoir Sampling in the Presence of Stream Evolution VLDB Conference, Seoul, South

On Biased Reservoir Sampling in the Presence of Stream Evolution VLDB Conference, Seoul, South

Charu C. Aggarwal T J Watson Research Center IBM Corporation Hawthorne, NY USA On Biased Reservoir Sampling in the Presence of Stream Evolution VLDB Conference, Seoul, South Korea, 2006 Synopsis Construction in Data Streams Synopsis

506 views • 39 slides
Extreme Event-Size Extreme Event-Size Fluctuations in Biased Fluctuations in Biased Random

Extreme Event-Size Extreme Event-Size Fluctuations in Biased Fluctuations in Biased Random

Extreme Event-Size Extreme Event-Size Fluctuations in Biased Fluctuations in Biased Random Walks on Networks Random Walks on Networks Vimal Kishore Physical Research Laboratory Ahmedabad, India phy.vimal@gmail.com vimal@prl.res.in Plan of

817 views • 25 slides
Sampling Online Social Networks Athina Markopoulou 1,3 Joint work with: Minas Gjoka 3 , Maciej

Sampling Online Social Networks Athina Markopoulou 1,3 Joint work with: Minas Gjoka 3 , Maciej

Sampling Online Social Networks Athina Markopoulou 1,3 Joint work with: Minas Gjoka 3 , Maciej Kurant 3 , Carter T. Butts 2,3 , Patrick Thiran 4 1 Department of Electrical Engineering and Computer Science 2 Department of Sociology 3 CalIT2:

990 views • 67 slides
Monte Carlo Methods Ensembles (Chapter 5) Biased Sampling (Chapter 14) Practical Aspects

Monte Carlo Methods Ensembles (Chapter 5) Biased Sampling (Chapter 14) Practical Aspects

Monte Carlo Methods Ensembles (Chapter 5) Biased Sampling (Chapter 14) Practical Aspects Lecture 1 2 Lecture 1&2 3 Lecture 1&3 4 Different Ensembles Ensemble Name Constant Fluctuating (Imposed) (Measured) NVT Canonical

2.72k views • 63 slides
Multitree Generalized Steppingstone Sampling A New MCMC Method for Estimating the Marginal

Multitree Generalized Steppingstone Sampling A New MCMC Method for Estimating the Marginal

Multitree Generalized Steppingstone Sampling A New MCMC Method for Estimating the Marginal Likelihood of a Models Mark T. Holder, Paul O. Lewis, David L. Swofford, and David Bryant KU, UConn, Duke, U. Otago (NZ) Feb 16, 2013 Austin,

625 views • 48 slides
BIASED PERCEPTIONS IN DIRECTED NETWORKS Nazanin Alipourfard, Buddhika Netasinghe, Andrs

BIASED PERCEPTIONS IN DIRECTED NETWORKS Nazanin Alipourfard, Buddhika Netasinghe, Andrs

BIASED PERCEPTIONS IN DIRECTED NETWORKS Nazanin Alipourfard, Buddhika Netasinghe, Andrs Abeliuk , Vikram Krishnamurthy, Kristina Lerman 1 THE MAJORITY ILLUSION - We see the world through our own personal lenses. - Local knowledge, can

585 views • 27 slides
Estimating Joint Default Probabilities by Efficient Importance Sampling Chuan-Hsiang Han Dept.

Estimating Joint Default Probabilities by Efficient Importance Sampling Chuan-Hsiang Han Dept.

Estimating Joint Default Probabilities by Efficient Importance Sampling Chuan-Hsiang Han Dept. of Quantitative Finance Natl. Tsing-Hua University, Taiwan WSAF, Hong Kong July 3, 2009 Outline Credit Risk Modeling: Classical Models

385 views • 37 slides
Querying Geo-social Data by Bridging Spatial Networks and Social Networks Yerach Ben Yaron

Querying Geo-social Data by Bridging Spatial Networks and Social Networks Yerach Ben Yaron

Querying Geo-social Data by Bridging Spatial Networks and Social Networks Yerach Ben Yaron Doytsher Galon Kanza 1 Motivation Social networks provide valuable information on social relationships among people (users) Associating

1.25k views • 46 slides
Introduction Social and Economic Networks MohammadAmin Fazli Social and Economic Networks 1

Introduction Social and Economic Networks MohammadAmin Fazli Social and Economic Networks 1

Introduction Social and Economic Networks MohammadAmin Fazli Social and Economic Networks 1 Why Study Networks Social networks permeate our social and economic lives They play a central role in the transmission of information, .

562 views • 35 slides
portable social networks with microformats social networks pownce.com ma.gnolia.com

portable social networks with microformats social networks pownce.com ma.gnolia.com

creating portable social networks with microformats social networks pownce.com ma.gnolia.com edenbee.com upcoming.yahoo.com last.fm twitter.com flickr.com Social network fatigue? More important than that: freedom of movement; reducing

404 views • 25 slides
Graphs and social networks Social networks Active area of research motivated in part by

Graphs and social networks Social networks Active area of research motivated in part by

csci 210: Data Structures Graphs and social networks Social networks Active area of research motivated in part by growing public fascination withe the complex connectedness of modern society Social networks have grown in

590 views • 11 slides
SOCIAL NETWORKS OF ELDERLY PEOPLE Hayden Manseau 1 1. THE PROBLEM 2 THE IMPACT OF SOCIAL

SOCIAL NETWORKS OF ELDERLY PEOPLE Hayden Manseau 1 1. THE PROBLEM 2 THE IMPACT OF SOCIAL

SOCIAL NETWORKS OF ELDERLY PEOPLE Hayden Manseau 1 1. THE PROBLEM 2 THE IMPACT OF SOCIAL NETWORKS IN ELDERLY POPULATION Social Network Elderly Social Networks Reduced Social Networks Is defined as a subjective Could be family members,

129 views • 11 slides
VHE Gamma-Ray Astronomy via Particle Detection at the Ground Level Or A biased review of VHE

VHE Gamma-Ray Astronomy via Particle Detection at the Ground Level Or A biased review of VHE

VHE Gamma-Ray Astronomy via Particle Detection at the Ground Level Or A biased review of VHE Gamm-Ray Survey Instruments Petra Huentemeyer petra@mtu.edu The Players: EAS Arrays Particle Sampling Gamma Rays Tibet AS- 1990-present

418 views • 27 slides
Adaptive Sampling for Sensor Networks Seminarvortrag von Philippe Bourquin Betreuung: Silvia

Adaptive Sampling for Sensor Networks Seminarvortrag von Philippe Bourquin Betreuung: Silvia

Adaptive Sampling for Sensor Networks Seminarvortrag von Philippe Bourquin Betreuung: Silvia Santini Overview Sensor Networks: an Introduction 1. Problems / Requirements in SNs 2. Adaptive Sampling 3. Example 1: Feedback Control

189 views • 8 slides
Existence & Emergence of Navigability in Social Networks Emmanuelle Lebhar (CNRS &

Existence & Emergence of Navigability in Social Networks Emmanuelle Lebhar (CNRS &

Existence & Emergence of Navigability in Social Networks Emmanuelle Lebhar (CNRS & CMM-U. de Chile) 1 1- Social networks 2 Social networks Networks of human beings interactions. High school dating Co-authorship in Physics 3

1.29k views • 125 slides

More recommend