Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010 QUERYING AND MINING QUERYING AND MINING DATA STREAMS Elena Ikonomovska Jožef Stefan Institute – Department of Knowledge Technologies
Outline Definitions Datastream models Similarity measures Historical background Foundations Estimating the L 2 distance Estimating the Jaccard similarity: Min-Wise Hashing Key applications Maintaining statistics on streams Hot items Some advanced results (Appendix) Estimating rarity and similarity (the windowed model) Tight bounds for approximate histograms and cluster ‐ based summaries Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Data stream models: Time series model A stream is a vector / point in space / p p Items are arriving in order of their indices: { , , ,...} { , , ,...} x x x x x x x x 1 1 2 2 3 3 … coordinates of the vector 1 2 3 4 x 1 x 2 x 3 x 4 The value of the i-th item is the value of the i-th coordinate of the vector Distance (similarity) between two streams is the distance between the two points Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Data stream models: Turnstile model Each arriving item is an update to some component of g p p the vector: 1 2 3 4 1 2 3 4 (2, 4) ⇒ 10 5 24 12 10 9 24 12 (2, x 2 (2 x (5) ) (5) ) indicates the 5 th update to the 2 nd indicates the 5 -th update to the 2 -nd component of the vector (2) + x i value: x i = x i (1) + x i (3) … i i i i positive or negative update only nonnegative updates ⇒ cash register model Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
L p distances (p ≥ 0 ) (p ) p Stream 1 {x 1 ,x 2 ,x 3 ,…} & stream 2 {y 1 ,y 2 ,y 3 ,…} in {1,…,m} { 1 } { 1 } { } 2 3 2 3 p -y i p | 1/p L p = Σ i |x i L 0 distance (Hamming distance) ⇔ the number of L 0 distance (Hamming distance) the number of indices i such that x i ≠ y i A measure of dis(similarity) of two streams [CDI02] L ∞ = max i |x i - y i | 2 -y i 2 | 1/2 distance L 2 = Σ i |x i 2 )- for approximating self-join sizes L 2 norm (f 2 [AGM’99] Q = COUNT(R A R) |dom(A)| = m Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Basic requirements q Naïve approach: store the points/vectors in memory Naïve approach: store the points/vectors in memory and compute any distance/similarity measure or a statistic (norm, frequency moment) ( , q y ) Typically: Large quantities of data – single pass g q g p Memory is constrained – O(log m) Real-time answers – linear time algorithms O(n) g ( ) Allowed approximate answers ( ε , δ ) ε & δ are user-specified parameters p p Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Historical background g [AMS’96] approximate F 2 (inserts only) [AMS 96] approximate F 2 (inserts only) [AGM’99] approximate L 2 norm (inserts and deletes) [FKS’99] approximate L 1 distance [ ] pp 1 [Indyk’00] approximate L p distance for p � (0,2] p-stable distributions (Caushy is 1-stable, Gaussian is 2-stable ) [CDI’02] efficient approximation of L 0 distance Approximate distances on windowed streams [DGI’02] approximate L p distance [Datar-Muthukrishnan’02] approximate Jaccard similarity Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Estimating the L 2 distance [AGM’99] g [ ] 2 Data streams (x 1 , x 2 …, x n ) and (y 1 , y 2 … y n ) For each i = 1, 2, …n define a i.i.d. random variable X i P[X i = 1] = P[X i = -1] = 1/2 � E[X i ]=0 Base idea: Simply maintain Σ i=1,..,n X i ( x i - y i ) For some i, j and items (i, x i (j) ), (i, y i (j) ) : (j) is added and X i X i · x i (j) is subtracted · y i E[( Σ i=1,..,n X i (x i -y i )) 2 ] = 1 0 E[ Σ i=1 n X i i ( 2 (x i -y i ) 2 + Σ i ≠ j X i X j (x i -y i )(x j -y j ) ] = i y i ) j ( i y i )( j y j ) ] [ i=1,..,n i ≠ j i Σ i=1,..,n (x i -y i ) 2 The problem amounts to obtaining an unbiased estimate The problem amounts to obtaining an unbiased estimate Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Standard boosting technique g q Run the algorithm in parallel k= θ (1/ ε 2 ) times Run the algorithm in parallel k θ (1/ ε ) times Maintain sums Σ i=1,..,n X i ( x i - y i ) for k different random 1. assignments for the random var. X i,k i,k Take the average of their squares for a given run r 2. ⇒ v (r) (reduce the variance/error!) Chebyshev Repeat the procedure l = θ (log(1/ δ )) times X i,k,l 3. Output the median over {v (1) ,v (2) ,…,v (l) } Chernoff 4. Maintains nkl values in parallel for the random 5. variables Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Result The Chebyshev inequality + Chernoff: The Chebyshev inequality Chernoff: ⇒ this estimates the square of L 2 within (1± ε ) factor with probability > (1 - δ ) p y ( ) Random variables needed: nkl ! The random variables can be four-wise independent p This is enough so that Chebyshev still holds [AMS’96] pseudorandomly generated on the fly O(kl) = O(1/ ε 2 log(1/ δ )) words + a logarithmic-length array of seeds O(log m) Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Estimating the L p distance g p p -stable distributions [I’00] p [ ] D is a p-stable distribution if: For all real numbers a 1 , a 2 , …, a k If X 1 , X 2 ,…,X k are i.i.d. random var. drawn from D ⇒ Σ a X has the same distribution as X( Σ | a | p ) 1/p ⇒ Σ a i X i has the same distribution as X( Σ i | a i | p ) 1/p for random variable X with distribution D Cauchy distribution is 1-stable L 1 Gaussian distribution is 2-stable L 2 Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
The algorithm g z 1 , z 2 ,…z is the stream vector z 1 , z 2 ,…z n is the stream vector Again… run in parallel k= θ (1/ ε 2 log(1/ δ )) procedures & maintain sums Σ i z i X i for each run procedures & maintain sums Σ i z i X i for each run 1,…k The value of Σ i z i X i in the l -th run is Z ( l ) e va ue o e u s i i i Z (l) is a random variable itself Let D is p -stable: e s p s ab e: Z (l) = X (l) ( Σ i | z i | p ) 1/p for some random variable X (l) drawn from D for some random variable X drawn from D Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Estimating the L p distance cont. g p The output is: p (1/ γ ) median {|Z (1) |, |Z (2) |,…, |Z (k) |} where γ is the median of |X|, for X random variable distributed according to D D Chebyshev : This estimate is within a multiplicative factor (1 ± ε ) of the true norm with probability (1- δ ) ( ) p y ( ) Observation [CDI’02]: L p is a good approximation of the L 0 norm for p sufficiently small ll p= ε /log(m) where m is the maximum absolute value of any item in the stream Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
The Jaccard similarity S A ={a 1 ,a 2 ,..a n } S B ={b 1 ,b 2 ,…,b n } Let A (and B) denote the set of distinct elements |A ∩ B|/|AUB| = Jaccard similarity Example: (view sets as columns) m=6 A B item 1 0 1 |AUB|=5 item 2 1 0 1 1 1 1 simJ(A,B) = 2/5 = 0.4 simJ(A,B) 2/5 0.4 0 0 1 1 item 6 item 6 0 0 1 1 Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Signature idea g Represent the sets A and B by signatures Sig(A) and Represent the sets A and B by signatures Sig(A) and Sig(B) Compute the similarity over the signatures p y g E[simH(Sig(A),Sig(B))]=simJ(A,B) Simplest approach S p pp Sample the sets (rows) uniformly at random k times to get k-bit signature Sig ( instead of m bits ) Problems! Sparsity – sampling might miss important information Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Tool: Min-Wise Hashing π ‐ randomly chosen permutation over {1,…,m} y p { , , } For any subset A ⊆ [m] the min-hash of A is: h π (A) = min i ∊ A { π (i)} π ( ) i ∊ A { ( )} Index of the first row with value 1 random permutation of the rows One bit of the k-bit signature of A, Sig(A) O bi f h k bi i f A Si (A) When π is chosen uniformly at random from the set of all permutations on [m] for any two subsets A B of all permutations on [m] for any two subsets A,B of [m] then: Pr[h (A) = h (B)] = |A ∩ B|/|AUB| Pr[h π (A) h π (B)] |A ∩ B|/|AUB| Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Example p Consider the following permutations: for m=5 1 = (1 2 3 4 5) k=1 2 = (5 4 3 2 1) k=2 3 = (3 4 5 1 2) k=3 And the sets: A = {1,3,4} B = {1,2,5} The min-hash values are as follows: h 1 (A) = 1 h 1 (B) = 1 k=1 h 2 (A) = 4 h 2 (B) = 5 k=2 h (A) = 3 h 3 (A) = 3 h (B) = 5 h 3 (B) = 5 k=3 k=3 � the expectation of the fraction of permutations where min- hash values agree is simJ(A,B) Advanced School on Data Exchange, Integration, and Streams - Dagstuhl, November 2010
Recommend
More recommend
