Random permutations and the two-parameter Poisson-Dirichlet - PowerPoint PPT Presentation

Random permutations and the two-parameter Poisson-Dirichlet distribution. Sasha Gnedin Queen Mary, University of London Sasha Gnedin Random permutations and the 2-parameter PD

Sasha Gnedin Random permutations and the 2-parameter PD

Sasha Gnedin Random permutations and the 2-parameter PD

Sasha Gnedin Random permutations and the 2-parameter PD

The Pitman-Yor definition • PD( α, θ ) is a probability law for a sequence of random frequencies � P ↓ = ( P 1 , P 2 , · · · ) , with P 1 > P 2 > · · · > 0 , P j = 1 , j obtained by arranging in decreasing order another sequence P = ( � � P 1 , � P 2 , · · · ) Sasha Gnedin Random permutations and the 2-parameter PD

The Pitman-Yor definition • PD( α, θ ) is a probability law for a sequence of random frequencies � P ↓ = ( P 1 , P 2 , · · · ) , with P 1 > P 2 > · · · > 0 , P j = 1 , j obtained by arranging in decreasing order another sequence P = ( � � P 1 , � P 2 , · · · ) j − 1 � � P j = W j (1 − W j ) , j = 1 , 2 , . . . , i =1 L where W i ’s independent, with W j = Beta (1 − α, θ + α j ) Sasha Gnedin Random permutations and the 2-parameter PD

Two algorithms for size-biased ordering • Conventional sampling without replacement algorithm: For p 1 , p 2 , . . . with s = � p j < ∞ , a size-biased pick ˜ p 1 := p J is defined by setting P ( J = j ) = p j / s . Removing J from N , resp. p J from p 1 , p 2 , · · · , and iterating the SB-picking yields a SBP of N , resp. of p 1 , p 2 , · · · Sasha Gnedin Random permutations and the 2-parameter PD

• Ranking algorithm to arrange p 1 , p 2 , · · · in SB order: k th iteration only deals with p 1 , · · · , p k . After 1 , · · · , k have been arranged as i 1 , · · · , i k with ( q 1 , · · · , q k ) := ( p i 1 , · · · , p i k ) the relative rank ρ k +1 of k + 1 is determined by moving k + 1 left-to-right through i 1 , · · · , i k until settling in position ρ k +1 = m ∈ { 1 , · · · , k + 1 } with odds p k +1 : ( q m + · · · + q k ) . The infinite SB order is defined by ρ 1 , ρ 2 , . . . Sasha Gnedin Random permutations and the 2-parameter PD

• Ranking algorithm to arrange p 1 , p 2 , · · · in SB order: k th iteration only deals with p 1 , · · · , p k . After 1 , · · · , k have been arranged as i 1 , · · · , i k with ( q 1 , · · · , q k ) := ( p i 1 , · · · , p i k ) the relative rank ρ k +1 of k + 1 is determined by moving k + 1 left-to-right through i 1 , · · · , i k until settling in position ρ k +1 = m ∈ { 1 , · · · , k + 1 } with odds p k +1 : ( q m + · · · + q k ) . The infinite SB order is defined by ρ 1 , ρ 2 , . . . • k steps yield 1 , · · · , k (resp. p 1 , · · · , p k ) in size-biased order, showing that the finite orders are consistent under restrictions (cf also P-Tran ’12). Sasha Gnedin Random permutations and the 2-parameter PD

• Ranking algorithm to arrange p 1 , p 2 , · · · in SB order: k th iteration only deals with p 1 , · · · , p k . After 1 , · · · , k have been arranged as i 1 , · · · , i k with ( q 1 , · · · , q k ) := ( p i 1 , · · · , p i k ) the relative rank ρ k +1 of k + 1 is determined by moving k + 1 left-to-right through i 1 , · · · , i k until settling in position ρ k +1 = m ∈ { 1 , · · · , k + 1 } with odds p k +1 : ( q m + · · · + q k ) . The infinite SB order is defined by ρ 1 , ρ 2 , . . . • k steps yield 1 , · · · , k (resp. p 1 , · · · , p k ) in size-biased order, showing that the finite orders are consistent under restrictions (cf also P-Tran ’12). • Works also if � p j = ∞ although in this case the SB order is not a well-order. Sasha Gnedin Random permutations and the 2-parameter PD

• Ranking algorithm to arrange p 1 , p 2 , · · · in SB order: k th iteration only deals with p 1 , · · · , p k . After 1 , · · · , k have been arranged as i 1 , · · · , i k with ( q 1 , · · · , q k ) := ( p i 1 , · · · , p i k ) the relative rank ρ k +1 of k + 1 is determined by moving k + 1 left-to-right through i 1 , · · · , i k until settling in position ρ k +1 = m ∈ { 1 , · · · , k + 1 } with odds p k +1 : ( q m + · · · + q k ) . The infinite SB order is defined by ρ 1 , ρ 2 , . . . • k steps yield 1 , · · · , k (resp. p 1 , · · · , p k ) in size-biased order, showing that the finite orders are consistent under restrictions (cf also P-Tran ’12). • Works also if � p j = ∞ although in this case the SB order is not a well-order. • When p 1 = p 2 = · · · we have the ranks ρ k independent, uniform on [ k ] := { 1 , · · · , k } , and the resulting order is the exchangeable infinite order (Aldous ’83), which restricts to [ k ] as uniformly distributed permutation. Sasha Gnedin Random permutations and the 2-parameter PD

Characterisation of PD by SBP • If � P 1 is independent of ( � P 2 , � P 3 , · · · ) / (1 − � P 1 ) then the stick-breaking factors Y j are independent and (excluding some trivial cases) P ↓ L = PD ( α, θ ) for some α, θ . – McCloskey ’65, P ’96, G-Haulk-P ’09 Sasha Gnedin Random permutations and the 2-parameter PD

The arrangement problem Ordered representations of PD involve • either an increasing jump process (random c.d.f.) ( F t , t ≥ 0), • or interval partition of [0 , 1] into components of [0 , 1] \ Z , for Z a random measure-0 closed set. Every such representation implies certain arrangement P ∗ of the frequencies P ↓ j ’s in accord with the natural ordering of jump-times, resp. component intervals. Sasha Gnedin Random permutations and the 2-parameter PD

The arrangement problem Ordered representations of PD involve • either an increasing jump process (random c.d.f.) ( F t , t ≥ 0), • or interval partition of [0 , 1] into components of [0 , 1] \ Z , for Z a random measure-0 closed set. Every such representation implies certain arrangement P ∗ of the frequencies P ↓ j ’s in accord with the natural ordering of jump-times, resp. component intervals. • The arrangement problem concerns features of this induced order P ∗ , characterization of PD and sub-families, as well as connection of P ∗ to the well-orders P ↓ and � P . Sasha Gnedin Random permutations and the 2-parameter PD

A combinatorial counterpart of the arrangement problem • Recall that � P j is the asymptotic frequency of the j th occupied table in the Dubins-Pitman Chinese Restaurant Sasha Gnedin Random permutations and the 2-parameter PD

A combinatorial counterpart of the arrangement problem • Recall that � P j is the asymptotic frequency of the j th occupied table in the Dubins-Pitman Chinese Restaurant Sasha Gnedin Random permutations and the 2-parameter PD

A combinatorial counterpart of the arrangement problem • Recall that � P j is the asymptotic frequency of the j th occupied table in the Dubins-Pitman Chinese Restaurant Sasha Gnedin Random permutations and the 2-parameter PD

A combinatorial counterpart of the arrangement problem • Recall that � P j is the asymptotic frequency of the j th occupied table in the Dubins-Pitman Chinese Restaurant When the occupancy numbers are n 1 , . . . , n k , ( n 1 + · · · + n k = n ) • sits at occupied table j with probability n j − α n + θ , • occupies a new table with probability θ + k α n + θ . Sasha Gnedin Random permutations and the 2-parameter PD

• Hence a n -sample from P ∗ has the structure of composition (ordered partition) Π ∗ n of integer n , with the CRP ‘table’ occupancy counts arranged in the corresponding order. The Π ∗ n ’s are consistent as n varies. Sasha Gnedin Random permutations and the 2-parameter PD

• Hence a n -sample from P ∗ has the structure of composition (ordered partition) Π ∗ n of integer n , with the CRP ‘table’ occupancy counts arranged in the corresponding order. The Π ∗ n ’s are consistent as n varies. Sasha Gnedin Random permutations and the 2-parameter PD

• Hence a n -sample from P ∗ has the structure of composition (ordered partition) Π ∗ n of integer n , with the CRP ‘table’ occupancy counts arranged in the corresponding order. The Π ∗ n ’s are consistent as n varies. Z , U 1 , · · · , U n • sample uniform[0,1] points U 1 , . . . , U n • scan the gaps in Z in the left-to-right order • record the sizes of clusters in each occupied gap Sasha Gnedin Random permutations and the 2-parameter PD

Subordinator ‘bridge’ representations of PD • For ( S t , t ≥ 0) a subordinator with S 0 = 0 and tilted by manipulating the distribution of ( T , S T ) F t = S t , 0 ≤ t ≤ T S T • depending on choice of subordinator (gamma, stable, generalized gamma) some restricted range of ( α, θ ) ∈ [0 , 1) × [0 , ∞ ) may be covered –McCloskey ’65, Kingman ’75, Perman-PY ’92, PY ’97, P ’03 Sasha Gnedin Random permutations and the 2-parameter PD