On confidence sets for compressed sensing inference problems Richard Nickl joint work with: A. Carpentier, D. Gross, J. Eisert Paris, June 9th, 2015 Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 1 / 23
The setting Prototypical high-dimensional data We observe noisy inner products Y i = � X i , θ � + ε i , i = 1 , . . . , n ; The noise ε i is distributed i.i.d. N (0 , σ 2 ) , or sub-Gaussian (Bernoulli); with a known upper bound on the variance σ 2 > 0. “Vector” model “Matrix” model X i are sensing vectors in C p , and X i are d × d sensing matrices , and � a , b � = � p j =1 a ∗ � A , B � = tr ( A ∗ B ) . j b j . � . � is the Frobenius norm. � . � is the l 2 norm The number n of observations is small compared The number n of observations is small compared to dimension (= d 2 ), but: to dimension (= p ), but: • The vector θ ∈ C p is k -sparse, θ ∈ M ( k ). • The d × d matrix θ has low rank k , θ ∈ M ( k ). Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 2 / 23
Recovery rates in Compressed Sensing When the design X satisfies the restricted isometry property (RIP), one can use ℓ 1 -regularisation estimators ˜ θ to recover the true θ from a minimal number of measurements in a computationally efficient way. Minimax optimal performance over M ( k ) For instance Lasso or Dantzig selector achieve, with high probability, recovery rates in the norm � · � arising from �· , ·� of the form θ − θ � 2 � k log p � ˜ θ is k -sparse : ≡ r ( k ) n θ − θ � 2 � kd � ˜ θ has rank ≤ k : n ≡ r ( k ) . Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 3 / 23
Recovery rates in Compressed Sensing When the design X satisfies the restricted isometry property (RIP), one can use ℓ 1 -regularisation estimators ˜ θ to recover the true θ from a minimal number of measurements in a computationally efficient way. Minimax optimal performance over M ( k ) For instance Lasso or Dantzig selector achieve, with high probability, recovery rates in the norm � · � arising from �· , ·� of the form θ − θ � 2 � k log p � ˜ θ is k -sparse : ≡ r ( k ) n θ − θ � 2 � kd � ˜ θ has rank ≤ k : n ≡ r ( k ) . When X is formed of i.i.d. (sub-) Gaussian variables, the RIP holds. Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 3 / 23
Quantum state estimation via Compressed Sensing A quantum state describing an N -particle physical system is encoded in a d × d positive definite ‘density matrix’ θ of trace tr ( θ ) = 1. Here d = 2 N . Source = θ Measurement = X ^ ^ Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 4 / 23
Quantum state estimation via Compressed Sensing A quantum state describing an N -particle physical system is encoded in a d × d positive definite ‘density matrix’ θ of trace tr ( θ ) = 1. Here d = 2 N . Source = θ Measurement = X ^ ^ Quantum measurements have ‘average outcomes’ tr ( E i θ ) = � E i , θ � where the E i ’s are Pauli tensor matrices. Repeated experiments approximate these i = σ 2 can usually be expectations up to some noise ε i whose variance E ε 2 controlled experimentally. Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 4 / 23
Quantum state estimation via Compressed Sensing A quantum state describing an N -particle physical system is encoded in a d × d positive definite ‘density matrix’ θ of trace tr ( θ ) = 1. Here d = 2 N . Source = θ Measurement = X ^ ^ Quantum measurements have ‘average outcomes’ tr ( E i θ ) = � E i , θ � where the E i ’s are Pauli tensor matrices. Repeated experiments approximate these i = σ 2 can usually be expectations up to some noise ε i whose variance E ε 2 controlled experimentally. In experiments one attempts to prepare a pure quantum state: then θ ∈ M (1), at least approximately. Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 4 / 23
Random Pauli Design Liu (2011) proved that if the ( X i : i = 1 , . . . , n ) are random draws from { E 1 , . . . , E d 2 } , where the E i constitute the Pauli tensor product basis of M d ( C ), then the resulting design satisfies the matrix RIP. Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 5 / 23
Random Pauli Design Liu (2011) proved that if the ( X i : i = 1 , . . . , n ) are random draws from { E 1 , . . . , E d 2 } , where the E i constitute the Pauli tensor product basis of M d ( C ), then the resulting design satisfies the matrix RIP. Recovery of low rank quantum states θ ∈ M ( k ) is thus possible after n ≈ kd (log d ) γ ≡ m ( k ) ≪ d 2 measurements , a significant improvement over requiring d 2 basis coefficient measurements. See Gross (2011) and Gross et al. (2010, 2012). Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 5 / 23
Random Pauli Design Liu (2011) proved that if the ( X i : i = 1 , . . . , n ) are random draws from { E 1 , . . . , E d 2 } , where the E i constitute the Pauli tensor product basis of M d ( C ), then the resulting design satisfies the matrix RIP. Recovery of low rank quantum states θ ∈ M ( k ) is thus possible after n ≈ kd (log d ) γ ≡ m ( k ) ≪ d 2 measurements , a significant improvement over requiring d 2 basis coefficient measurements. See Gross (2011) and Gross et al. (2010, 2012). Likewise, in the vector model recovery is possible after n ≈ k log p ≡ m ( k ) measurements whenever θ is k -sparse. Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 5 / 23
Certificates (Active Learning) → For implementation one would like uncertainty quantification: given any ǫ > 0 we want a data driven stopping time ˆ n such that n − θ � 2 < ǫ, and whenever θ ∈ M ( k ) ⇒ ˆ n ≈ whp m ( k ) . ∀ θ : � ˜ θ ˆ precision M(k1) M(k0) ε = m(k 0 )/ 2 n ^ ε ^ = m(k 1 )/ 2 ε Time horizon n → Such ‘certificates’ are closely related to ‘adaptive confidence sets’ for θ . Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 6 / 23
Relation between adaptive confidence sets and certificates For any α > 0 we want the confidence set C n to cover θ , θ ∈M ( k ) P θ ( θ ∈ C n ) ≥ 1 − α − o (1) , inf [Honest Coverage] where M ( k ) , k ≤ d , is a ‘maximal’ model. Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 7 / 23
Relation between adaptive confidence sets and certificates For any α > 0 we want the confidence set C n to cover θ , θ ∈M ( k ) P θ ( θ ∈ C n ) ≥ 1 − α − o (1) , inf [Honest Coverage] where M ( k ) , k ≤ d , is a ‘maximal’ model. For any sub-model M ( k 0 ) , 1 ≤ k 0 ≤ k , we want the diameter of C n to satisfy E θ | C n | 2 � minimax rate r ( k 0 ) over M ( k 0 ) , sup [Optimal Diameter] θ ∈M ( k 0 ) Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 7 / 23
Relation between adaptive confidence sets and certificates For any α > 0 we want the confidence set C n to cover θ , θ ∈M ( k ) P θ ( θ ∈ C n ) ≥ 1 − α − o (1) , inf [Honest Coverage] where M ( k ) , k ≤ d , is a ‘maximal’ model. For any sub-model M ( k 0 ) , 1 ≤ k 0 ≤ k , we want the diameter of C n to satisfy E θ | C n | 2 � minimax rate r ( k 0 ) over M ( k 0 ) , sup [Optimal Diameter] θ ∈M ( k 0 ) Certificate ⇒ Confidence set: Given adaptive certificate, one can obtain adaptive confidence sets. Idea: solve for ǫ corresponding to ˆ n = n . Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 7 / 23
Relation between adaptive confidence sets and certificates For any α > 0 we want the confidence set C n to cover θ , θ ∈M ( k ) P θ ( θ ∈ C n ) ≥ 1 − α − o (1) , inf [Honest Coverage] where M ( k ) , k ≤ d , is a ‘maximal’ model. For any sub-model M ( k 0 ) , 1 ≤ k 0 ≤ k , we want the diameter of C n to satisfy E θ | C n | 2 � minimax rate r ( k 0 ) over M ( k 0 ) , sup [Optimal Diameter] θ ∈M ( k 0 ) Certificate ⇒ Confidence set: Given adaptive certificate, one can obtain adaptive confidence sets. Idea: solve for ǫ corresponding to ˆ n = n . Confidence set ⇒ Certificate : The converse is true if a confidence sets exist ‘sequentially’ in n , with coverage guaranteed for all relevant values of n . Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 7 / 23
Design Assumptions We now describe our results, which are proved under any of the following assumptions: The design X is (sub-) Gaussian and i.i.d. Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 8 / 23
Design Assumptions We now describe our results, which are proved under any of the following assumptions: The design X is (sub-) Gaussian and i.i.d. In the matrix model, we also allow for random Pauli design Remark : For Pauli design, we tacitly assume the ‘quantum shape constraint’ θ ∈ Θ + ≡ { u ∈ C d × d , u � 0 , tr ( u ) = 1 } . Richard Nickl (Univ. Cambridge) Confidence in Compressed Sensing Paris, June 2015 8 / 23
Recommend
More recommend
