Improved efficiency for covering codes matching the sphere- - - PowerPoint PPT Presentation

ISIT 2020

Improved efficiency for covering codes matching the sphere- covering bound

Aditya Potukuchi and Yihan Zhang

Introduction: Covering codes

Introduction: Covering codes

• A subset

is said to be a covering code with relative covering radius if for every , we have that

𝒟 ⊆ {0,1}n δ z ∈ {0,1}n min

c∈𝒟 d(c, z) ≤ δn

Introduction: Covering codes

• A subset

is said to be a covering code with relative covering radius if for every , we have that

𝒟 ⊆ {0,1}n δ z ∈ {0,1}n min

c∈𝒟 d(c, z) ≤ δn

Hamming distance

Introduction: Covering codes

• A subset

is said to be a covering code with relative covering radius if for every , we have that

𝒟 ⊆ {0,1}n δ z ∈ {0,1}n min

c∈𝒟 d(c, z) ≤ δn

• ``every point is close to some point in the code''

Hamming distance

Introduction: Covering codes

• A subset

is said to be a covering code with relative covering radius if for every , we have that

𝒟 ⊆ {0,1}n δ z ∈ {0,1}n min

c∈𝒟 d(c, z) ≤ δn

• ``every point is close to some point in the code''
• Play a role in rate distortion theory and source coding

Hamming distance

Introduction: Covering codes

• A subset

is said to be a covering code with relative covering radius if for every , we have that

𝒟 ⊆ {0,1}n δ z ∈ {0,1}n min

c∈𝒟 d(c, z) ≤ δn

• ``every point is close to some point in the code''
• Play a role in rate distortion theory and source coding
• Dual notion of the usual codes, interesting combinatorial objects

Hamming distance

Upper and lower bounds on size

Upper and lower bounds on size

• Sphere-covering bound: Every covering code of block length and (relative

covering) radius must have size at least

n δ

2n (

n ≤ δn)

Upper and lower bounds on size

• Sphere-covering bound: Every covering code of block length and (relative

covering) radius must have size at least

n δ

2n (

n ≤ δn)

Upper and lower bounds on size

• Sphere-covering bound: Every covering code of block length and (relative

covering) radius must have size at least

n δ

2n (

n ≤ δn)

Upper and lower bounds on size

• Sphere-covering bound: Every covering code of block length and (relative

covering) radius must have size at least

n δ

2n (

n ≤ δn)

• Existence: A random subset
• f size

is almost surely a covering code of radius

𝒟 ⊂ {0,1}n

100n ⋅ 2n (

n ≤ δn)

δ

Upper and lower bounds on rate

Upper and lower bounds on rate

• Sphere-covering bound: Any covering code of radius must have rate at

least

δ 1 − H(δ) + o(1)

Upper and lower bounds on rate

• Sphere-covering bound: Any covering code of radius must have rate at

least

δ 1 − H(δ) + o(1)

• Existence: There exist covering codes of radius and rate

δ 1 − H(δ) + o(1)

Upper and lower bounds on rate

• Sphere-covering bound: Any covering code of radius must have rate at

least

δ 1 − H(δ) + o(1)

• Existence: There exist covering codes of radius and rate

δ 1 − H(δ) + o(1)

• So, the optimal rate-radius tradeoff is well understood

Upper and lower bounds on rate

• Sphere-covering bound: Any covering code of radius must have rate at

least

δ 1 − H(δ) + o(1)

• Existence: There exist covering codes of radius and rate

δ 1 − H(δ) + o(1)

• So, the optimal rate-radius tradeoff is well understood
• Interested in constructing such codes

Concatenation preserves covering radius

Concatenation preserves covering radius

• For

, the concatenated code is defined as

𝒟1, 𝒟2 ⊆ {0,1}n 𝒟1 ⊕ 𝒟2 := {(c1, c2) | c1 ∈ 𝒟1, c2 ∈ 𝒟2}

Concatenation preserves covering radius

• For

, the concatenated code is defined as

𝒟1, 𝒟2 ⊆ {0,1}n 𝒟1 ⊕ 𝒟2 := {(c1, c2) | c1 ∈ 𝒟1, c2 ∈ 𝒟2}

• Fact: If

are covering codes of radius respectively, then has radius

𝒟1, 𝒟2 ⊆ {0,1}n δ1, δ2 𝒟1 ⊕ 𝒟2 δ1 + δ2 2

Concatenation preserves covering radius

• For

, the concatenated code is defined as

𝒟1, 𝒟2 ⊆ {0,1}n 𝒟1 ⊕ 𝒟2 := {(c1, c2) | c1 ∈ 𝒟1, c2 ∈ 𝒟2}

• Fact: If

are covering codes of radius respectively, then has radius

𝒟1, 𝒟2 ⊆ {0,1}n δ1, δ2 𝒟1 ⊕ 𝒟2 δ1 + δ2 2

• If

, concatenation preserves radius (and also rate)

δ1 = δ2

Concatenation preserves covering radius

• For

, the concatenated code is defined as

𝒟1, 𝒟2 ⊆ {0,1}n 𝒟1 ⊕ 𝒟2 := {(c1, c2) | c1 ∈ 𝒟1, c2 ∈ 𝒟2}

• Fact: If

are covering codes of radius respectively, then has radius

𝒟1, 𝒟2 ⊆ {0,1}n δ1, δ2 𝒟1 ⊕ 𝒟2 δ1 + δ2 2

• If

, concatenation preserves radius (and also rate)

δ1 = δ2

• Enough to construct codes of small block length and bootstrap

Linear covering codes

Linear covering codes

• Theorem [Blinovsky '90]: A random linear code of rate

is a covering code with radius with high probability ( ).

1 − H(δ) + O(1/n) δ 1 − o(1)

Linear covering codes

• Theorem [Blinovsky '90]: A random linear code of rate

is a covering code with radius with high probability ( ).

1 − H(δ) + O(1/n) δ 1 − o(1)

• Corollary (Folklore): For every

, the concatenation of all linear codes of block length and rate gives a code of rate and radius

ϵ > 0 n 1 − H(δ) + Θ(1/n) 1 − H(δ) + ϵ δ

Linear covering codes

• Theorem [Blinovsky '90]: A random linear code of rate

is a covering code with radius with high probability ( ).

1 − H(δ) + O(1/n) δ 1 − o(1)

• Corollary (Folklore): For every

, the concatenation of all linear codes of block length and rate gives a code of rate and radius

ϵ > 0 n 1 − H(δ) + Θ(1/n) 1 − H(δ) + ϵ δ

• Gives a construction of covering codes that is ``explicit''

An issue with the construction

An issue with the construction

• Concatenation of all linear codes of block length and rate

gives a code of rate and radius

n 1 − H(δ) + Θ(1/n) 1 − H(δ) + ϵ δ

An issue with the construction

• Concatenation of all linear codes of block length and rate

gives a code of rate and radius

n 1 − H(δ) + Θ(1/n) 1 − H(δ) + ϵ δ

• Concatenation of codes of the same rate preserves rate

An issue with the construction

• Concatenation of all linear codes of block length and rate

gives a code of rate and radius

n 1 − H(δ) + Θ(1/n) 1 − H(δ) + ϵ δ

• Concatenation of codes of the same rate preserves rate
• This gives guarantees for constructions as long as n = Ω(1/ϵ)

An issue with the construction

• Concatenation of all linear codes of block length and rate

gives a code of rate and radius

n 1 − H(δ) + Θ(1/n) 1 − H(δ) + ϵ δ

• Concatenation of codes of the same rate preserves rate
• This gives guarantees for constructions as long as n = Ω(1/ϵ)
• Suppose we wanted to construct codes of block length

, we need to concatenate codes so

N 2Ωδ(n2) N ≥ n ⋅ 2Ωδ(n2) = exp(1/ϵ2)

Main motivation

Main motivation

• What the actually have: To obtain codes of block length

and radius and rate , the previous construction requires that

N δ 1 − H(δ) + ϵ N ≥ exp(1/ϵ2)

Main motivation

• What the actually have: To obtain codes of block length

and radius and rate , the previous construction requires that

N δ 1 − H(δ) + ϵ N ≥ exp(1/ϵ2)

• We want a better dependence on

and (we call this ``efficiency'')

N ϵ

Main motivation

• What the actually have: To obtain codes of block length

and radius and rate , the previous construction requires that

N δ 1 − H(δ) + ϵ N ≥ exp(1/ϵ2)

• We want a better dependence on

and (we call this ``efficiency'')

N ϵ

• Open question: Obtain an explicit construction where N = poly(1/ϵ)

Main motivation

• What the actually have: To obtain codes of block length

and radius and rate , the previous construction requires that

N δ 1 − H(δ) + ϵ N ≥ exp(1/ϵ2)

• We want a better dependence on

and (we call this ``efficiency'')

N ϵ

• Open question: Obtain an explicit construction where N = poly(1/ϵ)
• We know that

is possible

N = 1/ϵ

Revisiting the previous construction

Revisiting the previous construction

• Why was the efficiency so bad?

Revisiting the previous construction

• Why was the efficiency so bad?
• Need to concatenate

codes of block length . We know that most of these have optimal rate-radius tradeoff

2Ωδ(n2) n

Revisiting the previous construction

• Why was the efficiency so bad?
• Need to concatenate

codes of block length . We know that most of these have optimal rate-radius tradeoff

2Ωδ(n2) n

• Can improve tradeoff if we could concatenate fewer codes, each of block

length , where we know most of these have optimal rate-radius tradeoff

n

Our main result

Our main result

• Main Theorem [informal]: For every

, there is a construction of a code of block length with rate and radius as long as

ϵ > 0 N 1 − H(δ) + ϵ δ N ≥ exp(1/ϵ log(1/ϵ))

Our main result

• Main Theorem [informal]: For every

, there is a construction of a code of block length with rate and radius as long as

ϵ > 0 N 1 − H(δ) + ϵ δ N ≥ exp(1/ϵ log(1/ϵ))

• Actually: For every

, there is a set of linear codes of block length and rate such that fraction of them have relative covering radius

δ > 0 2Oδ(n log n) n 1 − H(δ) + O(1/n) 1 − o(1) δ

Some more notation

Some more notation

• We will use the following (standard) notation for convenience (

)

{0,1}n ≡ 𝔾n

2

Some more notation

• We will use the following (standard) notation for convenience (

)

{0,1}n ≡ 𝔾n

2

• is the Hamming ball of radius centered at

Br r

Some more notation

• We will use the following (standard) notation for convenience (

)

{0,1}n ≡ 𝔾n

2

• is the Hamming ball of radius centered at

Br r

• For subsets

, denote

A, B ⊆ 𝔾n

2

A + B := {a + b | a ∈ A, b ∈ B}

Some more notation

• We will use the following (standard) notation for convenience (

)

{0,1}n ≡ 𝔾n

2

• is the Hamming ball of radius centered at

Br r

• For subsets

, denote

A, B ⊆ 𝔾n

2

A + B := {a + b | a ∈ A, b ∈ B}

• ``

is a covering code of radius '' `` ''

𝒟 ⊂ {0,1}n δ ≡ 𝒟 + Bδn = {0,1}n

Wozencraft Ensemble

Wozencraft Ensemble

• Identify

and via an isomorphism of the additive group

𝔾2n 𝔾n

2

Wozencraft Ensemble

• Identify

and via an isomorphism of the additive group

𝔾2n 𝔾n

2

• For

, define a linear map given by .

α ∈ 𝔾2n Mα : 𝔾2n → 𝔾2n Mα(x) = α ⋅ x

Wozencraft Ensemble

• Identify

and via an isomorphism of the additive group

𝔾2n 𝔾n

2

• For

, define a linear map given by .

α ∈ 𝔾2n Mα : 𝔾2n → 𝔾2n Mα(x) = α ⋅ x

• is also a linear map from

Mα 𝔾n

2 → 𝔾n 2

Wozencraft Ensemble

• Identify

and via an isomorphism of the additive group

𝔾2n 𝔾n

2

• For

, define a linear map given by .

α ∈ 𝔾2n Mα : 𝔾2n → 𝔾2n Mα(x) = α ⋅ x

• is also a linear map from

Mα 𝔾n

2 → 𝔾n 2

• The code whose generator matrix (also parity check matrix) comes from the

following distribution: is chosen uniformly, and the matrix is of the form:

α ∈ 𝔾2n

Wozencraft Ensemble

• Identify

and via an isomorphism of the additive group

𝔾2n 𝔾n

2

• For

, define a linear map given by .

α ∈ 𝔾2n Mα : 𝔾2n → 𝔾2n Mα(x) = α ⋅ x

• is also a linear map from

Mα 𝔾n

2 → 𝔾n 2

• The code whose generator matrix (also parity check matrix) comes from the

following distribution: is chosen uniformly, and the matrix is of the form:

α ∈ 𝔾2n

n × 2n

Wozencraft Ensemble

• Identify

and via an isomorphism of the additive group

𝔾2n 𝔾n

2

• For

, define a linear map given by .

α ∈ 𝔾2n Mα : 𝔾2n → 𝔾2n Mα(x) = α ⋅ x

• is also a linear map from

Mα 𝔾n

2 → 𝔾n 2

• The code whose generator matrix (also parity check matrix) comes from the

following distribution: is chosen uniformly, and the matrix is of the form:

α ∈ 𝔾2n

Mα In

n × 2n

Main lemma

Main lemma

• We will focus on rate

for simplicity. Easily extendable to other rates

= 1/2

Main lemma

• We will focus on rate

for simplicity. Easily extendable to other rates

= 1/2

• Define

, so

r := H−1(1/2)n + O(log n) |Br| ≫ 2n

Main lemma

• We will focus on rate

for simplicity. Easily extendable to other rates

= 1/2

• Define

, so

r := H−1(1/2)n + O(log n) |Br| ≫ 2n

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

Main lemma

• We will focus on rate

for simplicity. Easily extendable to other rates

= 1/2

• Define

, so

r := H−1(1/2)n + O(log n) |Br| ≫ 2n

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

number of points not covered by the code

Main lemma

• We will focus on rate

for simplicity. Easily extendable to other rates

= 1/2

• Define

, so

r := H−1(1/2)n + O(log n) |Br| ≫ 2n

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

Main lemma

• We will focus on rate

for simplicity. Easily extendable to other rates

= 1/2

• Define

, so

r := H−1(1/2)n + O(log n) |Br| ≫ 2n

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

• ``Very few points are not within the optimal covering radius of

w.h.p''

𝒟

Final construction

Final construction

• Consider the ensemble given by the following generator matrix: Let

be uniformly chosen

α ∈ 𝔾2n

Final construction

• Consider the ensemble given by the following generator matrix: Let

be uniformly chosen

α ∈ 𝔾2n

(n + O(log n)) × 2n

Final construction

• Consider the ensemble given by the following generator matrix: Let

be uniformly chosen

α ∈ 𝔾2n

Mα In

(n + O(log n)) × 2n

Final construction

• Consider the ensemble given by the following generator matrix: Let

be uniformly chosen

α ∈ 𝔾2n

Mα In

(n + O(log n)) × 2n

uniformly random entries

Final construction

• Consider the ensemble given by the following generator matrix: Let

be uniformly chosen

α ∈ 𝔾2n

Mα In

(n + O(log n)) × 2n

uniformly random entries

ways to choose this part

2n

Final construction

• Consider the ensemble given by the following generator matrix: Let

be uniformly chosen

α ∈ 𝔾2n

Mα In

(n + O(log n)) × 2n

uniformly random entries

ways to choose this part

2n

ways to choose this part

2O(n log n)

Final construction

• Consider the ensemble given by the following generator matrix: Let

be uniformly chosen

α ∈ 𝔾2n

Mα In

(n + O(log n)) × 2n

uniformly random entries

ML w.h.p. span of these rows cover most of with radius

⇒ {0,1}n r

ways to choose this part

2n

ways to choose this part

2O(n log n)

Final construction

• Consider the ensemble given by the following generator matrix: Let

be uniformly chosen

α ∈ 𝔾2n

Mα In

(n + O(log n)) × 2n

uniformly random entries

ML w.h.p. span of these rows cover most of with radius

⇒ {0,1}n r

Second part: w.h.p. the rest are covered by the span of these rows

ways to choose this part

2n

ways to choose this part

2O(n log n)

Proof ideas I: Main Lemma

Proof ideas I: Main Lemma

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

Proof ideas I: Main Lemma

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

• a ∈ 𝒟 + Br ⟺ (a + Br) ∩ 𝒟 ≠ ∅

Proof ideas I: Main Lemma

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

• a ∈ 𝒟 + Br ⟺ (a + Br) ∩ 𝒟 ≠ ∅
• Easy to see :

by our choice of

E[|(a + Br) ∩ 𝒟|] = 2−n ⋅ |Br| ≫ 1 r

Proof ideas I: Main Lemma

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

• a ∈ 𝒟 + Br ⟺ (a + Br) ∩ 𝒟 ≠ ∅
• Easy to see :

by our choice of

E[|(a + Br) ∩ 𝒟|] = 2−n ⋅ |Br| ≫ 1 r

Proof ideas I: Main Lemma

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

• a ∈ 𝒟 + Br ⟺ (a + Br) ∩ 𝒟 ≠ ∅
• Easy to see :

by our choice of

E[|(a + Br) ∩ 𝒟|] = 2−n ⋅ |Br| ≫ 1 r

• Use second moment method to show that

ℙ((a + Br) ∩ 𝒟 = ∅) ≪ 1/n

Proof ideas I: Main Lemma

• Main Lemma: Let

be a random code from the Wozencraft Ensemble. Then

𝒟 ℙ(|𝔾2n

2 ∖(𝒟 + Br)| ≥ (1/n)22n) ≤ 1/n2

• a ∈ 𝒟 + Br ⟺ (a + Br) ∩ 𝒟 ≠ ∅
• Easy to see :

by our choice of

E[|(a + Br) ∩ 𝒟|] = 2−n ⋅ |Br| ≫ 1 r

• Use second moment method to show that

ℙ((a + Br) ∩ 𝒟 = ∅) ≪ 1/n

• ....hard to execute

Proof ideas I: Main Lemma contd..

Proof ideas I: Main Lemma contd..

• depends on , e.g., if

is low, then is more likely to be nonempty

|(a + Br) ∩ 𝒟| a |a| |(a + Br) ∩ 𝒟|

Proof ideas I: Main Lemma contd..

• depends on , e.g., if

is low, then is more likely to be nonempty

|(a + Br) ∩ 𝒟| a |a| |(a + Br) ∩ 𝒟|

• Main idea: Let

be a random code from the Wozencraft Ensemble and be a uniform. Then

𝒟 b ∈ 𝔾2n

2

ℙ(|𝔾2n

2 ∖(𝒟 + Br + b)| ≥ (1/n)22n) ≤ 1/n2

Proof ideas I: Main Lemma contd..

• depends on , e.g., if

is low, then is more likely to be nonempty

|(a + Br) ∩ 𝒟| a |a| |(a + Br) ∩ 𝒟|

• Main idea: Let

be a random code from the Wozencraft Ensemble and be a uniform. Then

𝒟 b ∈ 𝔾2n

2

ℙ(|𝔾2n

2 ∖(𝒟 + Br + b)| ≥ (1/n)22n) ≤ 1/n2

Proof ideas I: Main Lemma contd..

• depends on , e.g., if

is low, then is more likely to be nonempty

|(a + Br) ∩ 𝒟| a |a| |(a + Br) ∩ 𝒟|

• Main idea: Let

be a random code from the Wozencraft Ensemble and be a uniform. Then

𝒟 b ∈ 𝔾2n

2

ℙ(|𝔾2n

2 ∖(𝒟 + Br + b)| ≥ (1/n)22n) ≤ 1/n2

• The above statement implies the Main Lemma

Proof ideas I: Main Lemma contd..

• depends on , e.g., if

is low, then is more likely to be nonempty

|(a + Br) ∩ 𝒟| a |a| |(a + Br) ∩ 𝒟|

• Main idea: Let

be a random code from the Wozencraft Ensemble and be a uniform. Then

𝒟 b ∈ 𝔾2n

2

ℙ(|𝔾2n

2 ∖(𝒟 + Br + b)| ≥ (1/n)22n) ≤ 1/n2

• The above statement implies the Main Lemma
• Random variable of interest:

, independent of

|(a + Br) ∩ 𝒟 + b| a

Proof ideas II: Second part

Proof ideas II: Second part

• Main point: Every subsequent vector added to the Wozencraft Ensemble

covers a lot of the remaining uncovered vertices on average.

Proof ideas II: Second part

• Main point: Every subsequent vector added to the Wozencraft Ensemble

covers a lot of the remaining uncovered vertices on average.

• Let

be points not covered with radius by the row-span of the matrix

Ui r

Mα In

(n + i) × 2n

uniformly random entries

Proof ideas II: Second part

• Main point: Every subsequent vector added to the Wozencraft Ensemble

covers a lot of the remaining uncovered vertices on average.

• Let

be points not covered with radius by the row-span of the matrix

Ui r

• Lemma: E[|Ui+1|||Ui|] = |Ui|2 ⋅ 2−2n

Mα In

(n + i) × 2n

uniformly random entries

Open problems

Open problems

• Construct covering codes with better efficiency (

)

N = poly(1/ϵ)

Open problems

• Construct covering codes with better efficiency (

)

N = poly(1/ϵ)

• What is the covering radius of Wozencraft Ensemble?

Open problems

• Construct covering codes with better efficiency (

)

N = poly(1/ϵ)

• What is the covering radius of Wozencraft Ensemble?
• Concretely: Let

be a random code from the Wozencraft Ensemble. Then is ?

𝒟 ℙ((𝒟 + Br+o(r)) = 𝔾n

2) = 1 − o(1)

Thank you