The Generalized Sieve Kernel The Algorithmic Ant and the Sandpile eo - PowerPoint PPT Presentation

Shortest Vector from Lattice Sieving: a Few Dimensions for Free 2 2 EUROCRYPT 2018 L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 25 / 59

Two classes of Algorithms for SVP The Shortest Vector Problem I : The basis B of an n -dimensional lattice L O: A shortest non-zero vector v ∈ L Algorithm Running time Memory n n / 2 e · 2 O ( n ) Enumeration poly( n ) Sieving 3 [2 . 292 n + o ( n ) , 2 . 415 n + o ( n ) ] [2 . 2075 n + o ( n ) , 2 . 292 n + o ( n ) ] The paradox In theory, Sieving is faster. In pratice it is quite a lot slower. 3 Given complexities are heuristic, heavily supported by experiments. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 26 / 59

Two classes of Algorithms for SVP The Shortest Vector Problem I : The basis B of an n -dimensional lattice L O: A shortest non-zero vector v ∈ L Algorithm Running time Memory n n / 2 e · 2 O ( n ) Enumeration poly( n ) Sieving 3 [2 . 292 n + o ( n ) , 2 . 415 n + o ( n ) ] [2 . 2075 n + o ( n ) , 2 . 292 n + o ( n ) ] The paradox In theory, Sieving is faster. In pratice it is quite a lot slower. 3 Given complexities are heuristic, heavily supported by experiments. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 26 / 59

Two classes of Algorithms for SVP The Shortest Vector Problem I : The basis B of an n -dimensional lattice L O: A shortest non-zero vector v ∈ L Algorithm Running time Memory n n / 2 e · 2 O ( n ) Enumeration poly( n ) Sieving 3 [2 . 292 n + o ( n ) , 2 . 415 n + o ( n ) ] [2 . 2075 n + o ( n ) , 2 . 292 n + o ( n ) ] The paradox In theory, Sieving is faster. In pratice it is quite a lot slower. 3 Given complexities are heuristic, heavily supported by experiments. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 26 / 59

Many trade-offs 2 0.45 n 8 0 0 ' 1 V ' N V M 1 1 ' 2 0.40 n B 3 4 T 1 1 L ' ' H W J P G Z B Time complexity Laa '15 ◮ Our main contribution can also 2 0.35 n LdW '15 / BL '15 be applied to other sieving BDGL16 algorithms. e c BGJ '15 a p 2 0.30 n ◮ Implementation limited to the S = e version of m i T [Micciancio Voulgaris 2010] . 2 0.25 n 2 0.20 n 2 0.25 n 2 0.30 n 2 0.35 n Space complexity L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 27 / 59

Many trade-offs 2 0.45 n 8 0 In this 0 ' 1 V ' N V work M 1 1 ' 2 0.40 n B 3 4 T 1 1 L ' ' H W J P G Z B Time complexity Laa '15 ◮ Our main contribution can also 2 0.35 n LdW '15 / BL '15 be applied to other sieving BDGL16 algorithms. e c BGJ '15 a p 2 0.30 n ◮ Implementation limited to the S = e version of m i T [Micciancio Voulgaris 2010] . 2 0.25 n 2 0.20 n 2 0.25 n 2 0.30 n 2 0.35 n Space complexity L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 27 / 59

Results Heuristic claim, asymptotic One can solve SVP in dimension n with a call to Sieve in dimension n − d where d = Θ( n / log n ) . Heuristic claim, concrete One can solve SVP in dimension n making a call to Sieve in dimension i for each i = 2 . . . n − d for d ≈ n · ln(4 / 3) ( d ≈ 15 for n = 80) ln( n / 2 π e ) Experimental claim: A bogey A Sieve implem. almost on par with enumeration (within a factor 4 in dims 70–80), still with room for many improvements. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 28 / 59

Results Heuristic claim, asymptotic One can solve SVP in dimension n with a call to Sieve in dimension n − d where d = Θ( n / log n ) . Heuristic claim, concrete One can solve SVP in dimension n making a call to Sieve in dimension i for each i = 2 . . . n − d for d ≈ n · ln(4 / 3) ( d ≈ 15 for n = 80) ln( n / 2 π e ) Experimental claim: A bogey A Sieve implem. almost on par with enumeration (within a factor 4 in dims 70–80), still with room for many improvements. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 28 / 59

Results Heuristic claim, asymptotic One can solve SVP in dimension n with a call to Sieve in dimension n − d where d = Θ( n / log n ) . Heuristic claim, concrete One can solve SVP in dimension n making a call to Sieve in dimension i for each i = 2 . . . n − d for d ≈ n · ln(4 / 3) ( d ≈ 15 for n = 80) ln( n / 2 π e ) Experimental claim: A bogey A Sieve implem. almost on par with enumeration (within a factor 4 in dims 70–80), still with room for many improvements. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 28 / 59

Sieving Algorithm 1 Sieve ( L ) L ← a set of N random vectors from L where N ≈ (4 / 3) n / 2 . while ∃ ( v , w ) ∈ L 2 such that � v − w � < � v � do v ← v − w end while return L The above runs in heuristic time (4 / 3) n + o ( n ) . Many concrete and asymptotic improvements: [Nguyen Vidick 2008, Micciancio Voulgaris 2010, Laarhoven 2015, Becker Gamma Joux 2015, Becker D. Gamma Laarhoven 2015, . . . ] . L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 29 / 59

Sieving Algorithm 2 Sieve ( L ) L ← a set of N random vectors from L where N ≈ (4 / 3) n / 2 . while ∃ ( v , w ) ∈ L 2 such that � v − w � < � v � do v ← v − w end while return L The above runs in heuristic time (4 / 3) n + o ( n ) . Many concrete and asymptotic improvements: [Nguyen Vidick 2008, Micciancio Voulgaris 2010, Laarhoven 2015, Becker Gamma Joux 2015, Becker D. Gamma Laarhoven 2015, . . . ] . L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 29 / 59

More than SVP Note that Sieve returns N ≈ (4 / 3) n short vectors, not just a shortest vector. Definition (Gaussian Heuristic: Expected length of the shortest vector) � n / 2 π e · vol( L ) 1 / n . gh( L ) = Observation (heuristic & experimental) � The output of Sieve contains almost all vectors of length ≤ 4 / 3 · gh ( L ): � � � L := Sieve ( L ) = x ∈ L s.t. � x � ≤ 4 / 3 · gh( L ) . L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 30 / 59

Sieve then Lift Main idea: Sieve in a projected sub-lattice, and lift all candidate solutions. SubSieve ( L , d ) ◮ Set L ′ = L ( b 1 , . . . , b d ) “left part of L ”, dim= d ◮ Set L ′′ = π ⊥ L ′ ( L ) “right part of L ”, dim= n − d ◮ Compute L = Sieve ( L ′′ ) ◮ Hope that π ⊥ L ′ ( s ) ∈ L (1) ◮ Lift all v ∈ L from L ′′ to L and take the shortest (Babai alg.) Pessimistic prediction for (1) Optimistic prediction for (1) � n − d � � 4 / 3 · gh( L ′′ ) . 4 / 3 · gh( L ′′ ) . · gh( L ) ≤ gh( L ) ≤ n Similar to linear pruning for enum. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 31 / 59

Sieve then Lift Main idea: Sieve in a projected sub-lattice, and lift all candidate solutions. SubSieve ( L , d ) ◮ Set L ′ = L ( b 1 , . . . , b d ) “left part of L ”, dim= d ◮ Set L ′′ = π ⊥ L ′ ( L ) “right part of L ”, dim= n − d ◮ Compute L = Sieve ( L ′′ ) ◮ Hope that π ⊥ L ′ ( s ) ∈ L (1) ◮ Lift all v ∈ L from L ′′ to L and take the shortest (Babai alg.) Pessimistic prediction for (1) Optimistic prediction for (1) � n − d � � 4 / 3 · gh( L ′′ ) . 4 / 3 · gh( L ′′ ) . · gh( L ) ≤ gh( L ) ≤ n Similar to linear pruning for enum. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 31 / 59

Sieve then Lift Main idea: Sieve in a projected sub-lattice, and lift all candidate solutions. SubSieve ( L , d ) ◮ Set L ′ = L ( b 1 , . . . , b d ) “left part of L ”, dim= d ◮ Set L ′′ = π ⊥ L ′ ( L ) “right part of L ”, dim= n − d ◮ Compute L = Sieve ( L ′′ ) ◮ Hope that π ⊥ L ′ ( s ) ∈ L (1) ◮ Lift all v ∈ L from L ′′ to L and take the shortest (Babai alg.) Pessimistic prediction for (1) Optimistic prediction for (1) � n − d � � 4 / 3 · gh( L ′′ ) . 4 / 3 · gh( L ′′ ) . · gh( L ) ≤ gh( L ) ≤ n Similar to linear pruning for enum. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 31 / 59

Sieve then Lift Main idea: Sieve in a projected sub-lattice, and lift all candidate solutions. SubSieve ( L , d ) ◮ Set L ′ = L ( b 1 , . . . , b d ) “left part of L ”, dim= d ◮ Set L ′′ = π ⊥ L ′ ( L ) “right part of L ”, dim= n − d ◮ Compute L = Sieve ( L ′′ ) ◮ Hope that π ⊥ L ′ ( s ) ∈ L (1) ◮ Lift all v ∈ L from L ′′ to L and take the shortest (Babai alg.) Pessimistic prediction for (1) Optimistic prediction for (1) � n − d � � 4 / 3 · gh( L ′′ ) . 4 / 3 · gh( L ′′ ) . · gh( L ) ≤ gh( L ) ≤ n Similar to linear pruning for enum. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 31 / 59

With BKZ pre-processing ◮ To ensure (1), we need the basis to be as reduced as possible ◮ We can easily afford BKZ preprocessing with block-size b = n / 2 ◮ Using simple BKZ models 4 we can predict gh( L ) and gh( L ′ ) Heuristic claim SubSieve ( L , d ) algorithm will successfully find the shortest vector of L for some d = Θ( n / ln n ). ⇒ Improve time & memory by a sub-exponential factor 2 Θ( n / log n ) 4 The Geometric Series Assumption L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 32 / 59

With BKZ pre-processing ◮ To ensure (1), we need the basis to be as reduced as possible ◮ We can easily afford BKZ preprocessing with block-size b = n / 2 ◮ Using simple BKZ models 4 we can predict gh( L ) and gh( L ′ ) Heuristic claim SubSieve ( L , d ) algorithm will successfully find the shortest vector of L for some d = Θ( n / ln n ). ⇒ Improve time & memory by a sub-exponential factor 2 Θ( n / log n ) 4 The Geometric Series Assumption L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 32 / 59

Quasi-HKZ preprocessing Idea: Attempt stronger pre-processing. Algorithm 3 SubSieve + ( L , d ) L ← Sieve ( L ′′ ) L = { Lift L ′′ →L ( v ) for v ∈ L } for j = 0 . . . n / 2 − 1 do v j = arg min s ∈ L � π ( v 0 ... v j − 1 ) ⊥ ( s ) � end for return ( v 0 . . . v n / 2 − 1 ) ◮ Insert ( v 0 . . . v n / 2 − 1 ) as the new b 1 . . . b n / 2 ◮ Repeat SubSieve + ( L , d ) for d = n − 1 , n − 2 , . . . , d min ◮ Hope that iteration d min + 1 provided a quasi-HKZ basis. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 33 / 59

Concrete prediction with quasi-HKZ preprocessing Pessimistic prediction for (1) Optimistic prediction for (1) d ≈ n ln 4 / 3 n ln 4 / 3 d ≈ ln( n / 2 π ) ln( n / 2 π e ) d 25 pessimistic simulation pessimistic approximation optimistic simulation optimistic approximation 20 15 10 5 250 n 50 100 150 200 Figure: Predictions of the maximal successful choice of d min . L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 34 / 59

Baseline Implementation (V0) Re-implemented GaussSieve [Micciancio Voulgaris 2010] ◮ No gaussian sampling ◮ Initial sphericity of L doesn’t seem to matter ◮ Initial vectors can be made much shorter ⇒ speed-up ◮ Prevent collisions using a hash table √ ◮ Terminate when the ball 4 / 3 · gh( L ) is half-saturated ◮ Sort only periodically ◮ Can use faster data-structures ◮ Vectors represented in bases B and GramSchmidt ( B ) ◮ Required to work in projected-sublattices ◮ Kernel in c++ , control in python ◮ Calls to fpylll to maintain B and GramSchmidt ( B ) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 35 / 59

Baseline Implementation (V0) Re-implemented GaussSieve [Micciancio Voulgaris 2010] ◮ No gaussian sampling ◮ Initial sphericity of L doesn’t seem to matter ◮ Initial vectors can be made much shorter ⇒ speed-up ◮ Prevent collisions using a hash table √ ◮ Terminate when the ball 4 / 3 · gh( L ) is half-saturated ◮ Sort only periodically ◮ Can use faster data-structures ◮ Vectors represented in bases B and GramSchmidt ( B ) ◮ Required to work in projected-sublattices ◮ Kernel in c++ , control in python ◮ Calls to fpylll to maintain B and GramSchmidt ( B ) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 35 / 59

Baseline Implementation (V0) Re-implemented GaussSieve [Micciancio Voulgaris 2010] ◮ No gaussian sampling ◮ Initial sphericity of L doesn’t seem to matter ◮ Initial vectors can be made much shorter ⇒ speed-up ◮ Prevent collisions using a hash table √ ◮ Terminate when the ball 4 / 3 · gh( L ) is half-saturated ◮ Sort only periodically ◮ Can use faster data-structures ◮ Vectors represented in bases B and GramSchmidt ( B ) ◮ Required to work in projected-sublattices ◮ Kernel in c++ , control in python ◮ Calls to fpylll to maintain B and GramSchmidt ( B ) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 35 / 59

Baseline Implementation (V0) Re-implemented GaussSieve [Micciancio Voulgaris 2010] ◮ No gaussian sampling ◮ Initial sphericity of L doesn’t seem to matter ◮ Initial vectors can be made much shorter ⇒ speed-up ◮ Prevent collisions using a hash table √ ◮ Terminate when the ball 4 / 3 · gh( L ) is half-saturated ◮ Sort only periodically ◮ Can use faster data-structures ◮ Vectors represented in bases B and GramSchmidt ( B ) ◮ Required to work in projected-sublattices ◮ Kernel in c++ , control in python ◮ Calls to fpylll to maintain B and GramSchmidt ( B ) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 35 / 59

Baseline Implementation (V0) Re-implemented GaussSieve [Micciancio Voulgaris 2010] ◮ No gaussian sampling ◮ Initial sphericity of L doesn’t seem to matter ◮ Initial vectors can be made much shorter ⇒ speed-up ◮ Prevent collisions using a hash table √ ◮ Terminate when the ball 4 / 3 · gh( L ) is half-saturated ◮ Sort only periodically ◮ Can use faster data-structures ◮ Vectors represented in bases B and GramSchmidt ( B ) ◮ Required to work in projected-sublattices ◮ Kernel in c++ , control in python ◮ Calls to fpylll to maintain B and GramSchmidt ( B ) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 35 / 59

XOR-POPCNT trick (V0 → V1) Already used in Sieving [Fitzpatrick et al. 2015] . More generally know as SimHash [Charikar 2002] . Idea: Pre-filter pairs ( v , w ) ∈ L with a fast compressed test. ◮ Choose a spherical code C = { c 1 . . . c k } ⊂ S n and a threshold t ≤ k / 2 ◮ Precompute compressions ˜ v = Sign ( � v , c i � ) ∈ { 0 , 1 } k ◮ Only test � v ± w � ≤ � v � if | HammingWeight ( v ⊕ w ) − k / 2 | ≥ t . ◮ Asymptotic speed-up Θ( n / log n ) ? ◮ In practice, k = 128 (2 words), t = 18: about 10 cycles per pairs. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 36 / 59

Progressive Sieving (V1 → V2) Concurrently and independetly invented in [Mariano Laarhoven 2018] . Idea: Increase the dimension progressively. ◮ Recursively, Sieve in the lattice L ( b 1 , . . . b n − 1 ) ◮ Start the sieve in dimension n with many short-ish vectors ◮ Fresh vectors get reduced much faster thanks to this initial pool. Refer to [Mariano Laarhoven 2018] for a full analysis of this trick. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 37 / 59

Dimensions for Free (V2 → V3) ◮ Apply the quasi-HKZ preprocessing strategy ◮ Do not force the choice of d min ◮ Simply increase d until the shortest vector is found. d 19 pessimistic simulation pessimistic approximation 18 optimistic simulation optimistic approximation 17 Experimental average 16 15 14 13 12 11 10 9 8 7 6 n 60 62 64 66 68 70 72 74 76 78 80 82 Figure: Predictions experiments for d min . L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 38 / 59

Performances T ( sec. ) Fit V0: 2 0 . 489 n − 21 . 6 Fit V1: 2 0 . 505 n − 24 . 6 10 3 Fit V2: 2 0 . 470 n − 24 . 8 Fit V3: 2 0 . 396 n − 23 . 6 Fit Enum: 2 0 . 0683 n · ln n − 17 . 9 V0 (Sieve) V1 (Sieve) V2 (Sieve) V3 (SubSieve) 10 2 fplll's Pruned Enum. 10 1 10 0 n 40 50 60 70 80 L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 39 / 59

Comparison to other Sieving implementation Algorithms [FBB + 14] V0 V1 V2 V3 [MV10] [ML17] [HK17] Features XOR-POPCNT trick x x x x pogressive sieving x x SubSieve x LSH (more mem.) x tuple (less mem.) x Dimension Running times n = 60 227s 49s 8s 0 . 9s 464s 79s 13s 1080s n = 70 - - 276s 10s 23933s 4500s 250s 33000s n = 80 - - - 234s - - 4320s 94700s CPU freq. (GHz) 3 . 6 3 . 6 3 . 6 3 . 6 4 . 0 4 . 0 2 . 3 2 . 3 L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 40 / 59

Summary Sieving vs. Sieving ◮ Exploit all outputs of Sieve ⇒ Dimensions for Free ◮ Our implementation is 10x faster than all previous Sieving ◮ It does not use LSH techniques: further speed-up expected Sieving vs. Enumeration ◮ Only a factor 4x slower than Enum for dimensions 70–80 ◮ Guesstimates a cross-over at dim ≈ 90 with further improvements (LSH/LSF, fine-tuning, vectorization, . . . ) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 41 / 59

Summary Sieving vs. Sieving ◮ Exploit all outputs of Sieve ⇒ Dimensions for Free ◮ Our implementation is 10x faster than all previous Sieving ◮ It does not use LSH techniques: further speed-up expected Sieving vs. Enumeration ◮ Only a factor 4x slower than Enum for dimensions 70–80 ◮ Guesstimates a cross-over at dim ≈ 90 with further improvements (LSH/LSF, fine-tuning, vectorization, . . . ) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 41 / 59

The Generalized Sieve Kernel (G6K, pronounced / ζ e.si.ka/) 5 5 Work in Progress with M. Albrecht, E. Postlethwaite, G. Herold, E. Kirshanova, M. Stevens L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 44 / 59

1st design principle: Go Green ! Idea: Recycle vectors between overlapping blocks. Rather than an function serving as an SVP oracle, design a stateful machine that takes advantages of the overlapping instances. In other words:

1st design principle: Go Green ! Idea: Recycle vectors between overlapping blocks. Rather than an function serving as an SVP oracle, design a stateful machine that takes advantages of the overlapping instances. In other words:

Moving with a bag of vectors Relations between the projected sublattices: L = ⊃ ⊃ . . . ⊃ . . . L [1:1] L [1: n ] L [1: n − 1] L [1:2] ↓ π ↓ π ↓ π L [2: n ] ⊃ L [2: n − 1] ⊃ . . . L [2:2] . . . · L [ n − 1: n ] ⊃ L [ n − 1: n − 1] ↓ π L [ n : n ] ◮ π can be inverted in many ways. Choose π − 1 to be the Babai lift: the shortest of all possible lifts. ◮ All maps ⊂ , π − 1 , π preserve shortness “somewhat” L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 46 / 59

Moving with a bag of vectors Relations between the projected sublattices: L = ⊃ ⊃ . . . ⊃ . . . L [1:1] L [1: n ] L [1: n − 1] L [1:2] ↓ π ↓ π ↓ π L [2: n ] ⊃ L [2: n − 1] ⊃ . . . L [2:2] . . . · L [ n − 1: n ] ⊃ L [ n − 1: n − 1] ↓ π L [ n : n ] ◮ π can be inverted in many ways. Choose π − 1 to be the Babai lift: the shortest of all possible lifts. ◮ All maps ⊂ , π − 1 , π preserve shortness “somewhat” L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 46 / 59

Moving with a bag of vectors Relations between the projected sublattices: L = ⊃ ⊃ . . . ⊃ . . . L [1:1] L [1: n ] L [1: n − 1] L [1:2] ↓ π ↓ π ↓ π L [2: n ] ⊃ L [2: n − 1] ⊃ . . . L [2:2] . . . · L [ n − 1: n ] ⊃ L [ n − 1: n − 1] ↓ π L [ n : n ] ◮ π can be inverted in many ways. Choose π − 1 to be the Babai lift: the shortest of all possible lifts. ◮ All maps ⊂ , π − 1 , π preserve shortness “somewhat” L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 46 / 59

Moving with a bag of vectors L = ⊃ ⊃ . . . ⊃ L [1: n ] L [1: n − 1] L [1:2] L [1:1] ↓ π ↓ π ↓ π ⊃ ⊃ . . . L [2: n ] L [2: n − 1] L [2:2] . . . . . . · L [ n − 1: n ] ⊃ L [ n − 1: n − 1] ↓ π L [ n : n ] Change of context/block [l:r] : transform the vectors in the bag ◮ Extend-Right : ⊂ (do nothing) ◮ Shrink-Left : π − 1 (Babai lift) ◮ Extend-Left : π (project) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 47 / 59

2nd design principle: be flexible BKZ theory use exact-SVP for each block consecutively, but maybe we’re better off making different choices. ◮ Maintain a cadidate for insertion at each position ◮ Decide where to insert after sieving L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 48 / 59

3rd design principle: seize opportunities Algorithm 4 Sieve ( L ) L ← a set of N random vectors from L where N ≈ (4 / 3) n / 2 . while ∃ ( v , w ) ∈ L 2 such that � v − w � < � v � do v ← v − w end while return L Even if � v − w � ≥ � v � , it could be worth considering the lifts of v − w . L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 49 / 59

The abstract machine State: ◮ A lattice basis B ◮ Positions 0 ≤ ℓ ′ ≤ ℓ ≤ r ≤ d . [ ℓ : r ] the sieving context , and [ ℓ ′ : r ] the lifting context . ◮ A database db of N vectors in L [ ℓ : r ] (preferably short). ◮ Insertion candidates c ℓ ′ , . . . , c ℓ where c i ∈ L [ i : r ] or c i = ⊥ . Instructions: ◮ Sieve ( S ): make vector shorter, improve insertion candidates ◮ Extend Right, Shrink Left, Extend Left ( ER , SL , EL ): change the sieve-context, updating the database ◮ Insert ( I ): update the basis and the database L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 50 / 59

The ideal BKZ with G6K BKZ can be written very simply: Repeat { S; I; ER; } When starting the second Sieve , vectors are already quite short ⇒ No need to restart progressive sieving from the beginning. The ER bug. It turns out that ER is not very compatible with our fastest sieve implementation. Somehow, the Sieve gets stuck in a subspace. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 51 / 59

The ideal BKZ with G6K BKZ can be written very simply: Repeat { S; I; ER; } When starting the second Sieve , vectors are already quite short ⇒ No need to restart progressive sieving from the beginning. The ER bug. It turns out that ER is not very compatible with our fastest sieve implementation. Somehow, the Sieve gets stuck in a subspace. L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 51 / 59

Pump Before: SubSieve f : Reset 0 , f , f , ( ER , S ) d − f , I 0 , I 1 , . . . , I d − f . ◮ No issues with EL ⇒ Progressive-Sieving toward the left instead. ◮ Can now Sieve again after insertion ◮ Can now insert the best candidate rather than a pre-chosen one pump-up pump-down � �� � � �� � ( I , S ) r − ℓ . ( EL , S ) r − ℓ , Pump ℓ ′ ,ℓ, r , s : Reset ℓ ′ , r , r , L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 52 / 59

WorkOut Workout: Pumps of increasing strength WorkOut κ,β, f , f + , s : Pump κ,κ + β − f + ,κ + β, s , Pump κ,κ + β − 2 f + ,κ + β, s , Pump κ,κ + β − 3 f + ,κ + β, s , . . . Pump κ,κ + f ,κ + β, s , ◮ Termination condition can vary (e.g. fixed number of dims for free, or reached satisfying shortest vector) ◮ steps size of pump strength is not necessarly 1 L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 53 / 59

Pump and Jump ◮ Block is left somewhat reduced by the pump in the previous block: ⇒ no need for a full workout. ◮ Many short vectors inserted, little improvement left around here: ⇒ directly Jump far away. PumpnJumpBKZ β ′ , f , j : Pump 0 , f ,β , Pump j , j + f , j + β , Pump 2 j , 2 j + f , 2 j + β , . . . L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 54 / 59

Implementation 3 layers ◮ c++: multi-threaded heavy duty operation (Sieve, db updates) ◮ cython: middleware, basis maintainance ◮ python: control, tuning, and monitoring Several Sieve inside: ◮ Standard Gauss-Sieve (mono-threaded) Mem = 2 . 208 n + o ( n ) , Time = 2 . 415 n + o ( n ) ◮ Becker-Gama-Joux with 1 level of filtration (multi-threaded) Mem = 2 . 208 n + o ( n ) , Time = 2 . 349 n + o ( n ) ◮ k-sieve k = 2 , 3 (multi-threaded) Mem = 2 . 208 n + o ( n ) , Time = 2 . 349 n + o ( n ) Mem = 2 . 189 n + o ( n ) , Time = 2 . 372 n + o ( n ) L´ eo Ducas (CWI, Amsterdam) G6K or, the Alg. Ant and the Sandpile 24 Sept 2018 55 / 59