Sparse Prefix Sums Michael Shekelyan, Anton Digns, Johann Gamper 1 - PowerPoint PPT Presentation

1 7 8 6 7 6 5 16 5 6 77 8 2 5 54 Sparse Prefix Sums Michael Shekelyan, Anton Dignös, Johann Gamper 1 0 0 0 1 7 7 15 0 0 0 6 6 0 0 0 Free University of Bozen-Bolzano, Italy 7 0 0 6 13 0 0 6 7 0 5 27 34 0 0 6 8 5 10 32 46 7 7 98 8 0 0 0 8 2 7 7 25.09.2017, Nicosia, Cyprus 8 0 0 54 62 2 7 7 8 0 0 54 62 2 7 7 Michael Shekelyan ! 1 ADBIS’17 - Sparse Prefix Sums ! 1

Outline Introduction • range sums • prefix sums • technique to compute range sums in constant-time • relative prefix sums • technique to achieve faster updating • related work Contribution • sparse prefix sums • compression of relative prefix sums preserving constant query time Experiments • low-resolution grids • high-resolution grids • impact of dimensionality • population grids based on satellite imagery Michael Shekelyan ! 2 ADBIS’17 - Sparse Prefix Sums ! 2

Range Sums original data table 1 7 8 6 7 6 5 16 5 6 77 8 2 5 54 • e.g. each cell counts the number of inhabitants in a city Michael Shekelyan ! 3 ADBIS’17 - Sparse Prefix Sums ! 3

Range Sums original data table 1 7 8 6 7 6 5 16 5 6 77 8 2 5 54 • e.g. each cell counts the number of inhabitants in a city Michael Shekelyan ! 4 ADBIS’17 - Sparse Prefix Sums ! 4

Range Sums original data table 1 7 8 6 S 7 6 5 16 5 6 77 8 2 5 T 54 • range sum: sum values in a sub-matrix/tensor • query: how many inhabitants live in range between S and T? • answer: 16+6+77 = 99 Michael Shekelyan ! 5 ADBIS’17 - Sparse Prefix Sums ! 5

Prefix Sums original data table S = origin 1 7 8 6 7 6 5 16 5 6 77 T 8 2 5 54 “sum of preceding values” • prefix sum: range sum with one corner at origin • query: how many inhabitants in range between origin and T? • answer: 1+7+5 = 13 Michael Shekelyan ! 6 ADBIS’17 - Sparse Prefix Sums ! 6

Prefix Sums • well-known concept across many disciplines probability theory computer graphics CDFs (1984) summed area integral tables images prefix sums computer vision database systems (2001) (1997) Michael Shekelyan ! 7 ADBIS’17 - Sparse Prefix Sums ! 7

Prefix Sums original data table S = origin 1 7 8 6 7 6 5 16 5 6 77 T 8 2 5 54 • prefix sum: range sum with one corner at origin • query: how many inhabitants in range between origin and T? • answer: 1+7+5 = 13 Michael Shekelyan ! 8 ADBIS’17 - Sparse Prefix Sums ! 8

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • data table with N cells has N prefix sums! • idea: store result for each prefix sum • result: O(1) querying Michael Shekelyan ! 9 ADBIS’17 - Sparse Prefix Sums ! 9

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • data table with N cells has N prefix sums! • idea: store result for each prefix sum • result: O(1) querying • example: 13 = 1+7+5 Michael Shekelyan ! 10 ADBIS’17 - Sparse Prefix Sums ! 10

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 8 35 7 6 8 8 14 14 21 21 5 16 8 8 13 35 35 42 42 56 18 53 144 5 6 77 8 13 40 46 53 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • each range sum can be computed from 2 d prefix sums Michael Shekelyan ! 11 ADBIS’17 - Sparse Prefix Sums ! 11

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • add prefix sum (144) Michael Shekelyan ! 12 ADBIS’17 - Sparse Prefix Sums ! 12

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • prefix sum (144) adds too much Michael Shekelyan ! 13 ADBIS’17 - Sparse Prefix Sums ! 13

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • subtract prefix sum (35) Michael Shekelyan ! 14 ADBIS’17 - Sparse Prefix Sums ! 14

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • still too much added on the left (5+5) Michael Shekelyan ! 15 ADBIS’17 - Sparse Prefix Sums ! 15

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • subtract prefix sum (18) Michael Shekelyan ! 16 ADBIS’17 - Sparse Prefix Sums ! 16

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • top-left (1+7) was first added once and then subtracted twice Michael Shekelyan ! 17 ADBIS’17 - Sparse Prefix Sums ! 17

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • add top-left again to balance out additions/subtractions Michael Shekelyan ! 18 ADBIS’17 - Sparse Prefix Sums ! 18

Prefix Sums original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 16+6+77 = 99 144-35-18+8 = 99 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 • range sum computed with prefix sums at corner cells of range Michael Shekelyan ! 19 ADBIS’17 - Sparse Prefix Sums ! 19

Prefix Sum Updating original data table prefix sums 1 7 8 1 1 1 1 1 8 8 16 6 1 1 1 7 7 14 14 22 7 6 8 8 8 14 14 21 21 35 5 16 8 8 13 35 35 42 42 56 5 6 77 8 13 18 40 46 53 53 144 8 2 5 16 21 26 48 54 63 68 159 54 16 21 26 54 108 117 122 213 16 21 26 102 108 117 122 213 update complexity update complexity O(1) with dense storage O(N) where N is # of cells • • Michael Shekelyan ! 20 ADBIS’17 - Sparse Prefix Sums ! 20

Prefix Sum Updating original data table prefix sums relative prefix sums 1 7 8 1 1 1 1 1 8 8 16 1 0 0 0 1 7 7 15 6 1 1 1 7 7 14 14 22 0 0 0 6 6 0 0 0 7 6 8 8 8 14 14 21 21 35 7 0 0 6 13 0 0 6 5 16 8 8 13 35 35 42 42 56 7 0 5 27 34 0 0 6 5 6 77 8 13 18 40 46 53 53 144 8 5 10 32 46 7 7 98 8 2 5 16 21 26 48 54 63 68 159 8 0 0 0 8 2 7 7 54 16 21 26 54 108 117 122 213 8 0 0 54 62 2 7 7 16 21 26 102 108 117 122 213 8 0 0 54 62 2 7 7 update complexity update complexity update complexity O(1) with dense storage O(N) where N is # of cells O(N 1/2 ) • • • Michael Shekelyan ! 21 ADBIS’17 - Sparse Prefix Sums ! 21

Relative Prefix Sums original data table prefix sums relative prefix sums 1 7 8 1 1 1 1 1 8 8 16 1 0 0 0 1 7 7 15 6 1 1 1 7 7 14 14 22 0 0 0 6 6 0 0 0 7 6 8 8 8 14 14 21 21 35 7 0 0 6 13 0 0 6 5 16 8 8 13 35 35 42 42 56 7 0 5 27 34 0 0 6 5 6 77 8 13 18 40 46 53 53 144 8 5 10 32 46 7 7 98 8 2 5 16 21 26 48 54 63 68 159 8 0 0 0 8 2 7 7 54 16 21 26 54 108 117 122 213 8 0 0 54 62 2 7 7 16 21 26 102 108 117 122 213 8 0 0 54 62 2 7 7 relative prefix sums • split into around N 1/2 blocks • each block has an anchor cell storing the prefix sum • each block has border cells storing prefix sum minus anchor cell • each block has local cells storing local prefix sum Michael Shekelyan ! 22 ADBIS’17 - Sparse Prefix Sums ! 22

Sparse Prefix Sums Michael Shekelyan, Anton Digns, Johann Gamper 1 - PowerPoint PPT Presentation

1 7 8 6 7 6 5 16 5 6 77 8 2 5 54 Sparse Prefix Sums Michael Shekelyan, Anton Digns, Johann Gamper 1 0 0 0 1 7 7 15 0 0 0 6 6 0 0 0 Free University of Bozen-Bolzano, Italy 7 0 0 6 13 0 0 6 7 0 5 27 34

Recap: Prefix Sums Given A : set of n integers Find B : prefix sums A: 3 1 1 7 2 5

Prefix sums on GPUs Motivating Problem Definitions Other Applications Parallel Algorithms

CPSC 490 DP and Range Queries Part 5: Binary Jumping, LCA; Part 1: Intro, Prefix Sums, Fenwick

Parallel Algorithms Parallel Prefix Sums Algorithm Theory WS 2012/13 Fabian Kuhn PRAM Parallel

This week, we are going to look again at another prefix. What is a prefix? Click on the right

This week, we are going to look at another prefix. What is a prefix? Choose the right answer. A

Dedekind sums ingredients Dedekind sums Fourier- Dedekind sums Restricted partition Mirco

Parallel prefix adders Kostas Vitoroulis, 2006. Presented to Dr. A. J. Al-Khalili. Concordia

Binomial = + + + Theorem + + + + + Image by MIT OpenCourseWare. Products of Sums = Sums

IP Prefix Advertisement in EVPN draft-rabadan-l2vpn-evpn-prefix-advertisement-01 Jorge Rabadan

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of

SHA-1 is a Shambles First Chosen-Prefix Collision on SHA-1 and Application to the PGP Web of

Polynomial optimization and sums of squares Sums-of-squares hierarchy Semidefinite

Foundations of Computer Science Last Time Lecture 9 Sums And Asymptotics Computing Sums

Finding good prefix networks using Haskell Mary Sheeran (Chalmers) 1 Prefix Given inputs x1,

Foundations of Computer Science Lecture 9 Sums And Asymptotics Computing Sums Asymptotics:

Sums of two squares A tale of two sums Melanie Abel Department of Mathematics University of

Gov 2000: 4. Sums, Means, and Limit Theorems Matthew Blackwell Fall 2016 1 / 60 1. Sums and

Parallel Computation Patterns Scan (Prefix Sum) Objective To master parallel scan (prefix

Treasury (spending / accessing funds, A-Board (requesting funds) SUMS technical support) Form

Border Gateway Protocol (BGP) Structure of the Internet Networks (ISPs, CDNs, etc.) group with

Border Gateway Protocol (BGP) Structure of the Internet Networks (ISPs, CDNs, etc.) group with

Sums of Money MDM4U: Mathematics of Data Management How many different sums of money can be made