SLIDE 1
Approximate Program Synthesis
James Bornholt Emina Torlak Luis Ceze Dan Grossman
University of Washington
Writing approximate programs is hard
Approximate Program Synthesis James Bornholt Emina Torlak Luis - - PowerPoint PPT Presentation
Approximate Program Synthesis James Bornholt Emina Torlak Luis - - PowerPoint PPT Presentation
Approximate Program Synthesis James Bornholt Emina Torlak Luis Ceze Dan Grossman University of Washington Writing approximate programs is hard Precise Implementation Writing approximate programs is hard Precise Implementation Writing
SLIDE 2
SLIDE 3
Writing approximate programs is hard
Precise Implementation
SLIDE 4
Writing approximate programs is hard
Precise Implementation Desired Quality
SLIDE 5
Writing approximate programs is hard
Approximate Compiler
ACCEPT, Chisel, iACT, ExpAX, …
Precise Implementation Desired Quality
SLIDE 6
Writing approximate programs is hard
Approximate Compiler
ACCEPT, Chisel, iACT, ExpAX, …
Precise Implementation Desired Quality Approximate Program
SLIDE 7
Writing approximate programs is hard
Approximate Compiler
ACCEPT, Chisel, iACT, ExpAX, …
Precise Implementation Desired Quality Approximate Program
s p e c i fj c a p p r
- x
i m a t i
- n
- p
p
- r
t u n i t i e s global
- ptimization
p r
- g
r a m a n n
- t
a t i
- n
s target language q u a l i t y m e t r i c
SLIDE 8
Synthesis: write programs automatically
Semantics Programs
SLIDE 9
Synthesis: write programs automatically
Target Behavior
Semantics Programs
f(x) = 4x + 1
SLIDE 10
Synthesis: write programs automatically
Target Behavior
Semantics Programs
f(x) = 4x + 1
SLIDE 11
Synthesis: write programs automatically
Target Behavior 4*x + 1
Semantics Programs
f(x) = 4x + 1
SLIDE 12
Synthesis: write programs automatically
Target Behavior 4*x + 1
Semantics Programs
f(x) = 4x + 1
SLIDE 13
Synthesis: write programs automatically
Target Behavior x + x + x + x + 1 4*x + 1
Semantics Programs
f(x) = 4x + 1
(x<<2) + 1
SLIDE 14
Synthesizing approximate programs
(x<<2) + 1
Semantics Programs
Target Behavior
f(x) = 4x + 1
x + x + x + x + 1 4*x + 1
SLIDE 15
Synthesizing approximate programs
Approximate Behavior (x<<2) + 1
Semantics Programs
Target Behavior
f(x) = 4x + 1
x + x + x + x + 1 4*x + 1
SLIDE 16
Synthesizing approximate programs
Approximate Behavior (x<<2) + 1
Semantics Programs
Target Behavior
f(x) = 4x + 1
x<<2 x + x + x + x + 1 4*x + 1
SLIDE 17
Synthesizing approximate programs
Approximate Behavior (x<<2) + 1
Semantics Programs
Target Behavior
f(x) = 4x + 1
x<<2 x + x + x + x + 1 4*x + 1
κ = 10 κ = 8 κ = 6 κ = 4
SLIDE 18
Synthesizing approximate programs
Approximate Behavior (x<<2) + 1
Semantics Programs
Target Behavior
f(x) = 4x + 1
x<<2 x + x + x + x + 1 4*x + 1
κ = 10 κ = 8 κ = 6 κ = 4
SLIDE 19
Synthesis automates approximation
Approximate Compiler
ACCEPT, Chisel, iACT, ExpAX, …
Precise Implementation Desired Quality Approximate Program
s p e c i fj c a p p r
- x
i m a t i
- n
- p
p
- r
t u n i t i e s global
- ptimization
p r
- g
r a m a n n
- t
a t i
- n
s target language q u a l i t y m e t r i c
SLIDE 20
Synthesis automates approximation
Approximate Program Synthesis
Precise Implementation Desired Quality Approximate Program
s p e c i fj c a p p r
- x
i m a t i
- n
- p
p
- r
t u n i t i e s global
- ptimization
p r
- g
r a m a n n
- t
a t i
- n
s target language q u a l i t y m e t r i c
SLIDE 21
Synthesis automates approximation
Approximate Program Synthesis
Precise Implementation Desired Quality Approximate Program
specifjc approximation
- pportunities
global
- ptimization
program annotations
target language q u a l i t y m e t r i c
SLIDE 22
Existing synthesizers don’t scale enough
† Alur et al. Syntax-Guided Synthesis. FMCAD 2013.
Approximate benchmarks
- fft
- kmeans
- inversek2j
- sobel
∀p, c. |dist(p, c) − f(p, c)| < 50%
SLIDE 23
Existing synthesizers don’t scale enough
† Alur et al. Syntax-Guided Synthesis. FMCAD 2013.
Approximate benchmarks
- fft
- kmeans
- inversek2j
- sobel
Off-the-shelf synthesizers†
- Symbolic
- Stochastic
- Brute-force
∀p, c. |dist(p, c) − f(p, c)| < 50%
SLIDE 24
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Existing synthesizers don’t scale enough
† Alur et al. Syntax-Guided Synthesis. FMCAD 2013.
Approximate benchmarks
- fft
- kmeans
- inversek2j
- sobel
Off-the-shelf synthesizers†
- Symbolic
- Stochastic
- Brute-force
∀p, c. |dist(p, c) − f(p, c)| < 50%
SLIDE 25
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Existing synthesizers don’t scale enough
† Alur et al. Syntax-Guided Synthesis. FMCAD 2013.
Approximate benchmarks
- fft
- kmeans
- inversek2j
- sobel
Off-the-shelf synthesizers†
- Symbolic
- Stochastic
- Brute-force
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> …
SLIDE 26
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Existing synthesizers don’t scale enough
† Alur et al. Syntax-Guided Synthesis. FMCAD 2013.
Approximate benchmarks
- fft
- kmeans
- inversek2j
- sobel
Off-the-shelf synthesizers†
- Symbolic
- Stochastic
- Brute-force
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program
SLIDE 27
Existing synthesizers don’t scale enough
† Alur et al. Syntax-Guided Synthesis. FMCAD 2013.
Approximate benchmarks
- fft
- kmeans
- inversek2j
- sobel
Off-the-shelf synthesizers†
- Symbolic
- Stochastic
- Brute-force
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert ∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program
SLIDE 28
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Existing synthesizers don’t scale enough
Programs
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program
SLIDE 29
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Existing synthesizers don’t scale enough
Programs
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program
SLIDE 30
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Existing synthesizers don’t scale enough
Programs
7.1×1043
programs
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program
SLIDE 31
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Use reference programs to guide synthesis
Programs
7.1×1043
programs
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program
SLIDE 32
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Use reference programs to guide synthesis
Programs
7.1×1043
programs
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program
SLIDE 33
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Use reference programs to guide synthesis
Programs
7.1×1043
programs
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program
SLIDE 34
float dist(float p[3], float c[3]) { float r = 0; r += (p[0] - c[0])*(p[0] - c[0]); r += (p[1] - c[1])*(p[1] - c[1]); r += (p[2] - c[2])*(p[2] - c[2]); float ret = sqrt(r); return ret; } float f(float p[3], float c[3]) { ?? } assert
Use reference programs to guide synthesis
Programs
7.1×1043
programs
∀p, c. |dist(p, c) − f(p, c)| < 50%
+ - * / & | ^ << >> … Program being synthesized Reference program + - * / & | ^ << >> …
SLIDE 35
Synthesis produces good approximations
Benchmark Speedup Error ffts 11.4× 21.3% fftc 12.0× 28.9% dist3 1.6× 14.9% sobelx 10.6× 0% sobely 10.7× 0% inversek2j1 34.8× 16.3% inversek2j2 10.0× 18.5%
Spec: < 50% average error
SLIDE 36
Synthesis produces good approximations
Benchmark Speedup Error ffts 11.4× 21.3% fftc 12.0× 28.9% dist3 1.6× 14.9% sobelx 10.6× 0% sobely 10.7× 0% inversek2j1 34.8× 16.3% inversek2j2 10.0× 18.5%
Missed compiler
- ptimisation
Missed compiler
- ptimization
Spec: < 50% average error
SLIDE 37
Synthesis produces clever approximations
float dist_approx(int a[3], int b[3]) { int c1 = abs(b[0] - a[0]); int c2 = abs(b[1] - a[1]); int c3 = abs(a[2] - b[2]); int c4 = c1 | c2; int c5 = abs(c3 > c4 ? c3 : c4); return (float)c5; } 3D Euclidean distance 1.6× faster, 14.9% error
SLIDE 38
Approximate Program Synthesis
Precise Implementation Desired Quality Approximate Program
Thanks!