Adaptive Software Speculation for Enhancing the Cost-Efficiency of Behavior-Oriented Parallelization Yunlian Jiang Xipeng Shen The College of William and Mary
High-level Parallelism Parallel computing is becoming ubiquitous High-level parallelism exists in many programs E.g. utilities, interpreters, scientific computations Difficult to parallelize Complexity in the code Uncertain parallelism Bit-level operations, Example*: unrestricted pointers, while ( s=nextSentence() ) exception handling, { parse(s); custom mem. management, if ( isCommand(s) ) third-party libraries updateParsingEnv(s); 2 }
Software Behavior-Oriented Parallelization [Ding+:PLDI07] Speculatively execute programs in parallel Efficiently detect dependence during runtime But, blind speculation causes cost-inefficiency. 3
Cost and Speedup 3 Cost SpeedUp 2 1 0 1 0.9 0.42 0.12 Speculation Success Rate 4
Cost and Speedup Cost 2 SpeedUp 1.5 1 0.5 0 1 0.9 0.42 0.12 Speculation Success Rate 5
Outline Introduction to BOP Adaptive BOP Experimental Result Conclusion 6
Outline Introduction to BOP Adaptive BOP Experimental Result Conclusion 7
Behavior-Oriented Parallelization (BOP) A tool for parallelizing sequential programs Need no parallel programming or debugging Basic scheme: software speculation Correctness protected through runtime system 8
Basic Scheme of BOP Parser Ctrl . . . . . . While ( S = ReadSentence()) { Main BeginPPR(); Possibly Parallel Parse( S ); Re Region ( (PPR) PPR) EndPPR(); Spec } . . . . . . Two reason for failed speculation: 1. Dependence violation 2. Spec runs too slow 9
Problem to tackle Cost inefficiency BOP blindly speculates every PPR instance Failed speculation may Cause slowdown to applications Protection overhead Resource (cache, bus) contention Waste computing resources CPU --- multi-programming environment Power --- Mobile computing 10
Outline Introduction to BOP Adaptive BOP Experimental Result Conclusion 11
Solution: Adaptive Speculation Basic strategy Predict profitability of PPR Speculate only likely profitable ones Prediction approaches Extended last-value-based Decayed-history-based 12
Extended last-value-based prediction Speculate or not? Speculate only if PPRsToSkip == 0. Adjust PPRsToSkip If this PPR is not speculated PPRsToSkip - - On a failed speculation PPRsToSkip = NextPenalty; NextPenalty *= α ; (increase penalty exponentially) On a successful speculation NextPenalty = 1; (reset the penalty on the next failure) 13
Extended last-value-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | Speculation! Success NextPenalty = 1 •PPRsToSkip=0 •NextPenalty=1 • α =2 14
Extended last-value-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | Speculation! Failed PPRsToSkip = 1 NextPenalty = 2 •PPRsToSkip=0 •NextPenalty=1 • α =2 15
Extended last-value-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | No Speculation! PPRsToSkip = 0 •PPRsToSkip=1 •NextPenalty=2 • α =2 16
Extended last-value-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | Speculation! Failed PPRsToSkip = 2 NextPenalty = 4 •PPRsToSkip=0 •NextPenalty=2 • α =2 17
Extended last-value-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | No Speculation! PPRsToSkip = 1 •PPRsToSkip=2 •NextPenalty=4 • α =2 18
Extended last-value-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | No Speculation! PPRsToSkip = 0 •PPRsToSkip=1 •NextPenalty=4 • α =2 19
Extended last-value-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | Speculation! Success NextPenalty = 1 •PPRsToSkip=0 •NextPenalty=4 • α =2 20
Extended last-value-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | ... Speculation! Success NextPenalty = 1 •PPRsToSkip=0 •NextPenalty=1 • α =2 21
Extended last-value-based prediction Limitations Can not keep history well Successful speculation clean history Phase changes ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 22
Decayed-history-based prediction Cumulative gain ( CG ) 1 : success cg = γ*g + (1 - γ) * cg g= 0: failed Expected Profitability ( EP ) EP = cg + SkippedPPRs * β Speculate only if EP > TH EP * TH EP : threshold of speculation; * SkippedPPRs: reset to 0 on a success increase by 1 on a non-speculated PPR 23
Decayed-history-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | •TH EP =0.6 Speculation! • β=0.2 Success • γ = 0.5 cg = 1 EP = 1 •SkippedPPRs=0 •cg=1 •EP =1 24
Decayed-history-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | Speculation! •TH EP =0.6 Failed • β=0.2 cg = 0.5 • γ = 0.5 EP = 0.5 •SkippedPPRs=0 •cg=1 •EP =1 25
Decayed-history-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | •TH EP =0.6 No Speculation! • β=0.2 SkippedPPRs=1 • γ = 0.5 EP = 0.7 •SkippedPPRs=0 •cg=0.5 •EP =0.5 26
Decayed-history-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | Speculation! •TH EP =0.6 Success • β=0.2 SkippedPPRs=0 • γ = 0.5 cg = 0.8 •SkippedPPRs=1 EP = 0.8 •cg=0.6 •EP =0.7 27
Decayed-history-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | •TH EP =0.6 Speculation! • β=0.2 Failed cg = 0.4 • γ = 0.5 EP = 0.4 •SkippedPPRs=0 •cg=0.8 •EP =0.8 28
Decayed-history-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | •TH EP =0.6 No Speculation! • β=0.2 SkippedPPRs = 1 • γ = 0.5 EP = 0.6 •SkippedPPRs=0 •cg=0.4 •EP =0.4 29
Decayed-history-based prediction | : Profitable PPR | : Unprofitable PPR | | | | | | | | | | | | | | | | | | ... •TH EP =0.6 Speculation! • β=0.2 Success SkippedPPRs = 0 • γ = 0.5 EP = 0.7 •SkippedPPRs=1 •cg=0.4 •EP =0.6 30
Outline Introduction Adaptive-Algorithms Experimental Result Conclusion 31
Experimental Result Prediction Accuracy Choose the best parameters for the algorithms Evaluate two algorithms Computation Efficiency Finishing time Time spent on all CPUs (Cost) 32
Accuracy for Last-value-based Adjust non-speculate numbers Non-speculation PPRsToSkip -1 Success Speculation NextPenalty= 1 Failed Speculation PPRsToSkip = NextPenalty NextPenalty *= α 33
Accuracy for Last-value-based α =1.4 Accuracy=81.6 34
Accuracy for Decayed history based G_TH : gain threshold Current state weight gain+ quota* β Non-speculative execution quota++ Speculative execution gain = γ*g + (1 - γ) * gain quota 0 35
Accuracy for Decayed history based Cumulative gain ( CG ) 1 : success cg = γ *g + (1- γ) * cg g= 0: failed Expected Profitability ( EP ) EP = cg + SkippedPPRs * β Speculate only if EP > TH EP * TH EP : threshold of speculation; * SkippedPPRs: reset to 0 on a success increase by 1 on a non-speculated PPR 36
Accuracy for Decayed history based TH EP = 0.25 β= 0.0075 γ = 0.4 Accuracy = 85.6% 37
Computation Efficiency Machine: Intel Pentium-D dual-core processors Compiler: gcc4.1 Benchmarks Gzip, Parser, Reduction Metrics Cost Total running time of all the processes Time Finishing time of a program 38
Efficiency comparison on gzip seq 40 org-bop 30 adapt-bop Cost(s) 20 10 0 1.6MB 320KB 192KB Buffer size 39
Efficiency comparison on gzip seq 20 org-bop 15 adapt-bop Time(s) 10 5 0 1.6MB 320KB 192KB Buffer size 40
Efficiency comparison on parser seq 40 org-bop 30 adapt-bop Cost(s) 20 10 0 50 10 2 Num of Sentences 41
Efficiency comparison on parser seq 20 org-bop 15 adapt-bop Time(S) 10 5 0 50 10 2 Num of Sentences 42
Efficiency comparison on Reduction seq 50 org-bop 40 adapt-bop Cost(s) 30 20 10 0 0 10% 50% 90% Denpendence 43
Efficiency comparison on Reduction seq 30 org-bop adapt-bop 20 Time(s) 10 0 0 10% 50% 90% Denpendence 44
Recommend
More recommend
