Performance Issues for Parallel Implementations of Bootstrap - PowerPoint PPT Presentation

Performance Issues for Parallel Implementations of Bootstrap Simulation Algorithm 22 nd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD 2010) Ricardo M. Czekster, Paulo Fernandes, Afonso Sales and Thais Webber Pontif´ ıcia Universidade Cat´ olica do Rio Grande do Sul (PUCRS) PaleoProspec Project - PUCRS/Petrobras Funded also by CAPES and CNPq - Brazil

Context Interest The solution of complex and large state-based stochastic models to extract performance indices. Solution Numerical (iterative methods) ◮ Power method [Stewart 94] ◮ Arnoldi [Arnoldi 51] ◮ GMRES [Saad and Schultz 86] Simulation ◮ Traditional [Ross 96] ◮ Monte Carlo [H¨ aggstr¨ om 02] ◮ Backward [Propp and Wilson 96] ◮ Bootstrap [Czekster et al. 10] ⋆ reliable estimations ⋆ high computational cost to generate repeated batches of samples

Context Markovian simulation Generation of independent samples (parallel execution) Parallel sampling ( e.g. , master-worker approach) Possible sequence of states using the transition matrix ◮ random walk or simulation trajectory ◮ huge size → huge memory cost ◮ Stochastic Automata Networks (structured formalism, underlying Markov Chain)

Objective Goal It is to present a parallel implementation of Bootstrap simulation, focusing on the overall technique performance by presenting a method to generate large amount of samples in less time. Discussion processing x communication times model size x amount of generated samples

Outline Stochastic Automata Networks (SAN) Bootstrap simulation Parallelization Experiments and results Conclusion and future works

Stochastic Automata Networks (SAN) • It allows the description of a large system in a structured manner by its parts (automata) SAN model Underlying Continuous-Time Markov Chain A B C 0 0 0 α 111 l 1 s 1 s 2 s 1 s 2 s 1 000 1 1 1 γ β 100 Type Event Rate Type Event Rate Type Event Rate syn s 1 α syn s 2 β loc l 1 f f = [( B == 0) && ( C == 0)] × γ

Stochastic Automata Networks (SAN) • It allows the description of a large system in a structured manner by its parts (automata) SAN model Underlying Continuous-Time Markov Chain A B C 0 0 0 α 0 1 l 1 s 1 s 2 s 1 s 2 s 1 1 1 1 γ β 2 Type Event Rate Type Event Rate Type Event Rate syn s 1 α syn s 2 β loc l 1 f f = [( B == 0) && ( C == 0)] × γ

Bootstrap Method It is a well known statistical method applied to many fields to improve accuracy when performing sample estimations for complex distributions. In the simulation context Bootstrap simulation provides more reliable estimations than the traditional simulation [SCSC’10] . Main feature Generation of repeated batches of samples that helps to improve the method accuracy.

Traditional simulation States Transition Matrix 2 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 1 2 0 . 30 0 . 25 0 . 45 0 n = trajectory length Time each visited state = sample 0 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 0 0 0 n

Traditional simulation States Transition Matrix 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 n = trajectory length Time each visited state = sample 0 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 0 0 0 n

Traditional simulation States Transition Matrix 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 n = trajectory length Time each visited state = sample 0 1 U = 0 . 08 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 0 0 0 n

Traditional simulation States Transition Matrix 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time each visited state = sample 0 1 U = 0 . 08 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 0 0 0 n

Traditional simulation States Transition Matrix 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time each visited state = sample 0 1 U = 0 . 08 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 1 0 0 n

Traditional simulation States Transition Matrix 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time each visited state = sample 0 1 2 U = 0 . 87 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 1 0 0 n

Traditional simulation States Transition Matrix 2 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time each visited state = sample 0 1 2 U = 0 . 87 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 1 0 0 n

Traditional simulation States Transition Matrix 2 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time each visited state = sample 0 1 2 U = 0 . 87 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 1 0 1 n

Traditional simulation States Transition Matrix 2 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time each visited state = sample 0 1 2 3 U = 0 . 32 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 1 0 1 n

Traditional simulation States Transition Matrix 2 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time each visited state = sample 0 1 2 3 U = 0 . 32 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 1 0 1 n

Traditional simulation States Transition Matrix 2 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 1 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time each visited state = sample 0 1 2 3 U = 0 . 32 mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 1 1 1 n

Traditional simulation States Transition Matrix 2 0 1 2 0 . 10 0 . 65 0 . 25 0 0 . 25 0 . 55 0 . 20 1 1 . . . 2 0 . 30 0 . 25 0 . 45 Initial state 0 0 n = trajectory length Time . . . each visited state = sample 0 1 2 3 n mean permanence π ′ π ′ π ′ 0 1 2 probability π = π ′ π ′ 1 1 1 n

Bootstrap simulation States Transition Matrix 0 1 2 0 0 . 10 0 . 65 0 . 25 Initial state 1 0 . 25 0 . 55 0 . 20 0 2 0 . 30 0 . 25 0 . 45 Time 0 n = trajectory length K : bootstrap z : number of bootstraps mean permanence � z i =1 ¯ x i probability π = z

Bootstrap simulation States Transition Matrix 0 1 2 0 0 . 10 0 . 65 0 . 25 Initial state 1 0 . 25 0 . 55 0 . 20 0 0 2 0 . 30 0 . 25 0 . 45 Time 0 1 U = 0 . 08 K 1 K 2 K z n = trajectory length 0 0 0 . . . 1 1 1 K : bootstrap 2 2 2 z : number of bootstraps mean permanence � z i =1 ¯ x i probability π = z

Bootstrap simulation States Transition Matrix 0 1 2 0 0 . 10 0 . 65 0 . 25 Initial state 1 0 . 25 0 . 55 0 . 20 0 0 2 0 . 30 0 . 25 0 . 45 Time 0 1 U = 0 . 08 K 1 K 2 K z n = trajectory length 0 0 0 . . . 1 1 1 K : bootstrap 2 2 2 z : number of bootstraps For each bootstrap, it is performed n trials to execute the resamplings ¯ mean permanence � z i =1 ¯ x i probability π = z

Bootstrap simulation States Transition Matrix 0 1 2 0 0 . 10 0 . 65 0 . 25 Initial state 1 0 . 25 0 . 55 0 . 20 0 0 2 0 . 30 0 . 25 0 . 45 Time 0 1 U = 0 . 08 K 1 K 2 K z n = trajectory length 0 0 0 . . . 1 1 1 K : bootstrap 2 2 2 z : number of bootstraps For each bootstrap, it is performed n trials to execute the resamplings ¯ mean permanence � z i =1 ¯ x i probability π = z

Bootstrap simulation States Transition Matrix 0 1 2 0 0 . 10 0 . 65 0 . 25 Initial state 1 0 . 25 0 . 55 0 . 20 0 0 2 0 . 30 0 . 25 0 . 45 Time 0 1 U = 0 . 08 K 1 K 2 K z n = trajectory length 0 0 0 . . . 1 1 1 K : bootstrap 2 2 2 z : number of bootstraps For each bootstrap, it is performed n trials to execute the resamplings ¯ mean permanence � z i =1 ¯ x i probability π = z

Bootstrap simulation States 2 Transition Matrix 0 1 2 0 0 . 10 0 . 65 0 . 25 Initial state 1 0 . 25 0 . 55 0 . 20 0 0 2 0 . 30 0 . 25 0 . 45 Time 0 1 2 U = 0 . 87 K 1 K 2 K z n = trajectory length 0 0 0 . . . 1 1 1 K : bootstrap 2 2 2 z : number of bootstraps For each bootstrap, it is performed n trials to execute the resamplings ¯ mean permanence � z i =1 ¯ x i probability π = z

Bootstrap simulation States 2 Transition Matrix 1 0 1 2 0 0 . 10 0 . 65 0 . 25 Initial state 1 0 . 25 0 . 55 0 . 20 0 0 2 0 . 30 0 . 25 0 . 45 Time 0 1 2 3 U = 0 . 32 K 1 K 2 K z n = trajectory length 0 0 0 . . . 1 1 1 K : bootstrap 2 2 2 z : number of bootstraps For each bootstrap, it is performed n trials to execute the resamplings ¯ mean permanence � z i =1 ¯ x i probability π = z