cluster for data mining algorithms Joo Saffran, Gabriel Garcia, - PowerPoint PPT Presentation

A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms João Saffran, Gabriel Garcia, Matheus A. Souza , Pedro H. Penna, Márcio Castro , Luís F. W. Góes and Henrique C. Freitas matheus.alcantara@sga.pucminas.br UnConventional High Performance Computing 2016 @ Euro-Par 2016 August 23

2 Summary • Introduction • Related work • Platforms and algorithms • Experimental setup • Results • Final remarks A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

3 Introduction • Computation surpassed the Petaflop barrier • Massively parallel architectures: GPUs and Many-cores Intel Xeon Phi 61 cores Nvidia Tesla K40 2880 CUDA cores A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

4 Introduction • Surpass the Exaflop barrier • Challenges: Power consumption and Financial cost A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

5 Introduction The use of those architectures might be unworkable... ... but we can not disregard the demand for performance! A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

6 Introduction Big Data: • Complex datasets whose size are very big • They (data) come from sensors, devices, multimedia, social media, etc... • Must be processed in real-time and in very large scale Big Data must exploit High Performance Computing! A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

7 Our work Given the cost and energy-efficiency constraints... ... we must consider low-cost and low-power architectures! The goal is: To evaluate the performance , power and energy consumption of an energy-efficient and low-cost Raspberry Pi Cluster running two data mining algorithms: Apriori and K-Means A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

8 Our work Contributions: • A verification of a low-cost energy-efficient Cluster and whether it can be used as an alternative for HPC • A comparison of this Cluster with a High Performance Many-Core Processor (The Intel Xeon Phi) • An evaluation which clarifies whether this platform can be used in the context of Big Data A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

9 Related work Kruger, M.J. (2015): Building a Parallella board cluster • The cluster proved to be better than an Intel i5 • Parallella lacks of hardware support for complex operations • No rigorous power evaluation was conducted d‘Amore et al. ( 2015): A practical approach to big data in tourism: A low cost Raspberry Pi Cluster • Focused on how to retrieve and use the data • No power or quantitative performance evaluation A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

10 Other initiatives The use of ARM processors: • Mont Blanc and Mont Blanc 2 projects HPC based on energy-efficient platforms • Glasgow Raspberry Pi Cloud ( PiCloud ) Cluster of Raspberry boards for cloud computing A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

11 Our work Platforms: • Intel Xeon Phi coprocessor • Raspberry Pi Cluster (8 boards interconnected) Application kernels: • Apriori • K-Means A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

12 Intel Xeon Phi • 61 cores (244 threads) interconnected by a bidirectional ring • 32 kB instruction and 32 kB data L1 caches per core • 256 kB of L2 cache per core • 16 GB of main memory A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

13 Raspberry Pi Cluster • 8 quad-core nodes (32 threads) interconnected by a network switch • 64 kB instruction and 64 kB data L1 caches per core • 512 kB of L2 cache shared by the 4 cores • 1 GB of main memory (each board) A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

14 Apriori kernel • Association rule machine-learning algorithm • Given a list of itemsets, identify association rules between those items based on their frequency • Highlight general trends in the database A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

15 Apriori kernel 1. MapReduce parallel pattern 2. Identify “starter sets” and distribute items among threads or processes 3. Each thread or node calculates the frequency 4. The subsets are regrouped 5. Check the minimum support A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

16 K-Means kernel • Widely used clustering approach • Given a set of n points, partition these points into k partitions, to minimize the mean squared distance from each point to the center Five partition K-Means example A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

17 K-Means kernel 1. Assign a thread or process a unique set of points and partitions 2. Each thread: i. Re-cluster its own points into the k partitions ii. Recalculate centroids 3. Synchronize the partitions between phases i and ii A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

18 Experimental Setup Intel Xeon Phi • OpenMP library • Intel C Compiler (ICC) • MICSMC tool, to monitor power consumption Raspberry Pi Cluster • OpenMPI and OpenMP libraries • GNU C Compiler (GCC) • Watt-meter installed before the power supply, to measure energy consumption (except the switch) A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

19 Experimental Setup Workload sizes Apriori (minimum support) • Standard: 70 • Large: 60 • Huge: 50 K-Means (# of data points) • Standard: 2 14 • Large: 2 15 • Huge: 2 16 A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

20 Experimental Setup The number of available resources varied proportionally • 30, 60, 120 and 240 threads in the Intel Xeon Phi • 4, 8, 16 and 32 threads in the Raspberry Pi Cluster A total of 10 runs were conducted • At most 11,04% standard deviation for Raspberry Pi Cluster • At most 7,07% standard deviation for Intel Xeon Phi Metrics: Execution time, Power consumption, Energy consumption Energy consumption = Execution time X Power consumption thus lower Energy consumption values means better Energy Efficiency A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

21 Results – Execution time (s) RASPBERRY PI CLUSTER Apriori scales better than K-Means Apriori has more independent work units than K-Means Standard Large Huge INTEL XEON PHI APRIORI K-MEANS A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

22 Results – Execution time (s) RASPBERRY PI CLUSTER Apriori scales better 95.02% than K-Means Apriori has more 56.79% independent work units than K-Means Standard Large Huge INTEL XEON PHI 54.84% 93.76% APRIORI K-MEANS A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

23 Results – Execution time (s) RASPBERRY PI CLUSTER Xeon Phi was faster than the Cluster Xeon Phi has cores with more processing power With almost equal number of threads (30 and 32), sometimes the Standard Cluster presented better Large results than Xeon Phi 23.81% Huge INTEL XEON PHI 69.89% APRIORI K-MEANS A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

24 Results – Execution time (s) RASPBERRY PI CLUSTER The Cluster scales better than Xeon Phi 78.51% The network switch 82.05% between 8 nodes was not a bottleneck in the Cluster Standard The synchronization Large time between 240 threads in the Xeon Phi 64.59% Huge was a bottleneck INTEL XEON PHI 73.85% APRIORI K-MEANS A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

25 Results – Power consumption (W) RASPBERRY PI CLUSTER Irregular kernels: Different execution time for each working unit Standard Large Huge INTEL XEON PHI APRIORI K-MEANS A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23

26 Results – Power consumption (W) RASPBERRY PI CLUSTER Irregular kernels: 68.06% Different execution time for each working unit 44.34% Average power decreases in Cluster when the workload increases: Slave processes finish their Standard computation, thus Large nodes reduces their 15.42% 1.39% power consumption Huge drastically INTEL XEON PHI In Xeon Phi , idle cores keep consuming a portion of power APRIORI K-MEANS A low-cost energy-efficient Raspberry Pi cluster for data mining algorithms - UCHPC 2016 - August 23