Bootable Cluster CD Supercomputing 2011 Ivan Babic Andrew Fitz Gibbon Mobeen Ludin Earlham College Shodor Foundation Earlham College ibabic09 at cs.earlham.edu fitz at cs.earlham.edu mmludin08 at cs.earlham.edu Tom Murphy Charlie Peck Skylar Thompson Contra Costa College Earlham College University of Washington tmurphy at contracosta.edu charliep at cs.earlham.edu skylar at cs.earlham.edu Aaron Weeden Earlham College amweeden06 at cs.earlham.edu November 14, 2011 1 / 17
Sponsors Sponsors ■ Intel Corporation Preliminary work What is the BCCD? Short history ■ Supercomputing Conference What is the problem? How is BCCD a solution? ■ Earlham College Live Demo! Live Demo! (cont’d) ■ Shodor Foundation Boot GalaxSee - Single node GalaxSee - Single ■ Contra Costa College node (cont’d) GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 2 / 17
Preliminary work Sponsors 1. Insert the USB stick. Preliminary work What is the BCCD? Short history 2. Open a terminal (button by Firefox at the top). What is the problem? How is BCCD a 3. Type in: wget -O - \ solution? Live Demo! http://cluster.earlham.edu/bccd-ng/sc11/make-bccd| \ Live Demo! (cont’d) Boot bash GalaxSee - Single node GalaxSee - Single node (cont’d) GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 3 / 17
What is the BCCD? Sponsors ■ A software tool for domain educators for teaching CSE Preliminary work What is the BCCD? Short history ■ A stable platform for the writing of curricula for CSE What is the problem? How is BCCD a solution? ■ A live CD with pre-configured clustering software Live Demo! Live Demo! (cont’d) ■ One part of a larger CSE community Boot GalaxSee - Single node Shodor Foundation GalaxSee - Single node (cont’d) National Computational Sciences Institute (NCSI) GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) CSE Reference Desk (CSERD) Miscellaneous information References ■ “Teaching Parallelism Made Easy” Questions (and answers) Just keep pushing “Enter” 4 / 17
Short history Sponsors ■ First versions developed by Paul Gray and Students at University of Preliminary work What is the BCCD? Northern Iowa Short history What is the problem? ■ Current version (BCCDv3) collaboratively developed by a number of How is BCCD a solution? educational institutions Live Demo! Live Demo! (cont’d) Boot ■ BCCDv3 much easier to maintain and able to respond to new needs GalaxSee - Single node GalaxSee - Single ■ BCCDv3 has the same user experience as BCCDv2, so curricula node (cont’d) GalaxSee - Multi-node written for v2 will still work with v3 GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 5 / 17
What is the problem? Sponsors ■ STEM education is more important than ever [1] Preliminary work What is the BCCD? Short history ■ Computers have become a large part of any science curriculum What is the problem? How is BCCD a solution? ■ BUT ... it’s hard to find suitable resources for teaching computational Live Demo! science and parallel programming at scale Live Demo! (cont’d) Boot GalaxSee - Single ■ Many schools do not have the resources or expertise to provide the node GalaxSee - Single hardware or software node (cont’d) GalaxSee - Multi-node GalaxSee - Multi-node ■ Those that do tend to provide the resources primarily for research (cont’d) Miscellaneous and not for education information References Questions (and ■ Even with a dedicated educational resource, it tends not to be answers) “hands-on”, and is difficult to see how it works. 6 / 17
How is BCCD a solution? Sponsors ■ Almost every school will have a computer lab. BCCD is designed to Preliminary work What is the BCCD? be used on existing systems non-disruptively. Short history What is the problem? ■ Many laptops are now dual-core. Virtualization software can be used How is BCCD a solution? to setup a cluster right in front of you. Live Demo! Live Demo! (cont’d) Boot ■ Regardless of how it’s used, BCCD provides an environment that GalaxSee - Single node facilitates education. GalaxSee - Single node (cont’d) GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 7 / 17
Live Demo! Sponsors ■ Double click on Oracle VM VirtualBox. Preliminary work What is the BCCD? Short history ■ Go to New, and click Next What is the problem? How is BCCD a solution? ■ Type in “BCCD” as the machine name. Live Demo! Live Demo! (cont’d) ■ Select “Linux”, then make sure “Debian” is selected Boot GalaxSee - Single node GalaxSee - Single ■ Enter 1024 MB for the memory. node (cont’d) GalaxSee - Multi-node GalaxSee - Multi-node ■ Unselect Start-up Disk, and click Continue when prompted. (cont’d) Miscellaneous information ■ Click Create. References Questions (and answers) ■ Click Storage. 8 / 17
Live Demo! (cont’d) Sponsors ■ Click Empty by the CD symbol, click the CD drop-down on the right, Preliminary work What is the BCCD? and select Choose. Short history What is the problem? ■ Click on the USB stick symbol on the left, and go into SC11Materials. How is BCCD a solution? Live Demo! Live Demo! (cont’d) Boot ■ Select bccd.iso. GalaxSee - Single node GalaxSee - Single ■ Go to System, and select Processor, and set the number of node (cont’d) GalaxSee - Multi-node processors to 2. GalaxSee - Multi-node (cont’d) Miscellaneous ■ Go to Network, set NAT to Internal, and set the name to “bccd”. information References Questions (and ■ Click Ok. answers) 9 / 17
Boot Sponsors ■ Make sure BCCD is selected, and click Start. Preliminary work What is the BCCD? Short history ■ Just keep pressing Enter. What is the problem? How is BCCD a solution? Live Demo! Live Demo! (cont’d) Boot GalaxSee - Single node GalaxSee - Single node (cont’d) GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 10 / 17
GalaxSee - Single node Sponsors ■ In the black terminal window, run top Preliminary work What is the BCCD? Short history ■ In the blue terminal window - What is the problem? How is BCCD a $ cd $HOME/GalaxSee solution? Live Demo! $ hostname > machines Live Demo! (cont’d) Boot $ make GalaxSee - Single node GalaxSee - Single node (cont’d) GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 11 / 17
GalaxSee - Single node (cont’d) Sponsors ■ Still in the blue terminal window - Preliminary work What is the BCCD? $ time mpirun -np 1 ./GalaxSee 500 400 5000 Short history What is the problem? $ time mpirun -np 2 ./GalaxSee 500 400 5000 How is BCCD a solution? Live Demo! Notice the number of GalaxSee processes running in the top Live Demo! (cont’d) window Boot GalaxSee - Single node You might not see speedup in VirtualBox, because of GalaxSee - Single node (cont’d) virtualization overhead. GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 12 / 17
GalaxSee - Multi-node Sponsors ■ Back in VirtualBox, right-click on the BCCD system, and select Preliminary work What is the BCCD? clone. Short history What is the problem? ■ Set the name to BCCD2, click Reinitialize, Next, and then Clone. How is BCCD a solution? Live Demo! ■ Click Start for BCCD2, and follow the same boot process as the Live Demo! (cont’d) Boot original. GalaxSee - Single node GalaxSee - Single node (cont’d) GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 13 / 17
GalaxSee - Multi-node (cont’d) Sponsors ■ $ module purge && module load modules mpich2 Preliminary work What is the BCCD? Short history ■ $ make clean && make What is the problem? How is BCCD a solution? ■ $ bccd-snarfhosts Live Demo! Live Demo! (cont’d) ■ $ bccd-syncdir --ni $HOME/GalaxSee \ Boot GalaxSee - Single node $HOME/machines GalaxSee - Single node (cont’d) GalaxSee - Multi-node ■ $ time mpirun -np 4 \ GalaxSee - Multi-node (cont’d) /tmp/$(hostname -s)-$(whoami)/GalaxSee \ Miscellaneous information References 500 400 5000 Questions (and answers) ■ Notice that not all your processes are shown in the local top window. ■ Once again, speedup is limited by virtualization. 14 / 17
Miscellaneous information Sponsors Where to find documentation? http://bccd.net Preliminary work What is the BCCD? Short history What software is available? module avail What is the problem? How is BCCD a solution? How to load new software? module load name Live Demo! Live Demo! (cont’d) How to unload software? module unload name Boot GalaxSee - Single node GalaxSee - Single How to reset networking? sudo /bin/bccd-reset-network , use node (cont’d) the password you set when you booted GalaxSee - Multi-node GalaxSee - Multi-node (cont’d) Miscellaneous information References Questions (and answers) 15 / 17
Recommend
More recommend
