uppmax introduction
play

UPPMAX Introduction 2019-09-09 Martin Dahl - PowerPoint PPT Presentation

Nov 05, 2022 •272 likes •991 views

UPPMAX Introduction 2019-09-09 Martin Dahl martin.dahlo@scilifelab.uu.se Objectives What is UPPMAX what it provides Projects at UPPMAX How to access UPPMAX Jobs and queuing systems How to use the resources of UPPMAX How to use the

  2. UPPMAX Introduction 2019-09-09 Martin Dahlö martin.dahlo@scilifelab.uu.se

  3. Objectives What is UPPMAX what it provides Projects at UPPMAX How to access UPPMAX Jobs and queuing systems How to use the resources of UPPMAX How to use the resources of UPPMAX in a good way! Efficiency!!!

  4. UPPMAX Uppsala Multidisciplinary Center for Advanced Computational Science http://www.uppmax.uu.se 2 (3) computer clusters

  5. UPPMAX Uppsala Multidisciplinary Center for Advanced Computational Science http://www.uppmax.uu.se 2 (3) computer clusters ● Rackham : ~ 500 nodes à 20 cores (128, 256 & 1024 GB RAM) + Snowy (old Milou) : ~ 200 nodes à 16 cores (128, 256 & 512 GB RAM)

  6. UPPMAX Uppsala Multidisciplinary Center for Advanced Computational Science http://www.uppmax.uu.se 2 (3) computer clusters ● Rackham : ~ 500 nodes à 20 cores (128, 256 & 1024 GB RAM) + Snowy (old Milou) : ~ 200 nodes à 16 cores (128, 256 & 512 GB RAM) ● Bianca : 200 nodes à 16 cores (128, 256 & 512 GB RAM) - virtual cluster

  7. UPPMAX Uppsala Multidisciplinary Center for Advanced Computational Science http://www.uppmax.uu.se 2 (3) computer clusters ● Rackham : ~ 500 nodes à 20 cores (128, 256 & 1024 GB RAM) + Snowy (old Milou) : ~ 200 nodes à 16 cores (128, 256 & 512 GB RAM) ● Bianca : 200 nodes à 16 cores (128, 256 & 512 GB RAM) - virtual cluster >12 PB fast parallel storage

  8. UPPMAX Uppsala Multidisciplinary Center for Advanced Computational Science http://www.uppmax.uu.se 2 (3) computer clusters ● Rackham : ~ 500 nodes à 20 cores (128, 256 & 1024 GB RAM) + Snowy (old Milou) : ~ 200 nodes à 16 cores (128, 256 & 512 GB RAM) ● Bianca : 200 nodes à 16 cores (128, 256 & 512 GB RAM) - virtual cluster >12 PB fast parallel storage Bioinformatics software

  9. UPPMAX The basic structure of supercomputer node = computer Login nodes

  10. UPPMAX The basic structure of supercomputer Login nodes

  11. UPPMAX The basic structure of supercomputer Login nodes

  12. UPPMAX The basic structure of supercomputer Compute and Storage Login nodes

  13. Objectives What is UPPMAX what it provides Projects at UPPMAX How to access UPPMAX Jobs and queuing systems How to use the resources of UPPMAX How to use the resources of UPPMAX in a good way! Efficiency!!!

  14. Projects UPPMAX provides its resources via projects

  15. Projects UPPMAX provides its resources via projects compute storage (core-hours/month) (GB)

  16. Projects your project

  17. Projects Two separate projects: SNIC compute: cluster Rackham 2000 - 100 000+ core-hours/month 128 GB storage UPPMAX Storage: storage system CREX 1 - 100+ TB storage

  18. Projects

  19. Projects

  20. Objectives What is UPPMAX what it provides Projects at UPPMAX How to access UPPMAX Jobs and queuing systems How to use the resources of UPPMAX How to use the resources of UPPMAX in a good way! Efficiency!!!

  21. How to access UPPMAX SSH to a cluster ssh -Y your_username @ cluster_name .uppmax.uu.se

  22. How to access UPPMAX SSH to Rackham

  23. SSH

  24. SSH

  25. How to use UPPMAX Login nodes use them to access UPPMAX never use them to run jobs don’t even use them to do “quick stuff” Calculation nodes do your work here - testing and running

  26. How to use UPPMAX Calculation nodes not accessible directly SLURM (queueing system) gives you access

  27. Objectives What is UPPMAX what it provides Projects at UPPMAX How to access UPPMAX Jobs and queuing systems How to use the resources of UPPMAX How to use the resources of UPPMAX in a good way! Efficiency!!!

  28. Job Job (computing) From Wikipedia, the free encyclopedia For other uses, see Job (Unix) and Job stream. In computing, a job is a unit of work or unit of execution (that performs said work). A component of a job (as a unit of work) is called a task or a step (if sequential, as in a job stream). As a unit of execution, a job may be concretely identified with a single process, which may in turn have subprocesses (child processes; the process corresponding to the job being the parent process) which perform the tasks or steps that comprise the work of the job; or with a process group; or with an abstract reference to a process or process group, as in Unix job control.

  29. Job Read/open files Do something with the data Print/save output

  30. Job Read/open files Do something with the data Print/save output

  31. Job The basic structure of a supercomputer Parallel computing job Not one super fast

  32. Job The basic structure of a supercomputer Parallel computing Not one super fast jobs

  33. Queue System More users than nodes Need for a queue nodes - hundreds users - thousands

  34. Queue System More users than nodes Need for a queue

  35. Queue System More users than nodes Need for a queue

  36. Queue System More users than nodes Need for a queue

  37. SLURM workload manager job queue batch queue job scheduler SLURM (Simple Linux Utility for Resource Management) free and open source

  38. Objectives What is UPPMAX what it provides Projects at UPPMAX How to access UPPMAX Jobs and queuing systems How to use the resources of UPPMAX How to use the resources of UPPMAX in a good way! Efficiency!!!

  39. SLURM 1) Ask for resource and run jobs manually For testing, possibly small jobs, specific programs needing user input while running 2)Write a script and submit it to SLURM Submits an automated job to the job queue, runs when it’s your turn

  40. SLURM 1) Ask for resource and run jobs manually submit a request for resources ssh to a calculation node run programs

  41. SLURM 1) Ask for resource and run jobs manually salloc -A g2019015 -p core -n 1 -t 00:05:00 salloc - command mandatory job parameters: -A - project ID (who “pays”) -p - node or core (the type of resource) -n - number of nodes/cores -t - time

  42. SLURM -A this course project g2019015 you have to be a member -p 1 node = 20 cores 1 hour walltime = 20 core-hours -n number of cores (default value = 1) -N number of nodes -t format - hh:mm:ss default value= 7-00:00:00 jobs killed when time limit reaches - always overestimate ~ 50%

  43. SLURM Information about your jobs squeue -u <user>

  44. SLURM SSH to a calculation node (from a login node) ssh -Y <node_name>

  45. SLURM

  46. SLURM

  47. SLURM 1a) Ask for node/core and run jobs manually Interactive - books a node and connects you to it interactive -A g2019011 -p core -n 1 -t 00:05:00

  48. SLURM 2) Write a script and submit it to SLURM put all commands in a text file - script tell SLURM to run the script (use the same job parameters)

  49. SLURM 2) Write a script and submit it to SLURM put all commands in a text file - script

  50. SLURM 2) Write a script and submit it to SLURM put all commands in a text file - script job parameters tasks to be done

  51. SLURM 2) Write a script and submit it to SLURM put all commands in a text file - script

  52. SLURM 2) Write a script and submit it to SLURM tell SLURM to run the script (use the same job parameters) sbatch test.sbatch

  53. SLURM 2) Write a script and submit it to SLURM tell SLURM to run the script (use the same job parameters) sbatch test.sbatch sbatch - command test.sbatch - name of the script file

  54. SLURM 2) Write a script and submit it to SLURM tell SLURM to run the script (use the same job parameters) sbatch -A g2019011 -p core -n 1 -t 00:05:00 test.sbatch

  55. SLURM Output Prints to a file instead of terminal slurm-<job id>.out

  56. Squeue Shows information about your jobs squeue -u <user> jobinfo -u <user>

  57. Queue System SLURM user guide go to http://www.uppmax.uu.se/ click User Guides (left-hand side menu) click Slurm user guide or just google “uppmax slurm user guide” link: http://www.uppmax.uu.se/support/user-guides/slurm-u ser-guide/

  58. UPPMAX Software 100+ programs installed Managed by a 'module system' Installed, but hidden Manually loaded before use ■ - Lists all available modules module avail module load <module name> - Loads the module module unload <module name> - Unloads the module module list - Lists loaded modules module spider <word> - Searches all modules after 'word'

  59. UPPMAX Software Most bioinfo programs hidden under bioinfo-tools Load bioinfo-tools first, then program module or

  61. UPPMAX Commands uquota

  62. UPPMAX Commands projinfo

  63. UPPMAX Commands projplot -A <proj-id> (-h for more options)

  64. Objectives What is UPPMAX what it provides Projects at UPPMAX How to access UPPMAX Jobs and queuing systems How to use the resources of UPPMAX How to use the resources of UPPMAX in a good way! Efficiency!!!

  65. UPPMAX Commands Plot efficiency $ jobstats -p -A <projid>

Recommend

Quality of Life tips 2019-11-26 Anders Sjlander anders.sjolander@uppmax.uu.se Terminal

Quality of Life tips 2019-11-26 Anders Sjlander anders.sjolander@uppmax.uu.se Terminal

Quality of Life tips 2019-11-26 Anders Sjlander anders.sjolander@uppmax.uu.se Terminal improvements There are a lot of small tips that will improve your experience greatly. This lecture will cover some of them: Navigating the

757 views • 33 slides
INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

TRAINING COURSE ON SOCIAL PROTECTION &amp; FORMALIZATION TRINIDAD AND TOBAGO MARCH 15, 2017 INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION Design of the NIS Assistance from the

547 views • 23 slides
INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

Patterns of variation in the expression of case and agreement Andrs Brny Leiden University Centre for Linguistics 9 November 2019, BaSIS Workshop a.barany@hum.leidenuniv.nl INTRODUCTION Introduction 2/42 INTRODUCTION

1.37k views • 42 slides
Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures Definition and concepts Process and product properties Principles SE Approaches Motivation Software and the economy The economies of

1.03k views • 49 slides
Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1 https://www.cdc.gov/std/stats17/infographic.htm Introduction Epidemiology in Western Colorado CPHE Surveillance Data Report, May 2018 Chlamydia Epidemiology in Western

977 views • 39 slides
DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every meeting (2) Some ideas of diagrams for section 7.1 (Introduction) of TP. Giles Barr 25 Jan 2018, Pembroke College On DUNE, the reality is there is a

474 views • 10 slides
INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

ADVANCED COMPUTER SECURITY INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT IS SECURITY? A systems security policies describe What the system is supposed to do Store and provide access

479 views • 16 slides
Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout World Wide

387 views • 14 slides
14 Introduction Introduction Bad guys can put malware into Example: hosts

14 Introduction Introduction Bad guys can put malware into Example: hosts

Introduction Introduction source Encapsulation ISO/OSI reference model message M application segment H t H t M transport network datagram H n H n H t M presentation: allow applications to frame link H l H n H t M interpret

583 views • 4 slides
Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout

322 views • 8 slides
Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Introduction Zynq Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL Processing System Processor Peripherals Data Buses AXI Bus Conclusion BTE5380 - Embedded Systems Octobre 2014 Andreas Habegger

1.81k views • 30 slides
2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04 Project References S E 5 PART.1 SMILE5 Introduction M I L Introduction STORION-SMILE5 Residential Energy Storage System Inverter 5kW Output Input

1.06k views • 25 slides
Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an application server? Why use an application server? Course introduction Services provided by CICS How CICS applications are defined Scaling CICS to meet

1.84k views • 146 slides
Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic building blocks Node Links (relationship between nodes) Spatial information Network data

584 views • 9 slides
Introduction Introduction (1) Main argument of the paper: universities are not only

Introduction Introduction (1) Main argument of the paper: universities are not only

Le jugement des pairs comme outil de management des universits Peer-review as a management tool Christine Musselin (CSO, Sciences Po et CNRS) Avril 2013, Lirrsistible ascension du capitalisme acadmique Introduction Introduction (1)

590 views • 28 slides
JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of the brand new JA-100 alarm system Visions Basic architecture System components Setting Help us with final validations You are the FIRST ones

1.69k views • 111 slides
Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Moreno Baricevic Gilberto Daz Axel Kohlmeyer Stefano Cozzini ULA ICTP CNR-IOM DEMOCRITOS Merida, VENEZUELA Trieste, ITALY Trieste, ITALY Introduction Introduction to storage and to storage and filesystems filesystems Introduction

1.19k views • 51 slides
1 Introduction Introduction Communication System Baseband an adjective that describes

1 Introduction Introduction Communication System Baseband an adjective that describes

Introduction Introduction Physical Layer Mobile network Global ISP Provides the means to transmit bits Basic Concepts of from sender to receiver, Data Transmission Defines the mechanical, electrical, and Home network timing

463 views • 7 slides
Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical

Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical

INFOGR Computer Graphics Jacco Bikker - April-July 2015 - Lecture 1: Introduction Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical Details Assignments Field Study State of

869 views • 62 slides
Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction

Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction

Virtual and Augmented Reality Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction Who am I? What do I do here? 2 About me Short research bio: - 2014 - finished PhD at FMFI UK - 2014 2016 -

1.3k views • 39 slides
PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience

PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience

PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience for your presentation. It tells them what your presentation is about, why it is important, and finally, what to listen for in your speech. Get the

505 views • 3 slides
How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer

How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer

Introduction to Beamer Hafida Benhidour How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer Introduction to Beamer Hafida Benhidour Department of computer science King Saud University November

732 views • 49 slides
Shenzhen Cuilu jewelry Co., Ltd was founded in 1996 and its a large private enterprise

Shenzhen Cuilu jewelry Co., Ltd was founded in 1996 and its a large private enterprise

Introduction Introduction - - Brief Brief Introduction Introduction of of Cuilu Cuilu Group Group Introduction Introduction - - Brief Brief Introduction Introduction of of Cuilu Cuilu Group Group Shenzhen Cuilu jewelry Co., Ltd

147 views • 14 slides
Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

V olunteer C ommunications N etwork (V-C-N.org) Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org Academy Course - 2012-02.24-2208 262: Introduction to ICS-214 : V-C-N.org : Slide: 1 V olunteer C

735 views • 34 slides

More recommend