A Practical Guide to Deep Learning at the Department of Mathematics - PowerPoint PPT Presentation

A Practical Guide to Deep Learning at the Department of Mathematics Vegard Antun (UiO) March 19, 2019 1 / 61

Layout of the talk Part I Computer resources, the linux operating system, large scale computations. Part II Neural networks, mathematical framework, practical example. 2 / 61

Computer resources CPU Cache Memory Hard drive 3 / 61

Memory Hierarchies tertiary storage (tapes) secondary storage 5 ms 3.5 months (disks) main memory 50 ns 1.5 minutes cache(s) On die memory - 1 ns 2 s 0.3 ns < 1 s University of Oslo INF1060, Pål Halvorsen

Computer resources GPU CPU Cache Memory Memory Hard drive 5 / 61

Time measurements Total time for 10 epochs on CIFAR10. Batch size 10. ◮ CPU: 8 min, 35 sec ◮ GPU: 53 sec ( ≈ 10 times faster) Loading 50 MR scans (each 40 MB) on nam shub 20 15 Seconds 10 5 0 Network Local disk RAM 6 / 61

Operating systems (OS) Operating system Hardware 7 / 61

The Linux Filesystem Hierarchy The uppermost directory in the Linux file system is / [ ∼ ]$ ls Desktop Downloads Pictures www_docs Documents pc WINDOWS [ ∼ ]$ pwd /mn/sarpanitu/ansatte -u4/vegarant [ ∼ ]$ cd / [ / ]$ ls admin etc lib misc opt sbin tf usit bin hf lib64 mn proc site tmp usr boot home local mnt rh srv ub uv dev ifi med net root sv uio var div jus media odont run sys use 8 / 61

Some important directories ◮ /bin Most basic executable files ( ls, cp, cd ) ◮ /lib Libraries used by the executables ◮ /boot Files related to the boot loader ◮ /dev All devices, /dev/random, /dev/null, /dev/pst/0 ◮ /etc Configuration files, /etc/hostname, /etc/passwd ◮ /home/username Your home folder ∼ / (not on UiO-system) ◮ /root Home directory of root user ◮ /tmp Temporary files - Not preserved during reboots ◮ /usr Read-only user data. Multiuser applications ◮ /var Variable files, i.e. files which changes during execution 9 / 61

Environment variables Variable with a name and a value, used by one or more applications. To view all type env Some important environment variables ◮ PATH All directories where we search for executables ◮ PYTHONPATH All directories where we search for python modules ◮ HOME Your home directory i.e. the position of ∼ / ◮ EDITOR Default editor ◮ TF_CPP_MIN_LOG_LEVEL Level of verbosity for tensorflow 11 / 61

Environment variables - Example [ ∼ ]$ echo $PYTHONPATH /path/to/module1 :/ path/to/module2 [ ∼ ]$ [ ∼ ]$ export PYTHONPATH=$PYTHONPATH :/ path/to/new_module [ ∼ ]$ [ ∼ ]$ echo $PYTHONPATH /path/to/module1 :/ path/to/module2 :/ path/to/new_module 12 / 61

The ∼ /.bashrc The scrip language you type in the terminal is called “BASH“ (Bourne Again SHell) We often want the environment to stay persistent between logins. Set defaults in the files ◮ ∼ /.bashrc Run each time you open a terminal on your computer [ ∼ ]$ cat ∼ /. bashrc export PYTHONPATH=$PYTHONPATH :/ path/to/new_module export TF_CPP_MIN_LOG_LEVEL =1 alias la=’ls -a --color=auto ’ alias ll=’ls -lh --color=auto ’ # Describes the command line prompt PS1=’[ \h \w ]$ ’ 13 / 61

The ∼ /.bashrc and ∼ /.bash profile files ◮ ∼ /.bashrc Run each time you open a terminal on your computer ◮ ∼ /.bash_profile Run each time you log in remotely. To have two different settings in ∼ /.bashrc and ∼ /.bash profile is often inconvenient. To only use the ∼ /.bashrc file, place the following lines in your ∼ /.bash profile [ ∼ ]$ cat .bash_profile if [ -f ∼ /. bashrc ]; then . ∼ /. bashrc fi Note: Files starting with ’.’ don’t show whenever you type ls . In order to see these files, type ls -a 14 / 61

Login to remote machines via SSH Login to the universities network from a personal linux or mac computer [ ∼ ]$ ssh -X username@login .math.uio.no The -X options enabels X11 forwarding i.e. you can open GUI based applications. Once you are logged in you can continue to the desired computer by typing [ ∼ ]$ ssh -X computername [ ∼ ]$ # Example , logging into the hadad computer [ ∼ ]$ ssh -X hadad 15 / 61

Login to remote machines via SSH Next we will see how to make this preceedure require less typing! 16 / 61

SSH config file Create the file ∼ /.ssh/config and add the following lines host uio hostname login.math.uio.no user your_username ForwardX11 no You can then logon to the university’s network by ssh -X uio We assume you have this setup in the rest of this presentation 17 / 61

SSH keys To make the UiO passwords secure they often require a lot of typing. SSH-keys provides an easy way to maintain high sequrety while having shorter passwords. 18 / 61

Generate and set up SSH-key [ ∼ ]$ ssh -keygen -t rsa -b 4096 -C "your@email.com" This command will create two files ◮ ∼ /.ssh/id_rsa Private key. Do not share it. ◮ ∼ /.ssh/id_rsa.pub Public key. Can be shared with anyone. Copy the public key to the remote host (UIO) ssh − copy − i d − i ∼ / . ssh / i d r s a . pub < username > @login . math . uio . no 19 / 61

SSH and jump connections Your comp. login.math.uio.no math comp. ◮ Jump connection sends the ssh trafic directly through a computer like a regular ruter ◮ You avoid some typing and you do not allocate a terminal on the jump computer ◮ Does only allow for one jump 20 / 61

SSH and jump connections To use jump connection add the following to your ∼ /.ssh/config # Setup for the math computers , this example belet -ili Host belet -ili1 Hostname belet -ili.uio.no ProxyJump vegarant@login .math.uio.no User vegarant or you can add the jump connection directly ssh − J < username > @login . math . uio . no < username > @ < hostname > . uio . no 21 / 61

Terminal window managers ◮ Common choices are “tmux“ or “screen“. 22 / 61

Monitor CPU usage ◮ Use the htop command to view CPU-usage and priority 23 / 61

Reducing the priority of your process ◮ Linux processes can have “niceness“ values {− 20, . . . , 19 } where a smaller value gives higher priority. ◮ Negative nice values can only be given by root user/administrator. ◮ The default priority of any process you start will be 0 i.e. you will typicaly reduce the priority. [ ∼ ]$ nice -n 19 python3 my_python_script .py & 24 / 61

Monitor GPU usage ◮ All of our GPUs are from Nvidia. To view their current usage use nvidia-smi ◮ To call this command every 5 second use the watch command [ ∼ ]$ watch -n 5 nvidia -smi [ ∼ ]$ # or use [ ∼ ]$ nvidia -smi -l 5 25 / 61

GPU resources at Dep. of Mathematics Name GPU CPU cores Mem. scratch nam-shub-01 4 × RTX 2080 ti 28 128GB 30GB zadkiel 1 × RTX 2080 4 16 GB − belet-ili 1 × GTX 1080 4 16 GB − cleopatra 1 × GTX 1080 4 16 GB − euphrosyne 1 × GTX 1080 4 16 GB − hadad 1 × GTX 1080 4 16 GB − 26 / 61

AI HUB ◮ An experimental service for machine learning provided by USIT, to gain experience with hardware and software for deep learning. ◮ Reserved for students on weekdays (Mon-Fri) from 09:00 to 17:00. ◮ Need to login via Abel (add ssh keys as before). None presistent Name GPU CPU cores Mem. scratch 4 × RTX 2080 Ti ml1 28 128 GB 17TB 4 × RTX 2080 Ti ml2 28 128 GB 17TB 4 × RTX 2080 Ti ml3 28 128 GB 17TB ◮ AI mailing list: itf-ai-announcements@usit.uio.no 28 / 61

Deep learning frameworks ◮ Many old frameworks like: MatConvNet, Caffe, Theano ... ◮ For most scientists Tensorflow (and maybe Pytorch) would be the prefered option. 29 / 61

Tensorflow ◮ Developed by Google, and have a large community. ◮ Relatively well documented ◮ Have APIs in Python, JavaScript, C++, Java, Go, Swift. ◮ Models can be deployed into applications, such as websites and phones. 30 / 61

How to run Tensorflow? ◮ No unified way to do this on all systems. ◮ The machines ml1, ml2 and ml3, have tensorflow v1.12 and PyTorch v1.0. Just type python3 to get started. ◮ On math computers we use the module system (and maybe singularity) module avail # See which modules are avaiable module load tensorflow/<version > # Load tensorflow module rm tensorflow/<version > # Unload tensorflow module list # view loaded modules ◮ ML software located under python-ml/<version> and tensorflow/<version> . Do not load both. 31 / 61

Singularity ◮ Singularity (similar to docker) is container with a minimal operating system. ◮ Shares the kernel with the host operating system so that CPU overhead is almost non. ◮ You can install whatever software you like within the container, with the nessesary libaries. ◮ Makes reproducible research much easier! ◮ Check out Tormod Landet’s excelent guide to singularity http://folk.uio.no/tormodla/singularity/ ◮ On maths computers precompiled singularity images are located at /mn/sarpanitu/singularity/images/Machine_learning 32 / 61

Neat commands ◮ ag or ack – Search for pattern in each source file in the tree from the current directory and downward. ◮ fzf – Fuzzy finder. Search for filenames in the tree from the current directory and downwards. ◮ which <command> – E.g. which python Gives the location of the program python. ◮ nohup nice -n 19 python -u my_script.py > output.txt & – Start prosess which aren’t shut down when you exit the login shell. 33 / 61