Unix commands for beginners D. Puthier TAGC/Inserm, U1090, - PowerPoint PPT Presentation

Unix commands for beginners D. Puthier TAGC/Inserm, U1090, denis.puthier@univ-amu.fr Matthieu Defrance, ULB, matthieu.dc.defrance@ulb.ac.be Stéphanie Le gras, Igbmc, slegras@igbmc.fr Christophe Blanchet, IFB, Christophe.BLANCHET@france-bioinformatique.fr

MATE Desktop Demo Quick overview. Installation: http://www.france-bioinformatique.fr/?q=fr/core/cellule-infrastructure/documentation-cloud Dashboard: https://cloud.france-bioinformatique.fr/cloud/instance/

The terminal… Demo Type ‘ls’ in the terminal ( l i s t files) # list files root@vm: ls

How can I speak to the terminal ● Answer : you can speak in BASH (Bourne Again Shell) * ○ BASH is one of numerous shell dialect (ksh, csh, zsh,...). ○ All this shell languages are extremely similar. ○ These languages are based on commands . ○ These modular commands allows one to perform tasks . * Reférence (calembour) au premier langage Shell écrit par Stephen Bourne :)

Command prototype(s) (1) ● One command performs a task (sort, select, open, align reads,...). ● A command has arguments that may be facultative and modify the way it works. ● These arguments may take some values. ● Most of the time an instruction (command line) starts with a command name (or path to the command). ● In the example below we will say minus v ’. # Argument without any associated value # depending on the command v means v erbose, v ersion (or other) fastqc -v # An argument with an associated value man -k jpeg

Command prototype(s) (1) ● Most of the time arguments can be written in their short of long form (more explicit/better readability). ● Long form are generally precede with ‘ -- ’ (for instance ‘ minus minus apropos’ ) # Long form without any associated value. fastqc --version # Long form with an associated value. man --apropos jpeg

Getting help ! Call you friends or better use man ( man uel) # Demo root@vm: man ls # getting help about ls root@vm: man man # getting help about man ... Help shortcuts: /foo : search for ‘foo’. n : ( n ext) next occurence of ‘foo’. p : ( p revious) previous occurrence of ‘foo’. q : quit help page.

Our first command: ls

The ls command and some of its arguments ● ls can take several arguments. ● Main arguments: ○ -l : ( l ong) get l ot of information. ○ -a ( a ll) show all files including hidden files*. ○ -1 : show results as 1 column. ○ -t ( t ime) sort results by date/ t ime. ○ -r ( r everse) reverse sort order. ● One can combine arguments ○ ls -l -a ○ ls -la * Under linux hidden files start with a ‘.’ (e.g ‘.thehiddenfile.txt’).

The ls command and some of its arguments # Demo root@vm: ls # list files root@vm: ls -a # list files including hidden files * root@vm: ls -l # get lot of information about files root@vm: ls -1 # list file (one column) root@vm: ls -t # List file by modification date ** # Combining arguments root@vm: ls -rtl # lot of info, sort by date, reverse order * WARNING with spaces. Instruction should start with a command. The ls-a command does not exists ! ** Default sorting is case-sensitive sorting.

Create directories and files

File system tree ● The file system can be viewed as a tree in which nodes are directories or files. ● This tree has a root: / ● The root folder (/) contains ○ A root folder an various additional folder* ■ Under IFB machine your root folder contains a Documents folder * Under IFB VM, you are the root/sysadmin, this is a particular case.

Hos should I refer to a file/directory ● 1) By specifying the path from the root. Absolute path . e.g; /root/Documents /root/Music ● 2) By referring to the current location/directory (the working/current directory). Relative path .

Syntax for relative path # The upper directory relative to the working directory .. # Two directories up ../.. # Three ../../.. # The current working directory ./

File system: Demo pwd ( p rint w orking d irectory); cd ( c hange d irectory). * root@vm: pwd # The current working directory (/root) root@vm: cd /root/Documents # We go into Documents root@vm: pwd # /root/Documents root@vm: cd .. # go up one level (/root) root@vm: cd /root/Music # Go to the Music folder root@vm: pwd # /root/Music root@vm: cd ../.. # Go to the root of the file system root@vm: ls # You should see the root directory root@vm: cd /root/Music # Let’s go to the root/Music directory root@vm: cd ../Documents # And to the Document folder * Use complétion ( tab key ) for file, directories and commands.

File system: some hints ● If you are the /root your data are stored in /root ○ i.e ‘user directory’ or home . ● ~ (tilda) contains the path to your home (same as $HOME). root@vm: cd / # At the root root@vm: pwd # / root@vm: cd ~/Documents # The Document directory of your home folder. root@vm: cd ~ # go to your home dir. root@vm: cd /usr/local/bin # Go to /usr/local/bin root@vm: ls ~ # list files in your ‘home’ directory root@vm: cd ~/Music # Go to the Music folder inside your home dir. root@vm: cd # == cd ~

Make directories ● We will use the mkdir ( m a k e dir ectory) command. root@vm: mkdir projet_roscoff # Create a directory root@vm: cd ./projet_roscoff # == cd projet_roscoff * root@vm: mkdir rna-seq # Let’s create a folder root@vm: mkdir chip-seq dna-seq # and several sub-folders root@vm: ls -1 # list files and folders root@vm: cd chip-seq # == ./chip-seq root@vm: pwd # the current working dir root@vm: cd ../.. # go back home * ./ is most of the time facultative

Hands on ● 1) go to ~/projet_roscoff/chip-seq ● 2) From this directory create a directory named annotation in ~/projet_roscoff/ ● Go inside annotation directory ● Check you are in the write place ● Go back home.

Hands on ● 1) go to ~/projet_roscoff/chip-seq ● 2) From this directory create a directory named annotation in ~/projet_roscoff/ ● Go inside annotation directory ● Check you are in the write place ● Go back home. ● # Solution root@vm: cd ~/projet_roscoff/chip-seq root@vm: mkdir ../ annotations root@vm: cd ../ annotations root@vm: cd

Manipulate files

Download and uncompress files ● We will use the wget command to download files. ● To uncompress we will use gunzip if the file was compressed with the gzip algorithm (extension .gz) root@vm: cd ~/projet_roscoff/annotations # on se déplace dans annotations # On télécharge le fichier root@vm: wget http://pedagogix-tagc.univ-mrs.fr/courses/data/roscoff/hg19_exons.bed.gz root@vm: ls # le fichier compressé root@vm: ls # le fichier compressé root@vm: gunzip hg19_exons.bed # on le décompresse root@vm: ls # le fichier a perdu l’extension gz

The hg19_exons.bed file Contains coordinates (start/end) of humand exons in bed format. Bed format (Bed6) ( http://genome.ucsc.edu/FAQ/FAQformat.html#format1 ) * Tabulated format (how to check that ???) Chromosome Start End Name Score Strand (Others…) * Start and End position are always given relative to the 5’/3’ orientation of the + strand. Coordinates are ‘zero-based, half-open’.

Visualising file content ● With a pager : less or more (do more or less the same). ● With head ou tail to display the n first or n last lines of a file. ● The cat command allows to send file content to the screen. <ctrl> + c to c ancel. ● The shortcuts for less are the same as for the man command. Raccourcis dans less : ↑ : go up. ↓ : go down. > : go to first line. < : go to last line. /foo : search for ‘foo’. n : n ext occurrence of foo. p : p revious occurrence of foo q : q uit.

Hands on ● 1) Look at the ten first lines of hg19_exons.bed with head. ● 2) look at the ten last lines of hg19_exons.bed with tail. ● 3) Go through the hg19_exons.bed file with less . ● 4) Send file content to the screen with cat .

Exercices ● 1) Look at the ten first lines of hg19_exons.bed with head. ● 2) look at the ten last lines of hg19_exons.bed with tail. ● 3) Go through the hg19_exons.bed file with less . ● 4) Send file content to the screen with cat . # Solution root@vm: head -n 10 hg19_exons.bed root@vm: tail -n 10 hg19_exons.bed root@vm: less hg19_exons.bed root@vm: cat hg19_exons.bed

Counting line number This can be done with the wc ( word count ) command with -l ( line ) argument. root@vm: wc -l hg19_exons.bed # 484127 exons

Extract columns ● Use the cut command with the -f ( f ield) argument ● The columns must be tabulated or use the -d argument (‘ d elimiter’) root@vm: cut -f1 hg19_exons.bed # Column 1 root@vm: cut -f1,2 hg19_exons.bed # Columns 1 and 2 root@vm: cut -f3-5 hg19_exons.bed # Columns from 3 to 5 root@vm: cut -f3- hg19_exons.bed # Column 3 to the last column

Sort a file ● On should use the sort command (alphabetic sorting by default). ○ - k ( k ey): e.g ■ -k1,1: sort by column 1. ■ -k2,2 nr : sort by column 2 using a numeric sorting in reverse order. ■ -k2,2 g : sort by column 2 (decimal sorting). Example: sort hg19_exons.bed by chromosomes then by genomic coordinates: root@vm: sort -k1,1 -k2,2nr hg19_exons.bed

Redirections

Command pipes Input Output Input Output Input Output Commande Commande Commande Error Error Error ● Standard Input: a file or text stream . ● Standard output: screen by default. ● Standard error: may be capture for log purpose.