The age of orchestration From Docker basics to cluster management - PowerPoint PPT Presentation

The age of orchestration From Docker basics to cluster management NICOLA PAOLUCCI • DEVELOPER INSTIGATOR • ATLASSIAN • @DURDN

Three minute Docker intro? Time me and ring a bell if I am over it. Just kidding I’ll be over by a bit but it’s ok. We’re friends. 3

DEFINITIONS Traditional Virtual Machines APP A APP B BINS/LIBS BINS/LIBS ROBUST GUEST OS GUEST OS MONOLITHIC SLOW TO BOOT HEAVY OVERHEAD HYPERVISOR HOST OS SERVER 4

DEFINITIONS Docker Containers APP A APP B BINS/LIBS BINS/LIBS LEAN LIGHTWEIGHT ISOLATED PORTABLE EFFICIENT DOCKER ENGINE HOST OS SERVER 5

DEFINITIONS Seen from another angle, the core of Docker is four distinct things A standard format to Clearly de fj ned Caching mechanism Central registry of interfaces to re-use steps ready images package applications 6

DO WE USE IT? We have embraced Docker on two fronts S I H T F O T N R O A I P S T S O E S N For our internal PaaS In our products 7

Our internal PaaS, called Micros @durdn

Overall Micros Numbers Java 40% Node.js 600+ Python Microservices Docker containers Rest is pre-made stacks 9 9

Media Services @durdn

Our Conversion Stack

Media Services Numbers 6M 10 TBs Microservices containers spun per Processed every month month 12 12

CONCLUSIONS Docker has been a great fit for our Media Services team Easily scale Worked around tools Isolating data from Manage resource not easy to di fg erent customers control horizontally parallelise 13

Orchestration is the next arena @durdn

What’s Orchestration? Services Orchestration Data Center YOUR APPLICATION FRAMEWORKS PHYSICAL INFRA 15

@durdn

@durdn

@durdn

19

Docker’s Own Orchestration Tools Docker machine Docker compose Docker swarm Docker network 20

Where is the DEMO Lebowski?

TOOL NR.1 Docker machine $ docker-machine create -d virtualbox dev INFO[0000] Downloading boot2docker.iso from... INFO[0001] Creating SSH key... Docker machine INFO[0001] Creating VirtualBox VM... INFO[0006] Starting VirtualBox VM... Simple command line tool to provision local and remote hosts with Docker installed. INFO[0007] Waiting for VM to start... Fantastic to get up and running fast. It has INFO[0041] "dev" has been created and is now active drivers for many Internet service providers and PaaS. 22

Docker machine DEMO • Provision a machine with Docker installed and ready $ docker-machine create -d virtualbox dev • Pull a minimal image INFO[0000] Downloading boot2docker.iso from... • Run a few docker commands INFO[0001] Creating SSH key... • Tear down the machine INFO[0001] Creating VirtualBox VM... INFO[0006] Starting VirtualBox VM... INFO[0007] Waiting for VM to start... INFO[0041] "dev" has been created and is now active

Recap of what you saw • “docker-machine create” to provision the host, locally or remotely • “docker-machine ls” to list the machines • “docker-machine stop/rm” to stop and remove $ docker-machine create -d virtualbox dev INFO[0000] Downloading boot2docker.iso from... INFO[0001] Creating SSH key... INFO[0001] Creating VirtualBox VM... INFO[0006] Starting VirtualBox VM... INFO[0007] Waiting for VM to start... INFO[0041] "dev" has been created and is now active

TOOL NR.2 Docker compose 1 bitbucket: 2 image: atlassian/bitbucket-server 3 ports: 4 - "7990:7990" 5 - "7999:7999" 6 links: 7 - db 8 volumes_from: Docker compose 9 - license 10 user: root Describe the relation of your components in a 11 privileged: true simple YAML file called docker-compose.yml 12 db: and docker-compose takes care of starting 13 image: postgres them and linking them in order. 14 ports: 15 - "5432:5432" 16 environment: 17 - "POSTGRES_PASSWORD=somepassword" 18 license: 19 build: . 26

Docker compose DEMO • Provision a machine on a PaaS • Pull PostgreSQL and a Java $ docker-compose up -d app from the Registry • Use Compose to start the app • Tear down the machine

Recap of what you saw • “docker-machine create” to provision the host • Edit “docker-compose.yml” to describe our app $ docker-compose up -d • “docker-compose up -d” to start our application • “docker-machine rm compose-demo“ to remove it

TOOL NR.3 Docker swarm 1 bitbucket: 2 image: atlassian/bitbucket-server 3 ports: 4 - "7990:7990" 5 - "7999:7999" 6 volumes_from: 7 - license 8 user: root Docker swarm 9 privileged: true 10 environment: Deploy images and run containers on a full 11 - "constraint:instance==java" clusters as if you’re handling a single machine 12 db: 13 image: postgres 14 ports: 15 - "5432:5432" 16 environment: 17 - "POSTGRES_PASSWORD=somepassword" 18 - "constraint:instance==db" 19 license: 30 20 build: .

Docker swarm High level architecture swarm master scheduler discovery service swarm node swarm node swarm node container container container container container container @durdn

Swarm comes with strategies and filters • Strategies • Spread $ docker run -e \ • Binpack constraint:instance==database --name db • Random • Filters • Constraint • Affinity • Port • Dependency • Health

HELPER TOOL Discovery Service Consul from HashiCorp For our Swarm to know which nodes are added to the infrastructure and store information about them we need to use a key- value discovery service, like Consul. 33

TOOL NR.4 Docker network $ docker network create \ --driver overlay mynet Docker network New Docker command to manage advanced and transparent networking, like creating VXLAN-based overlay networks that span across data centers. 34

The plan for the Swarm DEMO • Provision a Docker swarm • Made up of three hosts • Master node $ docker pull swarm • Node with 2gb of RAM $ docker run --rm swarm create • simple Node 6856663cdefdec325839a4b7e1de38e8 • Use labels to deploy to nodes • Run Java app and PostgreSQL on different nodes

After Part 1 Current Architecture demo-master scheduler Consul: discovery service node1 node2 label: java label: database RAM: 2GB RAM: 512MB @durdn

docker-compose.yml 1 bitbucket: Multi-host Docker compose 2 image: atlassian/bitbucket-server 3 ports: 4 - "7990:7990" 5 - "7999:7999" 6 volumes_from: 7 - license 8 user: root 9 privileged: true 10 environment: 11 - "constraint:instance==java" 12 db: 13 image: postgres 14 ports: 15 - "5432:5432" 16 environment: 17 - "POSTGRES_PASSWORD=somepassword" 18 - "constraint:instance==db" 19 license: 20 build: . 38

What we did Final Architecture demo-master scheduler Consul: discovery service Java App PostgreSQL node1 node2 label: java label: database RAM: 2GB RAM: 512MB @durdn

Recap of Swarm DEMO • We created a 3-node cluster with “docker-machine” • We tagged the nodes with labels • We started our components using label constraints and not IP addresses

I hope you are hyped as I am for all this coolness, come talk to me afterwards!

Twitter: @durdn