CONTAINER ORCHESTRATION WITH SWARM MODE, MESOS/MARATHON AND KUBERNETES ADRIAN MOUAT
WHO AM I? Chief Scientist at Container Solutions Wrote "Using Docker" for O'Reilly 40% discount with code AUTHD Docker Captain @adrianmouat
WHAT IS ORCHESTRATION? "The planning or coordination of the elements of a situation to produce a desired effect, especially surreptitiously" Oxford English Dictionary
"The planning or coordination of the elements of a situation to produce a desired effect, especially surreptitiously"
ELEMENTS Containers Hosts Networking
"The planning or coordination of the elements of a situation to produce a desired effect , especially surreptitiously"
DESIRED EFFECT Running application Automatically scale Fault tolerant failover, node rebalancing, health checks Use resources efficiently Little manual intervention
"The planning or coordination of the elements of a situation to produce a desired effect, especially surreptitiously "
SURREPTITIOUSLY "In a way that attempts to avoid notice or attention; secretively" Oxford English Dictionary
SURREPTITIOUSLY Should happen in the background User doesn't need to details Complexity is hidden
HOW IMPORTANT IS ORCHESTRATION? Might not need it for small apps No orchestration == manual orchestration Manually place containers, network, scale, check, update
COMPARING ORCHESTRATORS
COMPARING ORCHESTRATORS All work and are improving rapidly Understand the differences Understand your requirements Please don't roll your own!
THE PLAYERS Kubernetes Mesos Docker Swarm Mode Plus others Nomad, PaaSs...
SIDE NOTE - THE BORG/OMEGA PAPERS Influential papers from Google Lessons learnt from 10 years with containers Both high-level and technical reports Kubernetes, Docker Swarm and Nomad
WELCOME TO THE SOCK SHOP!
MICROSERVICE REFERENCE APPLICATION Intended to help people get started with microservices Great for comparing frameworks etc Similar to "Pet Store" for Java ... or TodoMVC for JavaScript
LOTS OF IMPLEMENTATIONS ALREADY https://github.com/microservices-demo/microservices- demo/tree/master/deploy
ARCHITECTURE
DEMO
DOCKER SWARM MODE
SWARM MODE New in Docker 1.12 Docker Inc's official solution Part of core distribution Major improvement over old Swarm
CORE COMPONENTS Manager nodes coordinate via Raft no need for separate etcd/zookeeper Worker nodes
SNAP TO INSTALL $ docker swarm init Swarm initialized: current node (ejgr3igiwu7mdvescuncyfq19) is now a manager. T o add a worker to this swarm, run the following command: docker swarm join \ --token SWMTKN-1-5cubmn41arycoxa2hk4iu6gmauzafros4vhvnq8ftp0cuacn2u-2jxkuoimbzzoow00wuyxhf1mo \ 10.34.9.126:2377 T o add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
SECURE COMMS BY DEFAULT TLS set up using self-signed certs Certificates automatically rotated
OTHER FEATURES Spread scheduling chooses "least loaded" node more options later support for reserving & limiting cpu/mem Global or Replica mode "Mesh Networking"
CONSTRAINTS AND FILTERS Control which node a container can be scheduled on E.g: Only nodes labeled "staging" Only nodes which have the image Only the node running a given container
RUNNING SOCK SHOP https://raw.githubusercontent.com/microservices- demo/microservices-demo/master/deploy/swarmkit/start- swarmkit-services.sh (note the repo currently needs updated in-line with version shown in talk)
SWARM MODE ADVANTAGES Snap to install Secure by default Bundled with Docker
SWARM MODE DISADVANTAGES New Some Docker features unsupported DAB still WIP No concept of Pods
MESOS Predates Docker Used by large companies Twitter, Airbnb, eBay ... Mixed load Run Hadoop jobs next to Docker etc
CORE COMPONENTS Containerizer Mesos Agents Mesos Master ZooKeeper Frameworks
CONTAINERIZER Responsible for running isolated process In our case a Docker image Either use Docker or internal runtime
SIDENOTE - CONTAINER RUNTIMES Both Mesos and Kubernetes moving away from Docker Mesos to its own runtime K8s becoming "runtime agnostic" in reality rkt or runc You probably shouldn't care As long as it can run Docker/OCI images Developers will still use Docker
MESOS AGENTS Runs on nodes responsible for executing tasks Submit available resources to master
MESOS MASTER Sends tasks to agents Offers resources to frameworks
ZOOKEEPER Distributed HA database Used primarily for keeping track of current master
FRAMEWORKS Coordinate with master to schedule tasks Multiple frameworks available Can be run together Marathon, Aurora, Chronos, ElasticSearch
RUNNING SOCK SHOP https://raw.githubusercontent.com/microservices- demo/microservices-demo/master/deploy/mesos- marathon/marathon.json
MESOS ADVANTAGES Battle-tested GUI Frameworks
MESOS DISADVANTAGES Needs 3rd party tools networking loadbalancing service discovery Marathon JSON could be cleaner
KUBERNETES
KUBERNETES Based on Google's experience running containers Bakes in various features Load-balancing, secret management More opinionated
CORE CONCEPTS Pods Flat networking space Labels Services Deployments ReplicaSets
PODS Groups of containers deployed and scheduled together Atomic unit Containers in a pod share IP address Single container pods are common Pods are ephemeral
FLAT NETWORKING SPACE All pods are in the same network space Simple cross host communication
LABELS K/V pairs attached to objects (primarily pods) e.g: "version: dev", "tier: frontend" Label selectors then used to identify groups Used for load-balancing etc
SERVICES Stable endpoints addressed by name Forward traffic to pods Pods are selected by labels Round-robin load-balancing Separates endpoint from implementation
DEPLOYMENTS & REPLICASETS ReplicaSets monitor status of Pods start/stop pods as needed Deployments start ReplicaSets Rollout/Rollback & Updates
RUNNING SOCK SHOP https://github.com/microservices-demo/microservices- demo/blob/master/deploy/kubernetes/complete-demo.yaml
KUBERNETES ADVANTAGES Advanced features baked-in Lot of momentum behind the community
KUBERNETES DISADVANTAGES Harder to get started Extra concepts
OTHER CONSIDERATIONS Persistent Data Volumes Secrets Auto-scaling Multi-tenancy Health checks Updates and maintenance Images and platform Federation
CONCLUSIONS Different options with different strengths In some ways surprisingly similar Hard to predict a winner All are much better than rolling-your-own
Swarm Mode Easy to deploy Secure by default WIP Mesos/Marathon Battle-tested Lacks some features Frameworks
Kubernetes Lot of momentum Advanced features
REFERENCES Sock Shop Reference Application TNS Article on Sock Shop Google Papers Large-scale cluster management at Google with Borg (2015) Omega flexible, scalable schedulers for large compute clusters (2013) Borg, Omega, and Kubernetes (2016)
Recommend
More recommend
