Rise of the Machines Continuous Delivery at SEEK CO PRESENTED BY: - PowerPoint PPT Presentation

Rise of the Machines Continuous Delivery at SEEK CO PRESENTED BY:

CD @ SEEK “To the cloud”

Scorched Earth Deployments ● Create a pristine environment for every release. ● No need to migrate state of environment. ○ ∴ Configuration Management tools not required. ● State should not be changed at runtime. ○ ∴ Immutable Server ● Blue/Green deployment strategy: ○ Monitor health during deployment. ○ Auto-rollback if new environment is unhealthy.

Automate all the things! ● Delivery team is responsible for the automation of: ○ Server installation ○ Infrastructure creation ○ Deployment ● Automation is impossible without team managed infrastructure.

Infrastructure as Code SEEK use AWS CloudFormation to automate their infrastructure. ● Operations are responsible for the network-level infrastructure, ● including: VPC ○ Routing ○ Users and Roles ○ Shared services ○ Delivery team is responsible for infrastructure within the VPC, such ● as: Elastic Load Balancer ○ Security Group ○ Auto Scaling Group ○ Launch Configuration ○ Instance Profile ○

Server automation Provisioning Linux: ● SSH ⇒ Easy! ○ Provisioning Windows: ● WinRM ⇒ Hard! ○ AWS makes bootstrapping easier: ○ EC2 User Data ■ cfn-init ■ Packer Windows Plugins can be used to create Windows Amazon ○ Machine Images Combination of Packer and CloudFormation can be used regardless ● of target platform

Packaging Applications should be self-installable. ● Application installation should be separate from server automation. ● Installation should be independent of Continuous Deployment tool. ● Native package manager is ideal solution: ● Linux: apt-get or yum ○ Windows: ○ One-Get (available in next version of Windows) ■ Chocolatey (available now!) ■ Package manager is platform independent pattern ● <package-manager> install <package-name> ○ Windows Coding Horror: ● Interacting with Windows resources is not trivial, custom ○ PowerShell modules or PowerShell DSC Resources make this easier.

Pre-Baked Images ● Install a package on a base image. ○ But… don’t include configuration. ● Then, create an image (AMI). ● AMI is our environment independent deployable artifact. ● Future is containerised... ○ Docker is the next evolution of our approach.

Late-binding of context Keep your artifact environment agnostic: ● Frameworks want you to apply configuration at build or ○ package-time. Deploy tools want to apply configuration at install-time. ○ When deploying to the cloud, you should apply configuration at ○ launch-time. When AMI is launched it doesn’t know where it is: ● Need to inject environment-specific context at launch-time. ○ How do you do this? ○

Environment Variables A platform independent way of specifying environment-specific config that’s been around since 1979 Use environment variables for: ● DB connection parameters. ○ Location of service endpoints. ○ Don’t use them for: ● Altering runtime behaviour. ○ Overriding .NET configuration with environment variables: ● .NET 4.5.x: write your own Configuration Provider. ○ .NET 4.6: works out of the box. ○ Azure + .NET 4.5.x: works out of the box. ○ Service discovery is on the horizon: ● SEEK is investigating. ○ Externalised config is first step. ○

Confidence to Release Put enough steps and checkpoints in pipeline to get enough ● confidence to release. Too many checks will result in a slow pipeline and kill the benefits of ● Continuous Delivery. Take things away until you cry. ● Then... compensate some other way ○ For example, consumer-driven contract testing makes sure we ○ honour API contracts without the need to integrate.

Ship It! Distinction between Deployment and release: ● Deploy: create Blue environment and deploy application. ○ Release: cut-over from Green to Blue environment. ○ Smoke test in Blue environment before release using synthetic ● transaction. Same smoke test can be used to monitor health after release. ● Choose to enable manual release if change set is risky and additional ● manual testing is required.

Towards Polyglot Architecture Continuous Delivery + Microservices gives teams the ability to choose ● the right stack for the job. For SEEK, CD is opening up the possibility migrating legacy apps from ● Windows to Linux. Who knows, in the future, running Windows containers will be a ○ viable alternative. SEEKs approach to Continuous Delivery provides better logical ● separation automation steps. Time to first commit still a problem and this will only get worse with ● the proliferation of different stacks and tools.

Machine Factories

Manual Development Environments Are a bad experience for your developers. ■ Are time consuming and therefore costly. ■ Make it difficult for devs to move between projects. ■ Are rarely accurately reconstructed. ■ Encourages manual configuration rather than the more ■ reliable and repeatable practice of Infrastructure-as-code . Can lead to environments that have all of the parts, but not quite in the right order.

“It works on my box” Sign of environmental divergence. ■ The Butterfly effect : Minor differences in environment ■ configuration can produce obscure bugs in Prod. This issue is compounded when deployments carried out by ■ another person (e.g. Ops).

Dev vs Production Parity Environment Parity = Confidence. ■ Overcompensate lack of parity with more, possibly manual tests. ■ Environment Parity… ■ enables continuous delivery . ○ requires diligence and automation for accurate replication. ○ encourages infrastructure-as-code. ○

Wouldn’t it be great if we... Could use declarative formats for setup automation, to ■ minimize time and cost for new developers joining the project. Had a clean contract with the underlying operating system, ■ offering maximum portability between execution environments.

Wouldn’t it be great if we... Built apps suitable for deployment on modern cloud ■ platforms, obviating the need for servers and systems admin. Could minimize divergence between development and ■ production, enabling continuous deployment. Could scale up without significant changes to tooling, ■ architecture, or development practices.

The “12 Factor App” These 5 principles are what loosely describe the 12 factor ■ App, a guideline for modern SaaS products. 12 Factor Apps, Factor #10: Dev/prod parity ■ “ Keep development, staging, and production as similar as ■ possible ”. Vagrant can help us solve the environment parity issue. ■

Vagrant, Docker Compose & Friends Onward to immutable development environments

Scorched Earth Dev Environments Dev automation typically come in 2 flavours: ■ Use of CM Tools like Puppet, Chef (Boxen, Babushka) ○ Virtual Machine Runners (Vagrant, Docker Compose) ○ CM Tools fraught with challenges: ■ Not isolated ○ Hard to automate across OS/Platforms ○ Doesn’t force developers to automate ○

What is Vagrant? “A tool to help create and configure lightweight, reproducible, and portable development environments.” A replacement for manually configured boxes. ■ Enforces infrastructure-as-code. ○ A machine runner (typically a VM or a Container) ■ Separates authoring environment from development runtime ■ Isolated & repeatable ■

What is Vagrant? The following is a logical view of how Vagrant interacts with your authoring environment: Vagrant has spun up Virtualbox VM with the name ‘Web’, ■ and is running it on a Mac OSX desktop Code is synchronised from /c/proj/x -> /vagrant of ■ the running environment

What is Vagrant? The Web application spins up a server using the code in ■ /vagrant, which binds to a private network on the VM on port 80 Port 8080 is exposed to the outside ‘world’ (in this case, a ■ Mac OSX desktop) which forwards requests to port 80 in the running VM The Developer is free to author in whichever IDE of her ■ choosing

The TAO of Vagrant vagrant up to get a fully working dev runtime. ■ Development occurs in preferred IDE: Visual Studio, ■ Sublime, ViM, with your terminal, your shortcuts - etc. vagrant destroy && vagrant up will destroy the ■ machine and create a new, clean, isolated environment. Workflow repeatable and transferrable to other projects. ■

Developing on Ecosystems From monoliths to ecosystems

From Monolith to Microservices Applications are usually more interesting than a single UI, ■ they are usually connected to things. They form part of a functioning ecosystem. ○ Vagrant good for Monoliths and ecosystems. ■ Explosion of microservices highlights this need. ■

From Monolith to Microservices Commentary on Architecture: Multiple components, in disparate technologies ■ Distributed system ■ Layered architecture ■ = complexity ■ Increased complexity increases the number of things that can go wrong. We can combat this with parity and local simulation.