Secure and version controlled configuration with Puppet, Hiera and - PowerPoint PPT Presentation

Secure and version controlled configuration with Puppet, Hiera and GPG Jens Bräuer <braeuer.jens@gmail.com>

Agenda 1. Background 2. Puppet for infrastructure automation 2.1 Externalize configuration information 2.2 Problems with externalized information 3. Approach to be secure and version controlled Puppet ahead - Ask questions please.

Background Commercetools GmbH (CT) ● full service eCommerce solutions (Redbull, Brita, ...) ● backed by RackSpace ● total ~40 people Products ● "current" GRID ○ Play + Tomcat + MySQL + Solr ● "new" Sphere ○ Play + Scala + MongoDB + ElasticSearch

Background ● Operations ○ ~ 60 servers in total (+ some legacy) ■ ~ 20 GRID ■ ~ 20 Sphere ■ ~ 20 "support" (internal, build machines, etc) ● one OS (Ubuntu 12.04, almost, see legacy) ● heavy use of Puppet ○ for development (Vagrant Box) ○ for CI ○ for Staging + Production

Real world example - Configure ElasticSearch

ElasticSearch + Puppet class elasticsearch::server { $elasticsearch_dev_mode = true Simple $elasticsearch_cluster_name = "elasticsearch" approach: $elasticsearch_cluster_members = ["localhost"] Hard coded package { "elasticsearch": ensure => installed } # uses cluster_name and cluster_members file { "/etc/elasticsearch/elasticsearch.yml": owner => root, group => root, mode => 0644, content => template("elasticsearch/elasticsearch.yml.erb"), } service { "elasticsearch": ensure => running, enable => true, require => [Package["elasticsearch"]] } }

Hiera to the rescue extlookup, external node classifier (enc) Hiera, short for “hierarchy” [...] is a pluggable, hierarchical database that can query [...] backends for configuration data. class elasticsearch::server( $elasticsearch_dev_mode = hiera("elasticsearch.dev.mode") $elasticsearch_cluster_name = hiera("elasticsearch.cluster.name") $elasticsearch_cluster_members = hiera("elasticsearch.cluster.members") ) {

Configuration information - by src elasticsearch.cluster.members: - search1.sphere.cloud.commercetools.de sphere-stage - search2.sphere.cloud.commercetools.de postfix.relay.username: user postfix.relay.password: XXXXXXXXXXX postfix.relay.host: smtp.mailjet.com elasticsearch.dev.mode: true sphere postfix.relay.username: automation@ct.de postfix.relay.password: XXXXXXXXXXXX postfix.relay.host: mail.google.com global net.trust: - 1.2.3.4 - 2.3.4.4

Configuration information - merged elasticsearch.dev.mode: true elasticsearch.cluster.members: hiera - search1.sphere.cloud.commercetools.de - search2.sphere.cloud.commercetools.de postfix.relay.username: user postfix.relay.password: XXXXXXXXXXX postfix.relay.host: smtp.mailjet.com net.trust: - 1.2.3.4 - 2.3.4.4

Hiera to the rescue Supply default values in-code $elasticsearch_cluster_name = hiera("elasticsearch.cluster.name", "mycluster" ) Overwrite keys in-code class { "elasticsearch::server": elasticsearch_cluster_name => "othername" }

Hiera to the rescue Backends to get data from ● YAML ● JSON ● MySQL ● Mongo ● Redis ● ... Included in Puppet 3.x, Add-on in Puppet 2.7

Share (and contain) config information

Configuration hierarchy Common Sphere Vagrant GRID Build Needed By Devs Stage Prod CI Stage Prod Please NO Devs :)

Share config information Requirements ● version control: plain files ● JSON or YAML ● have sensitive and non-sensitive fields side by side (hiera-gpg) ● limit visibility of items Decisions ● separate config in extra repository? ● (only) credentials are sensitive

Share config information Edited like mail.relay.enable: true mail.relay.server: in.mailjet.com mail.relay.port: 587 mail.relay.user: aee97cb3ad288ef0add6c6b5b5fae48a mail.relay.password: PLAIN(supersecretpassword) Stored like mail.relay.enable: true mail.relay.server: in.mailjet.com mail.relay.port: 587 mail.relay.user: aee97cb3ad288ef0add6c6b5b5fae48a mail.relay.password: ENC(jA0EAwMC96....Wghh/AtP)

Config information - visibility Common root@prod.sphere.ct.de root@stage.sphere.ct.de root@ci.sphere.ct.de Sphere GRID jens@ct.de hajo@ct.de sven@ct.de Potential different Ops-Team root@prod.sphere.ct.de CI Stage Prod jens@ct.de hajo@ct.de sven@ct.de

Raziel http://en.wikipedia.org/wiki/Raziel

Raziel - Keeper of secrets ● Change code & configuration together. ● Secure storage of production configuration information. commit f8bb8219d166201141df8454aeaa8af0b8009d04 Author: Jens Braeuer <jens.braeuer@commercetools. de> Date: Sat Jan 26 13:54:22 2013 +0000 Add read-only Icinga access. Introduce "icinga.guest.password" key.

Raziel - Keeper of secrets ● written in Ruby (1.9.x) ● two parts ○ hiera backend to transparently integrate with Puppet ○ command line tool to enable editing PLAIN(...) to ENC(....) to PLAIN(...) ● Encryption ○ based on asymmetric & symmetric encryption ○ GPG + GPG-ME ○ ruby-gpgme

Raziel - Caveats ● comments lost due to Ruby parser ● "re-keying" workflow needs manual steps

Raziel - Some stats ● small ○ <1000 LOC ○ 410 LOC in tests ● Debian package ● Tested on ○ OS-X 10.7 / 10.8 ○ Ubuntu 12.04 ● Tested with ○ Puppet 2.7 and 3.0 Will be open-sourced.

Questions? How do you do this?

Thanks! Contact me braeuer.jens@gmail.com github.com/jbraeuer Thanks Hajo! Hajo Eichler <eichler.hajo@gmail.com>