Please write title, subtitle and speaker name in all capital letters PostgreSQL on K8S at Zalando: Two years in production FOSDEM 2020 PostgreSQL devroom Brussels ALEXANDER KUKUSHKIN 02-02-2020
Put images in the grey dotted box "unsupported placeholder" ABOUT ME Please write the title in all capital letters Use bullet points to summarize information Alexander Kukushkin rather than writing long paragraphs in the text box Database Engineer @ZalandoTech The Patroni guy alexander.kukushkin@zalando.de Twitter: @cyberdemn 2
Put images in the grey dotted box "unsupported placeholder" WE BRING FASHION TO PEOPLE IN 17 COUNTRIES Please write the title in all capital letters 17 markets 7 fulfillment centers 26.4 million active customers 5.4 billion € net sales 2018 250 million visits per month 15,000 employees in Europe 3
Put images in the grey Put images in the grey dotted box "unsupported dotted box "unsupported placeholder" placeholder" Please write the title in all Please write the title in all capital letters capital letters AGENDA Brief introduction to Kubernetes Spilo & Patroni Postgres-Operator Typical problems and horror stories 4
Put images in the grey dotted box "unsupported placeholder" Kubernetes at Zalando Please write the title in all capital letters ● > 140 Kubernetes clusters ○ 50/50 production/test ● Deployment to production only via CI/CD ● Access to production clusters is possible, but restricted ○ Requires the open incident ticket or approval by a colleague (4 eyes principle) 5
Put images in the grey dotted box "unsupported placeholder" PostgreSQL on K8s at Zalando Please write the title in all capital letters > 1400 6
Put images in the grey dotted box "unsupported placeholder" Terminology Please write the title in all capital letters Traditional infrastructure Kubernetes ● Physical server ● Node ● Virtual machine ● Pod ● Individual application ● Container (typically Docker) ● NAS/SAN ● Persistent Volumes ● Load balancer ● Service/Endpoint ● Application registry/hardware information ● Labels ● Password files, certificates ● Secrets 7
Put images in the grey dotted box "unsupported placeholder" Kubernetes overview Please write the title in all capital letters 8
Put images in the grey dotted box "unsupported placeholder" Stateful applications on Kubernetes Please write the title in all capital letters ● PersistentVolumes ○ Abstracts details how storage is provisioned ○ Supports many different storage types via plugins: ■ EBS, AzureDisk, iSCSI, NFS, CEPH, Glusterfs and so on ● StatefulSets ○ Guarantied number of Pods with stable (and unique) identifiers ○ Ordered deployment and scaling ○ Connecting Pods with corresponding persistent storage ( PersistentVolume + PersistentVolumeClaim ) 9
Please write the title in all capital letters Spilo Docker image Use bullet points to summarize information rather than writing long paragraphs in the text box ● All supported versions of PostgreSQL inside the single image ● Plenty of extensions (pg_partman, pg_cron, postgis, timescaledb, etc) ● Additional tools (pgq, pgbouncer, wal-e/wal-g) ● PGDATA on an external volume ● Patroni for HA ● Environment-variables based configuration 10
Please write the title in all capital letters What is Patroni Use bullet points to summarize information rather than writing long paragraphs in the text box ● Automatic failover solution for PostgreSQL ● A python daemon that manages one PostgreSQL instance ● Uses Kubernetes objects (Endpoint or ConfigMap) for leader elections ○ Makes PostgreSQL 1st class citizen on Kubernetes! ● Helps to automate a lot of things like: ○ A new cluster deployment ○ Scaling out and in ○ PostgreSQL configuration management 11
Please write the title in all capital letters Spilo & Patroni on K8S Node1 Service: demo-repl labelSelector: role=replica Pod : demo-1 PersistentVolume role: replica WATCH() Secret: demo Node2 S3 ) ( Endpoint: demo E T A D P U Pod : demo-0 PersistentVolume role: master Service: demo StatefulSet : demo 12
Please write the title in all capital letters Manual deployment to Kubernetes Use bullet points to summarize information rather than writing long paragraphs in the text box ● A few long YAML manifests to write ● Different parts of PostgreSQL configuration spread over multiple manifests ● No easy way to work with a cluster as a whole (update, delete) ● Manual generation of DB objects, i.e. users, and their passwords. 13
Put images in the grey dotted box "unsupported placeholder" Kubernetes rolling upgrade Please write the title in all capital letters ● Rotates all worker nodes in the K8s cluster ● Does it in a rolling matter, one-by-one ● If you are unlucky, it will cause the number of failover equal number of pods in your postgres cluster 14
Please write the title in all capital letters Kubernetes rolling upgrade Availability Zone 1 Availability Zone 2 Availability Zone 3 Node Node Node Node Node Node cluster: A cluster: A cluster: A primary replica replica cluster: B cluster: B cluster: B primary replica replica cluster: C cluster: C cluster: C replica primary replica Node (to-be-decommissioned) Node (new) Active Pod Terminated Pod 15
Please write the title in all capital letters Kubernetes rolling upgrade Availability Zone 1 Availability Zone 2 Availability Zone 3 Node Node Node Node Node Node cluster: A cluster: A cluster: A primary primary replica cluster: B cluster: B cluster: B primary replica primary cluster: C cluster: C cluster: C replica primary replica Node (to-be-decommissioned) Node (new) Active Pod Terminated Pod 16
Please write the title in all capital letters Kubernetes rolling upgrade Availability Zone 1 Availability Zone 2 Availability Zone 3 Node Node Node Node Node cluster: A cluster: A cluster: A replica primary replica cluster: B cluster: B cluster: B replica replica primary cluster: C cluster: C cluster: C replica primary replica Node (to-be-decommissioned) Node (new) Active Pod Terminated Pod 17
Please write the title in all capital letters Kubernetes rolling upgrade Availability Zone 1 Availability Zone 2 Availability Zone 3 Node Node Node Node Node cluster: A cluster: A cluster: A replica primary primary cluster: B cluster: B cluster: B replica replica primary cluster: C cluster: C cluster: C replica primary primary Node (to-be-decommissioned) Node (new) Active Pod Terminated Pod 18
Please write the title in all capital letters Kubernetes rolling upgrade Availability Zone 1 Availability Zone 2 Availability Zone 3 Node Node Node Node cluster: A cluster: A cluster: A replica replica primary cluster: B cluster: B cluster: B replica replica primary cluster: C cluster: C cluster: C replica replica primary Node (to-be-decommissioned) Node (new) Active Pod Terminated Pod 19
Please write the title in all capital letters Kubernetes rolling upgrade Availability Zone 1 Availability Zone 2 Availability Zone 3 Node Node Node Node cluster: A cluster: A cluster: A primary replica primary cluster: B cluster: B cluster: B replica primary primary cluster: C cluster: C cluster: C primary replica primary Node (to-be-decommissioned) Node (new) Active Pod Terminated Pod 20
Please write the title in all capital letters Kubernetes rolling upgrade Availability Zone 1 Availability Zone 2 Availability Zone 3 Node Node Node cluster: A cluster: A cluster: A primary replica replica cluster: B cluster: B cluster: B replica primary replica cluster: C cluster: C cluster: C primary replica replica Node (to-be-decommissioned) Node (new) Active Pod Terminated Pod 21
Put images in the grey dotted box "unsupported placeholder" Kubernetes rolling upgrade Please write the title in all capital letters Cluster Number of failovers A 3 B 2 C 2 22
Put images in the grey dotted box "unsupported placeholder" - behind the orange box and quote in capital letters We need automation!
Please write the title in all capital letters PostgreSQL cluster life-cycle Use bullet points to summarize information rather than writing long paragraphs in the text box create/update deploy or do cluster config a rolling upgrade provision/sync decommission db user (periodically) 24
Please write the title in all capital letters Goals Use bullet points to summarize information rather than writing long paragraphs in the text box ● Fully automated: ○ deployments ○ cluster upgrades ○ user management ○ minimize a number of failovers 25
Recommend
More recommend
