Vitess on Kubernetes … followed by a demo of VReplication Jiten Vaidya jiten@planetscale.com
A word about me ... Jiten Vaidya - Managed teams that operationalized Vitess at Youtube CEO at PlanetScale Founded in early 2018 to help operationalize Vitess • Sugu Sugumaran (CTO, Vitess community leader) Offerings • Open Source Vitess Support • Custom Vitess Development • Kubernetes Deployment Manager • Cross-cloud DBaaS
Show of hands ... ● How many of you know what Vitess is? ● How many of you are facing scaling problems? ● How many of you want to run mysql in Cloud?
What does Vitess solve? ● Database Scalability ● MySQL Guardrails ● Operation in Cloud
Vitess Architecture app server vtctld shard lockserver vttablet app server mysqld vtgate master app server vtgate vttablet vttablet vttablet vttablet mysqld mysqld mysqld mysqld big data replicas big data replicas App Vitess
Major users using Vitess In production Evaluating Quiz of Kings
Vtgate in Kubernetes ● Stateless proxy ● Vtgates can be created as load ● Accepts connections as a MySQL increase compatible server + over GRPC ● Start n vtgates as a Replica Set ● Supports an http endpoint + Web ● For co-located workloads start one UI for status vtgate per node and expose with a ● Computes target shards ClusterIP ● Sends queries to vttablets for targeted shards ● Receives, collates and serves response to application
Vtctld in Kubernetes ● Vitess Control Plane ● Start one or two vtctld processes ● Serves a Web UI per cell ○ Operational commands ● Start them as a Deployments ○ Status ● Expose them behind a Service ○ Topology browser ● Serves an API over GRPC ○ Used by vtctlclient tool ● Supports resharding workflows
lockserver (etcd) in Kubernetes ● Knits the Vitess cluster together ● One global cluster ● Backing store for metadata ● One cluster per cell (optional) ○ Service discovery ● Use etcd-operator to spin out a ○ Topology cluster ○ VSchema ● Expose etcd cluster behind a ● Not used for query serving Service ● Needed for any change in topology ○ Add a keyspace ○ Add a shard to keyspace ○ Add a tablet to a shard ○ Change master for a shard
Tablet (vttablet + mysqld) in Kubernetes ● Vitess Tablet is a combination ● 2 containers in the same pod of a mysqld instance and a ● Communicate over Unix socket corresponding vttablet process ● Unix socket created in Shared ● Each tablet requires a unique Volume id in Vitess cluster ● Local Persistent Volume for ● Tablets can be of type: master, data replica, rdonly ● One master, 2 replicas with ● Tablets of type “replica” can semi-sync replication enabled be promoted to master and for high availability should have low replica lag ● Replicas should not be co-located with other members of shard ( Anti-Affinity )
Authentication/Secrets management ● What secrets are needed? ○ Application -> Vtgate authentication ○ Vttablet -> mysqld authentication for various roles that Vitess supports (app, dba, replication, filtered replication etc). ○ TLS certs and keys for GRPC traffic over TLS (optional) ○ TLS certs and keys for binary logs over TLS (optional) ○ TLS certs and keys for client authorization and authentication over TLS (optional) ● Use Kubernetes Secrets and mount them in pods
High Availability ● Planned reparent ○ Coordinated via lockserver ○ Existing transactions are allowed to complete ○ New transactions are buffered by vtgate ○ New master is made writable ○ Replicas are made slaves of the new master ○ Query serving is resumed ● Unplanned reparent ○ Orchestrator ○ TabletExternallyReparented ● Resharding ○ No interruption to query traffic during resharding
Supporting multiple cells ● Vitess cell is the equivalent of a failure domain (e.g. AWS availability zones or regions) ● Not necessarily the same as Kubernetes failure domain. ● Choice to use global lockserver cluster OR use one lockserver cluster per cell. ● Global lockserver cluster typically outside of Kubernetes. ● Expose lockserver behind a service definition. ● If using etcd use etcd-operator to start per cell cluster.
Vitess clusters spanning Kubernetes clusters ● Global lockserver cluster should be outside of Kubernetes ● While creating each cell designate which Kubernetes cluster it resides in ● Must have a non-overlapping ip space and all addresses must be routable. ● pod to pod communication ○ Needed for mysql replication and query serving ○ Achieved by Peering and Routing
Part II: Vitess in Action ● Vitess super power: filtered replication aka VReplication ● Ability to consume binary replication log, and apply sharding or other logic to it ● Shard splits (aka resharding) ● Sharded Materialized Views
Splitting a Shard in Vitess Step 1 - Vitess in-front of app 1 shard. Vttablet (Shard -) app server vtgate mysqld
Live Resharding Filtered Replication Step 2 - Split Shard Vttablet (Shard 00-80) mysqld Vttablet (Shard 0) app server vtgate mysqld Vttablet (Shard 80-FF) mysqld 17
Live Resharding Step 3 - Split Shard Vttablet (Shard 00-80) mysqld app server vtgate Vttablet (Shard 80-FF) mysqld
Hot off the presses ... … Sharded Materialized Views (Ability to shard using more than one sharding key using replication streams) M shards using key1 -> N shards using key2 M * N replication streams
Materialized view example
Demo lookup user merchant unsharded sharded sharded user name_user_idx id merchant name name name uid product order Id id description uid pid mname
Questions? jiten@planetscale.com
Recommend
More recommend
