Vitess@QCon Sugu Sougoumarane, Co-creator CTO, PlanetScale - PowerPoint PPT Presentation

Vitess@QCon Sugu Sougoumarane, Co-creator CTO, PlanetScale @ssougou ssougou.blogspot.com

Database trends ● Transactional data explosion ● Move to cloud

Transactional data options ● Single instance RDBMS ● NoSQL ● App Sharded RDBMS ● NewSQL

Cloud ● Instances go down all the time ○ Good master promotion story ● Durability through replication ○ Semi-sync ● Small instances ● Discovery mechanism

Vitess ● Leverage MySQL ● Massive Scale ● Battle-hardened ● Runs “anywhere”

Vitess stats Started 2010 Marquee adopters 100+ contributors 6,000+ stars 500+ Slack members 15,000+ commits pxhere CC0

Key adopters 20% Migrated to Vitess 10,000+ Tablets on Kubernetes Cash App fully runs on Vitess

The Community In production Evaluating Quiz of Kings

MySQL Community Award “Application of the year 2018”

Growing with Vitess app server Vitess app server MainDB Connection pooling app server Deadlines Hot row protection Row count limit Blacklisting

Growing with Vitess app server Replica Vitess app server Master Replica routing app server Load balancing Master promotion with Orchestrator

Growing with Vitess Replica DB2 app server Vitess app server Unified View Replica DB1 app server

Growing with Vitess Replica DB2 app server Replica DB1:S1 Vitess app server Sharding agnostic Replica DB1:S2 app server Replica DB1:S3

Zone 1 Zone 2 Zone 3 Growing with Vitess Replica Replica Replica DB2 app server Replica Replica Replica DB1:S1 Vitess app server Multi-zone Replica Replica Replica app server DB1:S2 Isolation Replica Replica Replica DB1:S3

ALL YOUR DATABASE

Architecture shard n lockserver app server vtctld shard 1 vttablet app server mysqld vtgate master app server vtgate vttablet vttablet vttablet vttablet mysqld mysqld mysqld mysqld big data replicas big data replicas App Vitess

Sharding concepts VTGate Database ● VSchema ● Schema ● Primary Vindex ● Primary Key ● Secondary Vindex ● Secondary Key ○ Unique ○ Unique ○ Non-unique ○ Non-unique ● Shared Vindex ● Foreign Key ● Sequences ● Auto-increment

Resharding ● No downtime ● Vertical and Horizontal ● Splits and Merges ● Checks and balances

Demo lookup user merchant unsharded sharded sharded user merchant name_user_idx id name name name uid product order id id uid pid mname

Questions? ● Sign into slack channel at vitess.io ● Try bringing up a cluster on your laptop or kubernetes ● GH: http://github.com/vitessio/vitess

Appendix

Demo user merchant lookup select sharded sharded unsharded name, id as uid user merchant name_user_idx from user select * id name name from merchant name where uid in_keyrange( mname, ‘unicode_loose_md5’, product order order ‘-80’) id id id uid uid pid pid mname mname select * from product product product id id

DB Protection ● Connection pools ● Hot rows ● Damage control ○ Row-count limit ○ Time limit ○ Transaction time limits ● Blacklisting

Observability ● Stats end points ● Per-query stats ● Querylog

Life of a Query: VTGate vschema shardmap Vindexes vindex Keyspace ids shards AST Plan Engine Engine Parse PlanBuilder SQL Part 1 Part 2 topology health SQL ScatterConn Gateway vttablets

Life of a Query: VTTablet Filtered Replication schema PKs AST Plan Parse PlanBuilder ACL Consolidator SQL Pool Deadliner MySQL BinlogPlayer Transaction Watcher

Roadmap ● Materialized views ● Real-time rollups ● VSchema-based resharding ● Full cross-shard SQL support