MILLIONS OF TRANSACTIONS PER SECOND ON A SINGLE MACHINE CASE FOR A - PowerPoint PPT Presentation

MILLIONS OF TRANSACTIONS PER SECOND ON A SINGLE MACHINE CASE FOR A VIRTUALIZED DATABASE AND SCALE-IN ROGER JOHANSSON

Who am I? Roger Johansson Actor Model, Scalability, Distributed Systems, C#, Senior Solution Architect – Starcounter Go, Kotlin http://StarCounter.io Proto.Actor Founder – Ultra Fast Distributed Actors (Go, .NET, JVM) http://Proto.Actor https://github.com/AsynkronIT Twitter: @rogeralsing Github: rogeralsing Mail: roger@starcounter.com http://Github.com/rogeralsing/presentations

Agenda Setting the stage - Why we need in memory computing: • Application Platform • Micro-Apps The Starcounter information operating system: • Our approach to in memory computing • The future of hardware • The Mars project

Front-end Framework React, Polymer Communication Palindrom - REST, Web Sockets Application View Models, Entities, App Logic In Memory App Platform Mapping, Persistence, Queries Starcounter

Front-end Framework Front-end Framework Client Side Code Network Network Services + Contracts View Models Application Code Application O/R Mapper Network Database In Memory App Platform Traditional Stack Starcounter

UI A UI B UI C App A App B App C Model A Model B Model C Starcounter

Client Side Blending UI B UI A UI C UI A UI B UI C App A App B App C Model A Model B Model C Starcounter

Profile ObjectId 789 FirstName Stefan LastName Edqvist User ObjectId 123 FirstName Stefan LastName Edqvist

Client Side Blending UI B UI A UI C UI A UI B UI C App A App B App C Model A Model B Model C Mapper ACID, Snapshot Isolation Starcounter

UI A UI B UI C App A App B App C Model A Model B Model C Mapper Starcounter

Demo Dataflow

Application Process Object Offline copy of ObjectId 123 database data FirstName Alan LastName Turing Age 41 Map to and from Object Mapper Object / Storage Network / IPC Transfer Data over model network Disk ObjectId FirstName LastName Age Data is stored on 123 Alan Turing 41 disk, optimized for spinning disks and 124 Ada Lovelace 36 moving arms 125 Grace Hopper 86 Database Process

Application Process Object Offline copy of ObjectId 123 database data FirstName Alan LastName Turing Age 41 Map to and from Object Mapper Object / Relational Network / IPC Transfer Data over model network Memory ObjectId FirstName LastName Age Data is mainly 123 Alan Turing 41 kept in memory, enabling faster 124 Ada Lovelace 36 reads and writes 125 Grace Hopper 86 Database Process

Application Process Object Stateless Entity maps directly to ObjectId 123 shared memory Shared Memory ObjectId FirstName LastName Age Data is kept in 123 Alan Turing 41 memory and shared across 124 Ada Lovelace 36 processes 125 Grace Hopper 86 Database Process

C# Node.JS Object Object Multiple processes can read the same ObjectId 123 ObjectId 123 data Shared Memory ObjectId FirstName LastName Age 123 Alan Turing 41 124 Ada Lovelace 36 125 Grace Hopper 86 Database Process

C# Source Code public class Person { public string FirstName { get; set; }

C# Compiler Output public class Person { private string _firstName; public string FirstName { get => _firstName; set value => _firstName = value; }

Starcounter Weaver Output public class Person { private string _firstName; private ulong _id; public string FirstName { get => DbState.GetString(_id, FirstNameId); set value => DbState.SetString(_id, FirstNameId, value) }

O/R mapper – Object Graphs Customer Order OrderDetail Product Lazy Load Ripple load, Hundreds/Thousands Execute SQL Query per traversed relation of queries per view Eager Load Hard to maintain, Carefully defining load boundaries inefficient joined queries

Starcounter – Object Graphs Customer Order OrderDetail Product Chasing Pointers Traversing relations are simple pointer traversals

Demo - Write 1 mil transactions

Demo - Write 1 mil transactions In this demo we provided: • ACID Transactions • Snapshot Isolation • Guaranteed disk persistency Can we go even faster? • We are capped by disk IO • The Mars project

Price of 1MB in USD over time 1000,00000 100,00000 10,00000 1,00000 USD 0,10000 0,01000 0,00100 0,00010 0,00001 Year 1980 1985 1990 1995 2000 2005 2010 2015 2020

Registers Registers Cache Cache Main Memory Main Memory Solid State Disk Non Volatile Memory Magnetic Disk Solid State Disk Magnetic Disk Current Future

Starcounter on the CPUs of 2018 Volatile DRAM Write Delta Store (incremental updates) Read Merge CPU Main Store (batch update) Non Volatile DRAM

Few Big Servers, not many small Server Server

Few Big Servers, not many small Server Server