Surfing the Infinite Monkey Theorem for Fun and Profit Dr. Jim - PowerPoint PPT Presentation

Surfing the Infinite Monkey Theorem for Fun and Profit Dr. Jim Webber Chief Scientist

Overview • The Infinite Monkey Theorem • Neo4j and graph data • Evolutionary, inclusive architecture • Science! • Revenge of the monkeys • End

OK, so what’s Neo4j? It’s a Graph Database

stole companion loves from appeared loves enemy companion in appeared in appeared enemy in enemy appeared appeared A Good Man in in Goes to War Victory of the Daleks appeared in

prop planet character stole companion loves from appeared loves enemy character companion in appeared species character in appeared enemy in enemy species episode appeared appeared episode A Good Man in in Goes to War Victory of species the Daleks appeared in

The difference between genius and stupidity is that genius has its limits

Neo4j’s architecture today

Neo4j’s Graph Native Stack Language� Drivers App� or� Community�Driver Custom� Rest Cypher� HTTP� Endpoint Bolt� Endpoint Monitoring Security Cypher�Engine Parser Line� Command� APOC� Custom Cost-based� Optimizer Availability Interface Extensions Functions Slotted Interpreted� Runtime� (EE) Runtime High� Native� Graph� Engine Browser Neo4j� In-Memory� Page� Cache Fast� Write� Buffering Indexing� Native� Graph� Storage ACID Sync CAPI� Adapter Configuration Data� Stores Logging 16

In Reality … • The architecture is a somewhat idealised picture • You generally need to know the code to see the architecture • Packages are somewhat helpful • The biggest building blocks appear as modules • It takes time to intellectually map code to architecture

Architecture is stable, but… • Implementation choices starting to be invalidated by hardware • RAM is volatile, won’t be for long • GPU commonplace, FPGA will be commonplace in < 5 years • We can update modules, but boundaries not strongly fixed • E.g. kernel-level locking behaviour can be relaxed by knowing about query pattern  greater throughput, fewer spurious aborts • We can cluster • But leader- oriented protocols aren’t good enough w.r.t. writes • We must evolve this architecture continuously, iteratively • But contributors are flighty, dev team needs some stability

But enterprise users don’t like change • They like to sweat assets like spinning disk, low-RAM servers • We have seen people in production with < 2GB RAM! • They value stability and ease of upgrade • Though some rarely upgrade • And these folks pay us $ for software licenses

Crossing the Chasm

What’s the plan? Pop-up power Fridge Power Double power sockets Oven Power Dishwasher Power Boiling water tap power

Neo4j is a broad church

DBMS Kernel, clustering: ~100% employees Cypher Query planner, compiler, optimizer: ~90% employees Surface Scripts, drivers, monitoring etc: ~70% employees

Architecture is communication

The monkeys always win! http://mcgarnagle.files.wordpress.com/2011/08/40020-strip.gif

Thanks for listening @jimwebber