NOSQL WITH CACHING, SEARCH AND REAL-TIME ANALYTICS James Gorlick - PowerPoint PPT Presentation

NOSQL WITH CACHING, SEARCH AND REAL-TIME ANALYTICS James Gorlick Basho bit.ly/bdp-on-the-under-ln @paegun

About Me Application Developer with Lead and Project Management experience in a wide variety of business applications. Particularly interested in straight-through processing, removing data duplication, and increasing information availability throughout the organization in a secure manner. Previously with Dimensional Fund Advisors performed data science with much Fortran and many data sources while helping DFA "buy the market". Developed a number of in-house solutions for Myspace to serve purposes similar to what BDP does and will provide. @paegun

Word Count Just as developer activities extend beyond writing and testing software into what were typically operational concerns, data solutions are extending beyond storing and retrieving data from a single database management system into the deployment and management of heterogeneous services. A walk through the development and operational support of a reference application utilizing the Basho Data Platform, gains within the development and operations cycles will be demonstrated.

Excellent Data NLC Origin TLC Station Name ernment Office Region (G County or Unitary Authority strict or Unitary Author S2 Spatial_Unit C NUTS2 Spatial Unit Station Facility Owne Station Group 5131 ABW Abbey Wood London Greater London Greenwich UKI2 Outer London Southeastern Caerphilly - West Wales and the 3813 ABE Aber Wales - Cymru Caerphilly - Caerffili Caerffili UKL1 Valleys Arriva Trains Wales Rhondda Cynon Rhondda Cynon Taff - Taff - Rhondda West Wales and the 3801 ACY Abercynon Wales - Cymru Rhondda Cynon Taf Cynon Taf UKL1 Valleys Arriva Trains Wales Period Emirates Rhondda Cynon and Reporting Days in Period Period Bus journeys Underground DLR Journeys Tram Journeys Overground Airline Rhondda Cynon Taff - Taff - Rhondda West Wales and the Financial Period period beginning ending (m) journeys (m) (m) (m) Journeys (m) Journeys (m) 3982 ABA Aberdare Wales - Cymru Rhondda Cynon Taf Cynon Taf UKL1 Valleys Arriva Trains Wales year 8976 ABD Aberdeen Scotland Aberdeen City Aberdeen City UKM1 North Eastern Scotland First ScotRail 31 1-Apr-10 1-May-10 189.1 90.5 6.3 2.3 9090 AUR Aberdour Scotland 1 Fife Fife UKM2 Eastern Scotland First ScotRail 01_10/11 28 2-May-10 29-May-10 181.6 Gwynedd - 84.5 5.8 West Wales and the 2.2 2 02_10/11 4435 AVY Aberdovey Wales - Cymru Gwynedd - Gwynedd Gwynedd UKL1 Valleys Arriva Trains Wales 28 30-May-10 26-Jun-10 175.9 84.3 5.8 2.1 3 03_10/11 Gwynedd - West Wales and the 28 27-Jun-10 24-Jul-10 183.4 86.5 6.1 2.1 4 04_10/11 4440 ABH Abererch Wales - Cymru Gwynedd - Gwynedd Gwynedd UKL1 Valleys Arriva Trains Wales 28 25-Jul-10 21-Aug-10 160.4 82.9 5.8 2.0 5 05_10/11 Monmouthshire - 28 22-Aug-10 18-Sep-10 175.8 80.9 5.5 2.0 6 06_10/11 3602 AGV Abergavenny Wales - Cymru Monmouthshire - Sir Fynwy Sir Fynwy UKL2 East Wales Arriva Trains Wales 28 19-Sep-10 16-Oct-10 189.8 88.7 6.3 2.3 7 07_10/11 West Wales and the 28 17-Oct-10 13-Nov-10 179.9 90.3 6.7 2.2 5.6 8 08_10/11 2448 AGL Abergele & Pensarn Wales - Cymru Conwy - Conwy Conwy - Conwy UKL1 Valleys Arriva Trains Wales 28 14-Nov-10 11-Dec-10 178.8 90.6 6.4 2.3 5.4 9 09_10/11 Ceredigion - Sir West Wales and the 4303 AYW Aberystwyth Wales - Cymru 28 12-Dec-10 Ceredigion - Sir Ceredigion 8-Jan-11 140.1 Ceredigion 72.5 UKL1 4.8 Valleys 1.8 3.5 Arriva Trains Wales 10 10_10/11 2552 ACR Accrington North West 28 9-Jan-11 Lancashire 5-Feb-11 183.0 Hyndburn 84.4 UKD4 6.3 Lancashire 2.1 5.2 Northern Rail 11 11_10/11 8600 AAT Achanalt Scotland 28 6-Feb-11 Highland 5-Mar-11 177.2 Highland 87.8 UKM4 6.5 Highlands and Islands 2.2 5.2 First ScotRail 12 12_10/11 8601 ACN Achnasheen Scotland 26 6-Mar-11 Highland Period and 31-Mar-11 173.9 Highland 83.5 UKM4 6.0 Number of Highlands and Islands 2.1 Operated Kms 5.8 First ScotRail % of Excess 13 13_10/11 Reporting Days in Period Financial Month Lost Customer (Peak and Off Scheduled Journey T 8602 ACH Achnashellach Scotland 30 1-Apr-11 Highland 30-Apr-11 183.8 Highland 91.2 UKM4 6.4 Highlands and Islands 2.1 6.3 First ScotRail 1 Period period ending 01_11/12 year Hours Peak) Operated (mins) Northumberland and Tyne 28 1-May-11 28-May-11 186.1 87.8 6.3 2.2 6.4 2 02_11/12 7662 ACK Acklington North East Northumberland Alnwick UKC2 and Wear Northern Rail 28 29-May-11 25-Jun-11 181.7 88.9 6.1 2.2 6.9 3 03_11/12 2 28 28-May-11 02_11/12 May-11 2,322,089 5,444,365 97.3% Abellio Greater 28 26-Jun-11 23-Jul-11 186.7 92.5 6.4 2.3 7.6 4 04_11/12 7286 ACL Acle East Norfolk Broadland UKH1 East Anglia Anglia 3 28 25-Jun-11 03_11/12 28 24-Jul-11 20-Aug-11 161.1 85.5 Jun-11 6.2 1,929,445 1.9 5,463,580 7.7 97.6% 5 05_11/12 London Midland 28 21-Aug-11 17-Sep-11 4 173.9 28 23-Jul-11 85.3 6.4 2.1 7.7 6 06_11/12 04_11/12 Jul-11 2,300,769 5,465,180 96.8% 4529 ACG Acocks Green West Midlands West Midlands Birmingham UKG3 West Midlands Trains 28 18-Sep-11 15-Oct-11 193.4 93.1 7.5 2.4 8.7 7 07_11/12 5 28 20-Aug-11 London Midland 05_11/12 Aug-11 2,059,198 5,595,332 97.1% 28 16-Oct-11 12-Nov-11 185.2 95.8 7.3 2.3 8.8 8 2287 ACB Acton Bridge 08_11/12 North West Cheshire Vale Royal UKD2 Cheshire Trains 6 28 17-Sep-11 06_11/12 Sep-11 2,462,085 5,516,502 96.8% 28 13-Nov-11 10-Dec-11 189.4 97.1 7.1 2.4 9.0 9 09_11/12 7 28 15-Oct-11 28 11-Dec-11 7-Jan-12 151.2 79.3 5.2 1.9 6.9 10 10_11/12 07_11/12 Oct-11 2,155,102 5,512,264 97.5% 28 8-Jan-12 4-Feb-12 181.4 89.8 6.9 2.2 8.7 11 11_11/12 8 28 12-Nov-11 08_11/12 Nov-11 2,054,499 5,657,701 97.5% 28 5-Feb-12 3-Mar-12 179.5 91.5 7.0 2.0 8.8 12 12_11/12 9 28 10-Dec-11 09_11/12 Dec-11 1,989,066 5,694,703 97.4% 28 4-Mar-12 31-Mar-12 191.2 92.7 7.3 2.3 9.1 13 13_11/12 10 28 7-Jan-12 28 1-Apr-12 28-Apr-12 171.8 88.7 7.0 2.1 8.5 1 10_11/12 Jan-12 2,043,901 5,275,158 96.4% 01_12/13 28 29-Apr-12 26-May-12 189.2 92.6 7.4 2.3 8.9 2 11 28 4-Feb-12 02_12/13 11_11/12 Feb-12 2,073,884 5,730,948 97.1% 28 27-May-12 23-Jun-12 176.5 89.7 7.0 2.2 8.6 3 03_12/13 12 28 3-Mar-12 12_11/12 Mar-12 2,604,341 5,559,547 95.9% 28 24-Jun-12 21-Jul-12 189.9 96.9 7.9 2.5 8.9 0.24 4 04_12/13 13 28 31-Mar-12 28 22-Jul-12 13_11/12 18-Aug-12 167.1 103.0 10.6 2.3 10.5 0.53 5 Mar-12 1,997,431 5,682,740 97.4% 05_12/13 28 19-Aug-12 15-Sep-12 177.0 94.0 8.7 2.4 9.8 0.38 6 1 28 28-Apr-12 06_12/13

E_TOOMANYOPTIONS

Prescriptive, Open

Storage

Consistent Hash (Route)

Ring Around The Rosie

Multi Data Center

CACHE

Read-Through

Write-Around

Excuse Me Miss

Search

Ridealong

Multi Service

Analysis

Ship The Compute?

Spread The Load

No Pets

Graph Reality

Window Seat

QUEUE

HOST

For More Information • London Datastore - http://data.london.gov.uk/ • Basho Data Platform - http://docs.basho.com/dataplatform/latest/ • Spark - http://spark.apache.org/ • Redis - http://redis.io • Kafka - http://kafka.apache.org/ • Solr - http://lucene.apache.org/solr/ • Riak - http://docs.basho.com/riak/latest/

REHASH?