Bigger Metad Big Data = OReilly Strata Conference February 29 2012 - PowerPoint PPT Presentation

Bigger Metad Big Data = O’Reilly Strata Conference February 29 2012

Pivot/Skate, etc ! � Founded 2003 Poor man’s GIS Panamap � Refounded 2006 Neighborhood boundaries Mass transit data � Refocused 2009 SaaS for mapping + on-demand data

Achtung! NoSQL is no panacea � Big Data isn’t about data � Big Data isn’t new � Big Data doesn’t present a Boolean quandary � With power comes responsibility AWS bills Lady Gaga tweets Innumeracy (correlation v causation)

Big v Important Big Important � Heterogeneous � Well-defined schema � Raw � High value (not free) � Distributed � Test-driven � Streaming/real time � Relational � Search for meaning � Historical � Time-sensitive � Enterprise-focused � Philosophical?

Data Exhaust

Platforms

A Brief History of Metadata Callimachus Library of Alexandria, Egypt

A Brief History of Metadata “Pinakes” (lists) � Title � Category � Author � Author birthplace � Father � Word count Callimachus

A Brief History of Metadata

A Brief History of Metadata

A Brief History of Metadata

A Brief History of Metadata

Where Is It?

Why Can’t We Be Together? Metadata Data

Recap Data Database Pinakes Catalog Taxonomy 300 BC 1595 AD 1876 1970

Oh, How I’ve Missed You

Together At Last

GIS Data is Unevolved + � = �

Enter the Data Curator

DIKW Model Data � Fact, Signal, Symbol Information � Structural v Functional � Symbolic v Subjective Knowledge � Processed � Procedural � Propositional

Popularity (Google Trends) &!" ">-?(6(?(" %#" "@A:<"6(?(@" 2-(3.4"5)6-7"89:7-6"2.(;:)<=" ".B3(CD)" %!" "@6(?("E.:-)CE?@" $#" $!" #" !" '()"*"%!!*" '()"%"%!!#" '()"$"%!!+" ,-."&$"%!!+" ,-."&!"%!!/" ,-."%0"%!!0" ,-."%/"%!!1" ,-."%+"%!$!" ,-."%#"%!$$"

I � Metadata One Person’s Metadata is Another Person’s Data

Thank you! ian@urbanmapping.com @urbanmapping R.I.P. Schema