VLDB Challenges VLDB Challenges in in Very Large Very Large - PowerPoint PPT Presentation

VLDB Challenges VLDB Challenges in in Very Large Very Large Enterprises Enterprises

Panelists Chair Chair • Dr. Michael L. Brodie, Chief Scientist, Verizon Problem Owners Problem Owners • Dr. Hans-Peter Steiert, Research & Technology, DaimlerChrysler AG Solution Owners Solution Owners • Adam Bosworth, VP, Engineering, BEA Systems Inc • James Hamilton, Architect, Microsoft SQL Server, Microsoft • Pat Selinger, IBM Fellow and VP, Data Management Architecture and Technology, IBM

Very Large Enterprises Em ployees Very Large Global Fortune Revenues Data ( 1 ,0 0 0 s) ( $ B US) Enterprise 5 0 0 5 0 0 Daim ler 7 N/ A $ 1 3 6 373 Chrysler Petabyte Verizon 2 6 1 1 $ 6 7 248 Problem Drivers Problem Drivers • Data Growth • Data Life Cycle

OLTP W orkload Grow th OLTP/sec Triples 2001-2004 Dec-04 Dec-02 Projected average workload Jan-01 0 1,000 2,000 3,000 OLTP Doubles by 12/04 Four Years 1/01 - 12/04 Projected Workload Growth Rate (average) Two Years 1/01 - 12/02 0% 40% 80% 120%

DSS W orkload Grow th Inflight Queries Double by 04 On 12/2004 On 12/2002 Projected workload (average) (concurrent inflight queries) On 1/2001 0 25 50 75 100 125 150 DSS Workload Triples by 04 4 Yr growth Projected workload growth rate (average) 2 Yr Growth 0% 20% 40% 60%

Database Grow th Size: OLTP Doubles; DSS Triples by 04 On 12 2004 OLTP On 12 2002 DSS On Jan 2001 OLTP N = 43 DSS N = 67 0 1 2 3 4 5 6 Respondents' Projected Database Size (average) (TB) Growth: DSS & OLTP Double by 04 Jan 01 - Dec 04 OLTP DSS Jan 01 - Dec 02 0% 50% 100% 150% Respondents' Projected Database Growth Rate (average)

VLE Storage Grow th VLE X Storage Triples 02-06 3500 3000 2500 Terabytes IP SAN/NAS 2000 OS Vertical 1500 OS Shared 1000 FC SAN 500 System390 0 2001 2002 2003 2004 2005 2006 Calendar Year

Data Life Cycle Life Cycle Actions Life Cycle Actions • Create • Store • Replicate • Protect • Update Data Droppings Problem • Archive • Exchange, exchange, exchange, … Factors Factors • History: 40+ years of Mergers & Acquisitions • Growth: Automation & Partnering • Protection: security, confidentiality, … • The Grand Challenge: semantics of data

Grand Challenge: $ 1 Trillion/ year Integration Cost Estimates Integration Cost Estimates • 24% of IT budgets: $180 B / year US (InfoWorld, January 2002) • 13% of IT spend: $752 B / year US (Giga estimate; May 2002) • 25-40% of all IT projects (various) • 6% of US IT spending: $610B / year US (IDC, May 2002) • 7% of IT spending: $1.3T / year worldwide (IDC, May 2002) • 28+% of worldwide consulting: $ 160 B/year (Gartner, March 2002) • 43% of e-business worldwide consulting: $53 B / year (Gartner) • 1.75% to annual IT budget on EAI and B2Bi (Forrester, Dec 2001) • 10-30% of IT budgets (David Sink, IBM, InformationWeek, May 27, 2002) Data Quality Cost Estimates Data Quality Cost Estimates • $600 B / year US (Data Warehouse Institute, 2002)

VLE VLDB Challenges Data Management Data Management • Global Data Managem ent – Significant improvement in dealing automatically with semantics • Database Engineering • Automated DBA • Com prehensive Data Managem ent Architecture • Data architecture: Web Services, mid-tier, distributed data Storage Management Storage Management • Data Protection – Integrated products: DBMS, replication, … • Data Utilization – Automated DBA, Storage Virtualization, Hierarchical Storage Management for distributed system