11/23/2009 Examples of Data Stream Applications Continuous, - PDF document

11/23/2009 Examples of Data Stream Applications � Continuous, unbounded, rapid, time-varying streams of data elements (tuples). Data Streams � Market Analysis � Streams of Stock Exchange Data � Critical Care From Niagara Falls to Aurora Borealis � Streams of Vital Sign Measurements Cold stuff! � Physical Plant Monitoring � Streams of Environmental Readings � Biological Population Tracking � Streams of Positions from Individuals of a Species � � DSMS DSMS = Data Stream Management System DBMS versus DSMS The (Simplified) Big Picture � Persistent relations � Transient streams (and persistent relations) Streamed Stored � Continuous queries � One-time queries Register Result Result Query � Sequential access � Random access � Unpredictable data � Access plan determined by DSMS characteristics and arrival query processor and patterns Input streams physical DB design Archive Scratch Store Stored Relations stanfordstreamdatamanager 3 4 stanfordstreamdatamanager (Simplified) Network Monitoring ���������� ����������� ���� ���� Intrusion Warnings Data Active Data Passive ������������� Online Human Passive Human Active Register Performance Monitoring ����������������� Very hard or Metrics required Queries ��������� inefficient ������������ required �ot supported DSMS ������������ Network measurements, Packet traces ����������������� High Priority Low Priority Archive Scratch Store Lookup ���������� Tables required �ot supported ����������� 5 stanfordstreamdatamanager 1

11/23/2009 Discussion 1 Continuous Queries “Existing DBMS systems are ill suited for such applications since they target � One time queries – Run once to completion over the business applications.” Do you think implementing monitoring systems using current data set. DBMSs is reasonable? � If yes � How are traditional systems and monitoring systems similar? � Continuous queries – Issue once and continuously evaluate � Think of works and researches happened in DBMSs before, that Aurora over a changing data set. benefits from or inspired by? � Example: � If No � Notify me when the temperature drops below 30 deg. F � Which of those five assumptions is more problematic than others? � Notify me when prices of stock XYZ > $300 � 1. DBMSs have a HADP model � Popular paradigm among the users of Internet (has large � 2. Current state of the data is the only thing that is important � 3. Triggers and alerts are second-class citizens amounts of frequently changing information) � 4. Data elements are synchronized and that queries have exact answers � Allow users to receive new results when available without having to � 5. No real-time services issue same query repeatedly. � Think of alternative architectures or models that can be used for � Need to support millions of queries to scale to the Internet. monitoring applications? Discussion 2 NiagaraCQ: A Scalable Continuous � What are some of the challenges in building continues Query System for Internet Databases query processors for temporal and/or spatio-temporal data streams? ������� ��������������������������� ��������������� ���������������������������� ����������������������� !������ * Some of the examples of spatio-temporal applications are E911, traffic !���������"�����#�������������$������� monitoring, and location aware services dealing with moving objects What � s NiagaraCQ? Basics - Expression Signature � 1�����������������������/������������#��������#"������������ � %�������#���������#���������������&��������������#����� �����������"������������������&������� '!$�������������������&�����"��������"�(��'!$ )$� � */���������������������""�,�&�������,����������������������� �������������#��������������������������������������� � *��"���� � )�����+������� � +�������""�,�����-������������.�����,����������&����������#�� ������� � *�������������"�&����������&��������������������������"�������� �����/���������������-�������"��.� � �����������#����������������� � 0�""�����0��������"����������������������������������� ��������������� � ���������#�����������#���������������#������)�� �������������������������� ������������������������������� 2