Querying Sensor Networks Sam Madden 1 Sensor Networks Small - PowerPoint PPT Presentation

Querying Sensor Networks Sam Madden 1

Sensor Networks • Small computers with: – Radios – Sensing hardware – Batteries • Remote deployments – Long lived – 10s, 100s, or 1000s Battery Pack Smart Sensor, aka “ Mote ” 2

Motes Mica Mote 4Mhz, 8 bit Atmel RISC uProc 40 kbit Radio 4 K RAM, 128 K Program Flash, 512 K Data Flash AA battery pack Based on TinyOS* *Hill, Szewczyk, Woo, Culler, & Pister. “Systems Architecture Directions for Networked Sensors.” ASPLOS 2000. http://webs.cs.berkeley.edu/tos 3

Sensor Net Sample Apps Habitat Monitoring: Storm petrels on Great Duck Island, microclimates on James Reserve. Earthquake monitoring in shake- test sites. Vehicle detection: sensors along a road, collect data about passing vehicles. Traditional monitoring apparatus. 4

Programming Sensor Nets Is Hard – Months of lifetime required from small batteries » 3-5 days naively; can’t recharge often » Interleave sleep with processing – Lossy, low-bandwidth, short range communication »Nodes coming and going 200-800 instructions High-Level Abstraction Is Need high level »~20% loss @ 5m per bit transmitted! Needed! abstractions! »Multi-hop – Remote, zero administration deployments – Highly distributed environment – Limited Development Tools »Embedded, LEDs for Debugging! 5

A Solution: Declarative Queries • Users specify the data they want – Simple, SQL-like queries – Using predicates, not specific addresses – Same spirit as Cougar – Our system: TinyDB • Challenge is to provide: – Expressive & easy-to-use interface – High-level operators » Well-defined interactions » “Transparent Optimizations” that many programmers would miss • Sensor-net specific techniques – Power efficient execution framework • Question: do sensor networks change query processing? Yes! 6

Overview • TinyDB: Queries for Sensor Nets • Processing Aggregate Queries (TAG) • Taxonomy & Experiments • Acquisitional Query Processing • Other Research • Future Directions 7

Overview • TinyDB: Queries for Sensor Nets • Processing Aggregate Queries (TAG) • Taxonomy & Experiments • Acquisitional Query Processing • Other Research • Future Directions 8

TinyDB Demo 9

TinyDB Architecture SELECT T:1, AVG: 225 AVG(temp) Queries T:2, AVG: 250 Results WHERE light > 400 Multihop Schema: Network •“Catalog” of commands & Query Processor attributes Agg avg(temp) ~10,000 Lines Embedded C Code Filter light > Name: temp Time to sample: 50 uS ~5,000 Lines (PC-Side) Java 400 got(‘temp’) get (‘temp’) Tables Samples Cost to sample: 90 uJ ~3200 Bytes RAM (w/ 768 byte heap) Schema Calibration Table: 3 getTempFunc(…) Units: Deg. F ~58 kB compiled code TinyOS Error: ± 5 Deg F (3x larger than 2 nd largest TinyOS Program) Get f : getTempFunc() … TinyDB 10

Declarative Queries for Sensor Networks “Find the sensors in bright nests.” • Examples: Sensors 1 Epoch Nodeid nestNo Light SELECT nodeid, nestNo, light FROM sensors 0 1 17 455 WHERE light > 400 0 2 25 389 EPOCH DURATION 1s 1 1 17 422 1 2 25 405 11

Aggregation Queries “Count the number occupied SELECT AVG(sound) 2 nests in each loud region of FROM sensors the island.” EPOCH DURATION 10s Epoch region CNT(…) AVG(…) SELECT region, CNT(occupied) 3 0 North 3 360 AVG(sound) 0 South 3 520 FROM sensors 1 North 3 370 GROUP BY region 1 South 3 520 HAVING AVG(sound) > 200 EPOCH DURATION 10s Regions w/ AVG(sound) > 200 12

Overview • TinyDB: Queries for Sensor Nets • Processing Aggregate Queries (TAG) • Taxonomy & Experiments • Acquisitional Query Processing • Other Research • Future Directions 13

Tiny Aggregation (TAG) • In-network processing of aggregates – Common data analysis operation » Aka gather operation or reduction in || programming – Communication reducing » Operator dependent benefit – Across nodes during same epoch • Exploit query semantics to improve efficiency! Madden, Franklin, Hellerstein, Hong. Tiny AGgregation (TAG), OSDI 2002. 14

Query Propagation Via Tree-Based Routing Q:SELECT … • Tree-based routing A – Used in: Q R:{…} Q R:{…} » Query delivery » Data collection Q – Topology selection is B C Q important; e.g. » Krishnamachari, DEBS Q R:{…} Q Q 2002, Intanagonwiwat, ICDCS 2002, Heidemann, R:{…} D Q SOSP 2001 » LEACH/SPIN, R:{…} Q Q Q Heinzelman et al. MOBICOM 99 F » SIGMOD 2003 E Q – Continuous process » Mitigates failures 15

Basic Aggregation • In each epoch: – Each node samples local sensors once 1 – Generates partial state record (PSR) » local readings » readings from children 2 3 – Outputs PSR during assigned comm. interval 4 • At end of epoch, PSR for whole network output at root 5 • New result on each successive epoch • Extras: – Predicate-based partitioning via GROUP BY 16

Illustration: Aggregation SELECT COUNT(*) Interval 4 FROM sensors Sensor # 1 Epoch 1 2 3 4 5 4 1 2 3 3 Interval # 2 4 1 1 4 5 17

Illustration: Aggregation SELECT COUNT(*) Interval 3 FROM sensors Sensor # 1 1 2 3 4 5 4 1 2 3 3 2 Interval # 2 2 4 1 4 5 18

Illustration: Aggregation SELECT COUNT(*) Interval 2 FROM sensors Sensor # 1 1 3 1 2 3 4 5 4 1 2 3 3 2 Interval # 2 1 3 4 1 4 5 19

Illustration: Aggregation SELECT COUNT(*) 5 Interval 1 FROM sensors Sensor # 1 1 2 3 4 5 4 1 2 3 3 2 Interval # 2 1 3 4 1 5 4 5 20

Illustration: Aggregation SELECT COUNT(*) Interval 4 FROM sensors Sensor # 1 1 2 3 4 5 4 1 2 3 3 2 Interval # 2 1 3 4 1 5 1 4 1 5 21

Interval Assignment: An Approach SELECT 4 intervals / epoch COUNT(*)… Comm Interval Interval # = Level 2 5 4 3 1 5 L T Z Z 1 Z Z Z Z Z Z Level = 1 Z Z Z Z • CSMA for collision Epoch avoidance T L T Z Z Z Z L Z Z Z Z 2 Z Z Z Z Z Z Z Z 2 3 Z Z Z Z Z Z Z Z • Time intervals for power conservation L T Z Z Z Z Z Z Z Z Z Z Z Z 4 3 • Many variations( e.g. Yao & Gehrke, CIDR 2003 ) Z Z T Z Z Z Z L L Z Z Z Pipelining : Increase throughput by delaying 5 • Time Sync (e.g. Elson & 4 result arrival until a later epoch Estrin OSDI 2002) Madden, Szewczyk, Franklin, Culler. Supporting Aggregate Queries Over Ad-Hoc Wireless Sensor 22 Networks. WMCSA 2002 .

Aggregation Framework • As in extensible databases, we support any aggregation function conforming to: Agg n ={f init , f merge , f evaluate } F init {a 0 } → <a 0 > Partial State Record (PSR) F merge {<a 1 >,<a 2 >} → <a 12 > F evaluate {<a 1 >} → aggregate value Example: Average AVG init {v} → <v,1> AVG merge {<S 1 , C 1 >, <S 2 , C 2 >} → < S 1 + S 2 , C 1 + C 2 > AVG evaluate {<S, C>} → S/C Restriction: Merge associative, commutative 23

Types of Aggregates • SQL supports MIN, MAX, SUM, COUNT, AVERAGE • Any function over a set can be computed via TAG • In network benefit for many operations – E.g. Standard deviation, top/bottom N, spatial union/intersection, histograms, etc. – Compactness of PSR 24

Overview • TinyDB: Queries for Sensor Nets • Processing Aggregate Queries (TAG) • Taxonomy & Experiments • Acquisitional Query Processing • Other Research • Future Directions 25

Simulation Environment • Evaluated TAG via simulation • Coarse grained event based simulator – Sensors arranged on a grid – Two communication models » Lossless: All neighbors hear all messages » Lossy: Messages lost with probability that increases with distance • Communication (message counts) as performance metric 26

Taxonomy of Aggregates • TAG insight: classify aggregates according to various functional properties – Yields a general set of optimizations that can automatically be applied Properties Drives an API! Partial State Monotonicity Exemplary vs. Summary Duplicate Sensitivity 27

Partial State • Growth of PSR vs. number of aggregated values (n) – Algebraic: |PSR| = 1 (e.g. MIN) “Data Cube”, – Distributive: |PSR| = c (e.g. AVG) Gray et. al – Holistic: |PSR| = n (e.g. MEDIAN) – Unique: |PSR| = d (e.g. COUNT DISTINCT) » d = # of distinct values – Content Sensitive: |PSR| < n (e.g. HISTOGRAM) Property Examples Affects Partial State MEDIAN : unbounded, Effectiveness of TAG MAX : 1 record 28

Benefit of In-Network Processing Simulation Results 2500 Nodes 50x50 Grid Holistic Depth = ~10 Unique Neighbors = ~20 • Aggregate & depth dependent benefit! Uniform Dist. Distributive Algebraic 29

Monotonicity & Exemplary vs. Summary Property Examples Affects Partial State MEDIAN : unbounded, Effectiveness of TAG MAX : 1 record Monotonicity COUNT : monotonic Hypothesis Testing, Snooping AVG : non-monotonic Exemplary vs. MAX : exemplary Applicability of Sampling, Summary Effect of Loss COUNT: summary 30