Approximate Query Service on Autonomous IoT Cameras Mengwei Xu 1 , - PowerPoint PPT Presentation

Approximate Query Service on Autonomous IoT Cameras Mengwei Xu 1 , Xiwen Zhang 2 , Yunxin Liu 3 Gang Huang 1 , Xuanzhe Liu 1 , Felix Xiaozhu Lin 2 1 Peking University, 2 Purdue University, 3 Microsoft Research, 1

Video Analytics is a Killer App • Busy cross roads • Retailing store • Sports stadium • Parking lots • … 2

Video Analytics is a Killer App • Busy cross roads • Retailing store • Sports stadium Urban, residential areas • Parking lots ü Wired electricity • … ü Good internet 3

Video Analytics is a Killer App • Busy cross roads • Retailing store • Sports stadium • Parking lots ? • … • Construction sites • Cattle farms • Highways • Wildlifes • … Urban, residential areas Rural, off-grid areas 4

Autonomous Camera • Energy-independent and Compute-independent 5

Autonomous Camera • Energy-independent and Compute-independent Commodity SoCs, RPI-like, chargeable battery 6

Autonomous Camera • Energy-independent and Compute-independent Small-sized energy harvester e.g., “10Wh today” Commodity SoCs, RPI-like, chargeable battery 7

Autonomous Camera • Energy-independent and Compute-independent LPWAN like LoRaWAN, At most a few Kbps Small-sized energy harvester e.g., “10Wh today” Commodity SoCs, RPI-like, chargeable battery 8

Autonomous Camera • Energy-independent and Compute-independent LPWAN like LoRaWAN, At most a few Kbps Small-sized energy harvester Concise, numerical video summaries e.g., “10Wh today” Commodity SoCs, RPI-like, chargeable battery 9

Elf for Autonomous Cameras • Target video query: object counting Object Counts 10

Elf for Autonomous Cameras • Target video query: object counting Query: (car, 30 mins) Install 11

Elf for Autonomous Cameras • Target video query: object counting Query: (car, 30 mins) Install Sample & capture 12

Elf for Autonomous Cameras • Target video query: object counting Query: (car, 30 mins) 7:00AM-7:30AM [500 + 100] Cars 7:30AM-8:00AM [700 + 140] Cars Install 8:00AM-8:30AM [800 + 180] Cars 8:30AM-9:00AM [400 + 100] Cars Sample & capture 9:30AM-10:00AM [200 + 80] Cars 13

Elf for Autonomous Cameras • Target video query: object counting with confidence interval (CI) Query: (car, 30 mins) 7:00AM-7:30AM [500 + 100] Cars 7:30AM-8:00AM [700 + 140] Cars Install 8:00AM-8:30AM [800 + 180] Cars 8:30AM-9:00AM [400 + 100] Cars Sample & capture 9:30AM-10:00AM [200 + 80] Cars -80 200 +80 14

Elf for Autonomous Cameras • Target video query: object counting with confidence interval (CI) • The central problem: planning constrained energy for counting o Energy model: a budget that cannot be exceeded in a horizon (e.g., 24 hrs) o Trade-offs: frame sampling and NN selection o Target: smallest mean CI widths across all (30-min) windows in a horizon 15

Elf Overview Check our paper for details Aggregator Object counts with error with CIs Integration Sampled frames Selected Neural Net Reinforcement Learning Planner Focus of the talk Camera Operating System 16

Elf tech #1: per-window energy/CI fronts • What’s the best count action for a window? • A count action : determining (1) an NN and (2) # of frames to process 350 CI width (smaller, better) Energy Consumption = E ( NN ) * frame _ num 300 YOLOv3 250 200 150 100 0 2 4 Energy Consumption (kJ) 17

Elf tech #1: per-window energy/CI fronts • What’s the best count action for a window? • A count action : determining (1) an NN and (2) # of frames to process 350 YOLOv3 CI width (smaller, better) Energy Consumption = E ( NN ) * frame _ num YOLOv2 300 ssd-inception-v2 ssd-mobilenet-v2 250 200 150 100 0 2 4 Energy Consumption (kJ) 18

Elf tech #1: per-window energy/CI fronts • What’s the best count action for a window? No silver bullet. • A count action : determining (1) an NN and (2) # of frames to process 350 YOLOv3 CI width (smaller, better) When energy is low: cheaper NNs win YOLOv2 300 Bottlenecked by sampling error (frame quantity) • ssd-inception-v2 ssd-mobilenet-v2 250 200 150 100 0 2 4 Energy Consumption (kJ) 19

Elf tech #1: per-window energy/CI fronts • What’s the best count action for a window? No silver bullet. • A count action : determining (1) an NN and (2) # of frames to process 350 YOLOv3 CI width (smaller, better) When energy is low: cheaper NNs win YOLOv2 300 Bottlenecked by sampling error (frame quantity) • ssd-inception-v2 When energy is low: more accurate NNs win ssd-mobilenet-v2 250 Bottlenecked by NN error (frame quality) • 200 150 100 0 2 4 Energy Consumption (kJ) 20

Elf tech #1: per-window energy/CI fronts • What’s the best count action for a window? No silver bullet. • A count action : determining (1) an NN and (2) # of frames to process 350 YOLOv3 CI width (smaller, better) When energy is low: cheaper NNs win YOLOv2 300 Bottlenecked by sampling error (frame quantity) • ssd-inception-v2 When energy is low: more accurate NNs win ssd-mobilenet-v2 250 Bottlenecked by NN error (frame quality) • 200 150 100 0 2 4 Energy Consumption (kJ) 21

Elf tech #1: per-window energy/CI fronts • What’s the best count action for a window? No silver bullet. • A count action : determining (1) an NN and (2) # of frames to process 350 YOLOv3 CI width (smaller, better) When energy is low: cheaper NNs win YOLOv2 300 When energy is low: more accurate NNs win ssd-inception-v2 ssd-mobilenet-v2 250 Energy/CI front: the combination of all 200 “optimal” count actions with varied energy 150 100 0 2 4 Energy Consumption (kJ) 22

Elf tech #1: per-window energy/CI fronts • What’s the best count action for a window? No silver bullet. • A count action : determining (1) an NN and (2) # of frames to process 350 YOLOv3 CI width (smaller, better) When energy is low: cheaper NNs win YOLOv2 300 When energy is low: more accurate NNs win ssd-inception-v2 ssd-mobilenet-v2 250 Energy/CI front: the combination of all 200 “optimal” count actions with varied energy 150 • How to construct? Error integration • Depends on the video characteristics 100 0 2 4 Energy Consumption (kJ) 23

Elf tech #1: per-window energy/CI fronts • What’s the best count action for a window? No silver bullet. • A count action : determining (1) an NN and (2) # of frames to process 9:00AM-9:30AM 7:00PM-7:30PM When energy is low: cheaper NNs win 350 350 When energy is low: more accurate NNs win YOLOv3 YOLOv2 300 300 ssd-inception-v2 ssd-mobilenet-v2 250 250 Energy/CI front: the combination of all 200 200 150 150 “optimal” count actions with varied energy 100 100 • How to construct? Error integration 0 2 4 0 2 4 • Depends on the video characteristics Different windows have different energy/CI fronts 24

Elf tech #2: across-window joint planning • An Oracle Planner: best performance but unrealistic • knows all energy/CI fronts 25

Elf tech #2: across-window joint planning • An Oracle Planner: best performance but unrealistic • knows all energy/CI fronts An oracle planner Energy: 100% A greedy approach: giving energy to the window with the most Allocate 10% benefit (i.e., CI width reduction). [500 � 80] [450 � 70] [600 � 95] [580 � 75] [50 � 55] 26

Elf tech #2: across-window joint planning • An Oracle Planner: best performance but unrealistic • knows all energy/CI fronts An oracle planner Energy: 90% A greedy approach: giving energy to the window with the most Allocate 10% benefit (CI width reduction) [500 � 80] [450 � 70] [510 � 60] [580 � 75] [50 � 55] 27

Elf tech #2: across-window joint planning • An Oracle Planner: best performance but unrealistic • knows all energy/CI fronts An oracle planner Energy: 80% A greedy approach: giving energy to the window with the most Allocate 10% benefit (CI width reduction) [500 � 60] [450 � 70] [580 � 75] [400 � 55] [50 � 55] 28

Elf tech #2: across-window joint planning • An Oracle Planner: best performance but unrealistic • knows all energy/CI fronts An oracle planner Energy: 80% A greedy approach: giving energy to the window with the most benefit (CI width reduction) [500 � 60] [400 � 55] [580 � 75] [50 � 55] unknown 29

Elf tech #2: across-window joint planning • An Oracle Planner: best performance but unrealistic • knows all energy/CI fronts • planned offline • A learning-based planner: imitating the oracle planner • basis: reinforcement learning Classification • rationale: daily and temporal patterns Agent Counter Observations Compared with (M + N windows) oracle’s decision for 𝑿 𝒖 and run Regression reward function Agent # Frames Reward 𝜈 𝑢−4 𝜈 𝑢−3 𝜈 𝑢−2 𝜈 𝑢−1 𝜈 𝑢 Oracle Planner 𝜏 𝑢−4 𝜏 𝑢−3 𝜏 𝑢−2 𝜏 𝑢−1 𝜏 𝑢 …… …… 30 Recent N days Recent M windows