The Prices of Packets and Watts: Optimal Operation of Decentralized - PowerPoint PPT Presentation

The Prices of Packets and Watts: 
 Optimal Operation of Decentralized Stochastic Systems P. R. Kumar Dept. of Electrical and Computer Engineering Texas A&M University With Rahul Singh and Le Xie Rahul Singh 6th IFAC Workshop on Distributed Estimation and Control in Networked Systems (NecSys’16), Email: prk.tamu@gmail.com Sep 8, 2016 Web: http://cesg.tamu.edu/faculty/p-r-kumar/ 1 /43 Tokyo, Japan

Two problems ◆ Optimal scheduling of unreliable networks with hard end- to-end deadlines ◆ Optimal pricing and scheduling of generators and loads ◆ Both are stochastic distributed control problems ◆ Both have a solution based on “price” ◆ PhD Thesis of Rahul Singh, 2015 2 /43

Unreliable multi-hop networks with 
 end-to-end delay constraints 3 /43

Multi-hop network τ f Flow f Delivery rate r f ◆ F flows ◆ Flow f has an end-to-end deadline τ f ◆ Let r f = packet delivery rate of flow f (Timely throughput) 4 /43

Nodal constraints c i j p ij i ◆ Node i has an average power constraint T 1 ∑ ( ) lim # of packets transmitted by node i at time t ≤ c i T T →∞ t = 1 ◆ Packet transmission succeeds with probability p ij ◆ No interference: Directional antennas 5 /43

Challenge of scheduling a distributed system Congestion Flow 1 Transmit packet from 
 Flow 2 since Flow 1 has 
 downstream congestion Flow 2 No Congestion ◆ Does optimal scheduling require knowledge of the complete network state? ◆ Obtaining network state instantaneously itself requires solving end-to-end delay problem ◆ Even if each node could obtain complete state, DP is intractable – Huge state space: ( V Δ ) F Δ ◆ Is optimal scheduling of this distributed system difficult? 6 /43

Objective ∑ Max α f r f f Where: r f = Throughput of packets of flow f that have an end-to-end delay ≤ τ f = Timely throughput of flow f ◆ The timely throughput of flow f is weighted by α f ◆ How to schedule the network? 7 /43

Solution ◆ Constrained optimization problem over stationary randomized policies π T 1 ∑ ∑ Max limsup α f (# of packets of flow f delivered in time at time t ) T π T →∞ t = 1 f T 1 ∑ ( ) Subject to lim # of packets transmitted by node i at time t ≤ c i T T →∞ t = 1 L ( π , λ ) ◆ Lagrangian ⎧ T ⎪ 1 ∑ ∑ Max limsup α f (# of packets of flow f delivered in time at time t ) ⎨ T π ⎪ T →∞ ⎩ t = 1 f T ⎫ ∑ ∑ ( ) − λ i # of packets transmitted by node i at time t λ i c i ⎬ + ⎭ t = 1 i ◆ Packet-by-Packet Decoupling 1 ∑ ∑ { Max limsup α f 1(Packet is delivered on time) T π T →∞ f Packets of flow f released before time T ⎫ ∑ − λ i 1(Packet is transmitted by Node i ) ⎬ ⎭ 8 /43 i

Packet level decision making ∑ α f 1(Packet is delivered on time) − λ i 1(Packet is transmitted by Node i ) i Packet state ( i , τ ) j α f d p ij i Pay λ i ? V ( d , t ) = α f for all t ≥ 0 { } V ( i , τ ) = Min λ i + p i , j V ( j , τ − 1) + (1 − p i , j ) V ( i , τ − 1), V ( i , τ − 1) ◆ Packet solves Dynamic Program offline ◆ Easy to solve: Packet state space size is V Δ 9 /43

Optimal distributed solution ◆ The optimal solution completely decouples! ◆ Each packet makes decision to be transmitted or not, depending only on its own state ( location, time to deadline ) ◆ Optimal scheduling of a packet does not depend on – State of other nodes – State of other flows – Even other packets within its own flow! 10 /43

How to obtain prices? ◆ If price λ i is too low – Too many packets ask to be transmitted – Average power λ i constraint is exceeded ◆ If price λ i is too high – Too few packets ask to be transmitted – Average power available λ i is not used ◆ Suggests tatonnement n + 1 = λ i n + ε [Power consumed by node i − c i ]. λ i ◆ But even if price is exactly right, we will need to randomize some flow’s decisions to get the power to be exactly used up 11 /43

Dual Problem D ( λ ) = max π L ( π , λ ) ◆ The Dual function is ◆ “Max” is attained by Single Packet Transportation Problem ◆ Dual Problem is max λ ≥ 0 D ( λ ) ◆ No Duality Gap, since can be reduced to LP 12 /43

Optimality condition ◆ Suppose λ * is price vector ◆ π ( λ *) optimal randomized policy for single-packet transportation problem for each flow f ◆ Suppose at every node i , – Either power constraint is satisfied with equality by π ( λ *) T 1 ∑ ( ) lim # of packets transmitted by node i at time t = c i T T →∞ t = 1 – Or λ i *=0 ◆ Then π ( λ *) and λ * are optimal by Complementary Slackness 13 /43

Combine Single-Packet Transportation Problems of all Flows τ f Use state-action probabilities ξ f ( i , d f , s ) p i , d f ∑ ∑ ∑ max A f Reward f ∈ F s = 0 i ∈ V τ f ξ f ( i , j , s ) E ≤ P ∑ ∑ ∑ A f ∀ i ∈ V , Power Constraint i f ∈ F s = 0 j ≠ i ξ f ( i , m , s )(1 − p i , m ) ∑ ξ f ∑ ( j , i , s ) p j , i + Balance equations j ∈ V , j ≠ d f m ∈ V ∑ ξ f ( i , k , s − 1) ∀ i ≠ d f ,1 ≤ s ≤ τ f = k ∈ V ξ f ( i , j , s ) ≥ 0 ∑ ξ f ( s f , j ,0) = 1 ∀ f , Probabilities j ∈ V ◆ Very tractable LP solution: Low complexity | V | 2 F Δ | V | + | V | F Δ + F + | V | 2 F Δ ◆ Just variables , constraints ( V Δ ) F Δ ◆ Reduction from exponential complexity 14 /43


 Near Optimality for Peak Power Constraint c i j p ij ◆ Suppose Node i can only 
 i transmit c i packets 
 concurrently ◆ Simply truncate at c i – Similar to Whittle’s relaxation for restless bandits ◆ Theorem ⎛ ⎞ 1 ◆ Policy is asymptotically optimal as the total network O ⎜ ⎟ ⎝ N ⎠ capacity is scaled by N 15 /43

Comparison with Backpressure Policy ◆ Important development in scheduling over past 25 years ◆ Max Weight and Backpressure Policies – Tassiulas and Ephremides (1992), Neely, Modiano and Rohrs (2003), Lin and Shroff (2004), Lin, Shroff and Srikant (2006) ◆ Backpressure Policy is based on a Lagrangian decomposition of a fluid model ◆ Fluid model is appropriate for studying throughput – Successful design of throughput optimal policies ◆ But Delay depends on stochastic variations – needs stochastic model – A la difference between LLN and CLT ◆ Resulting Lagrange Multipliers are very different – Difference in queue lengths, i.e., backpressure, for fluid model – Price for energy of transmission in stochastic model 16 /43

Example: Explicit solution ◆ Optimal solution τ 1 = τ 2 = 3 β 1 = 5 Flow 1 p (1,2) =0.4 p (2,3) =0.3 1 2 3 ◆ Optimal prices p (2,1) =0.7 p (3,2) =0.6 Flow 2 β 2 = 2 P 1 =0.5 P 2 =0.4 P 3 =0.5 ( ) λ = 0.068,1.4,0 Flow 1: Node 1 transmit with 
 probability 0.5 if 
 ◆ Optimal solution time-till-deadline is 3, else drop ⇡ 1 (1 , 3) = 0 . 5 , ⇡ 1 (1 , 2) = 0 , Flow 2: ⇡ 1 (2 , 2) = 1 , ⇡ 1 (2 , 1) = 1 , Node 3 transmit with 
 ⇡ 2 (3 , 3) = 1 / 13 , ⇡ 2 (3 , 2) = 0 , probability 1/13 if 
 time-till-deadline is 3, else drop ⇡ 2 (2 , 2) = 1 , ⇡ 2 (2 , 1) = 1 . Node 2: Both Flows Transmit optimal single-packet transportation dynamic 17 /43

Example: Numerical Computation and Simulation Comparison Flow 2 ◆ Comparison with 
 p (2,3) = 0.5 p (3,4) = 0.5 Flow 1 β 1 = 1 p (1,2) = 0.5 EDF-BP 
 1 3 4 2 β 2 = 1 and 
 EDF-SP p (2,5) = 0.5 p (6,4) = 0.5 5 6 p (5,6) = 0.5 ◆ A 1 = A 2 =1 0.7 Optimal Policy ◆ C ( i , j ) = 1 EDF-BP 0.6 EDF-SP Timely Throughput 0.5 ◆ Δ 1 = Δ 2 +1 0.4 0.3 0.2 0.1 0 3 4 5 6 7 8 9 10 11 12 18 /43 Relative Deadline of Flow 2

Remarks ◆ Issues not considered here – Contention, interference, coding ◆ FCC announcement on July 14, 2016 ◆ 10.85 GHz spectrum in millimeter band released by FCC – 3.85 GHz licensed – 7 GHz unlicensed ◆ Another 18GHz unlicensed proposed to be released ◆ Advanced wireless initiative in US on city-scale testbeds ◆ Wireless information era coming? 19 /43

The Independent System Operator Problem in Power Systems 20 /43

The ISO problem in a simple static context Supply ◆ ISO has to balance supply and demand Price Generator ◆ Generator and load bid their supply and demand curves ISO ◆ ISO intersects to find right price Demand Load Supply Demand u Price Price p 21 /43

Uncertainties and dynamics in the era of renewables and demand response Generation Time Nuclear Coal power Wind farm Hydropower power plant plant plant • Dynamic constraints: ramping, thermal inertia • Uncertainty: Wind, temperature, water flow • All choices have costs/benefits • ISO How can ISO ensure maximum social welfare? • How much should be generated, and balance • How should generators and loads bid? Load Commerci Industrial Storage serving al load load service entity Consumption Time 22 /43

The ISO problem Load 3 Generator 1 x 3 ( t + 1) = f 3 ( x 3 ( t ), u 3 ( t ), w 3 ( t )) x 1 ( t + 1) = f 1 ( x 1 ( t ), u 1 ( t ), w 1 ( t )) ( x 3 ( t ), u 3 ( t )) ∈ G 3 ( t ) ( x 1 ( t ), u 1 ( t )) ∈ G 1 ( t ) T T ∑ Max E U 3 ( x 3 ( t ), u 3 ( t )) ∑ Max E U 1 ( x 1 ( t ), u 1 ( t )) t = 0 t = 0 ISO N T ∑ ∑ Max E U i ( x i ( t ), u i ( t )) Load 
 Generator 
 4 i = 1 t = 0 2 N ∑ Balance: u i ( t ) = 0 for all t i = 1 How to assign u i (t)’s ? Storage 
 Without knowledge of: Prosumer 
 5 6 • States x i (t) • Models f i (x i ,u i ,w i ) • Utilities U i (x i ,u i ) 23 /43