Motivation and challenges Generator Outage (Governor) Secondary Primary Tertiary Frequency Inertia Three or Four Phases to Recovery 50Hz Fault Fault+20sec Fault+30mins Tertiary Secondary Power deviation Primary (Governor) 0 • Primary Frequency Reserve • Secondary Frequency Reserve • Tertiary Frequency Reserve Crisis-aversion and reallocation of resources: just one role for the grid operator.
Motivation and challenges Ancillary Services ... to compensate for energy imbalances ... Kirby 2013 October 20-25 October 27 - November 1 8 8 Load Generation and Laod GW 6 6 Generation Hydro GW Thermal Wind 4 4 2 2 0 0 1 1 ~ Error Signal in Feedback Loop Balancing GW Balancing 0.8 0.8 Reserves Deployed GW 0 0 -0.8 -0.8 -1 -1 Sun Mon Tue Wed Thur Fri Sun Mon Tue Wed Thur Fri
Motivation and challenges Ancillary Services ... to compensate for energy imbalances ... Kirby 2013 October 20-25 October 27 - November 1 8 8 Load Generation and Laod GW 6 6 Generation Hydro GW Thermal Wind 4 4 2 2 0 0 1 1 ~ Error Signal in Feedback Loop Balancing GW Balancing 0.8 0.8 Reserves Deployed GW 0 0 -0.8 -0.8 -1 -1 Sun Mon Tue Wed Thur Fri Sun Mon Tue Wed Thur Fri Time-scale of power deviations are similar to secondary reserves following a fault
Motivation and challenges Ancillary Services ... to compensate for energy imbalances ... Kirby 2013 October 20-25 October 27 - November 1 8 8 Load Generation and Laod GW 6 6 Generation Hydro GW Thermal Wind 4 4 2 2 0 0 1 1 ~ Error Signal in Feedback Loop Balancing GW Balancing 0.8 0.8 Reserves Deployed GW 0 0 -0.8 -0.8 -1 -1 Sun Mon Tue Wed Thur Fri Sun Mon Tue Wed Thur Fri Time-scale of power deviations are similar to secondary reserves following a fault The Balancing Reserves at BPA are a sum of many error signals in the grid
Motivation and challenges NERC Coordinating Councils NERC Regional Entitites FRCC Florida Reliability Coordina ng Council MRO Midwest Reliability Organiza on NPCC Northeast Power Coordina ng Council RFC ReliabilityFirst Corpora on SERC SERC Reliability Corpora on SPP RE Southwest Power Pool Regional En ty TRE Texas Reliability En ty WECC Western Electric Coordina ng Council Interconnec on Requirements for Variable Genera on - September 2012 From Operating Manual – North American Energy Standards Board (NAESB)
Motivation and challenges NERC Coordinating Councils NERC Regional Entitites FRCC Florida Reliability Coordina ng Council MRO Midwest Reliability Organiza on NPCC Northeast Power Coordina ng Council RFC ReliabilityFirst Corpora on SERC SERC Reliability Corpora on SPP RE Southwest Power Pool Regional En ty TRE Texas Reliability En ty WECC Western Electric Coordina ng Council Interconnec on Requirements for Variable Genera on - September 2012 [FERC] granted NERC the legal authority to enforce reliability standards with all U.S. users, owners, and operators of the bulk power system, and made compliance with those standards mandatory and enforceable. Reliability Standards are also mandatory and enforceable in Ontario and New Brunswick, and NERC is seeking to achieve comparable results in the other Canadian provinces. NERC will seek recognition in Mexico once necessary legislation is adopted.
Motivation and challenges NERC Coordinating Councils NERC Regional Entitites FRCC Florida Reliability Coordina ng Council MRO Midwest Reliability Organiza on NPCC Northeast Power Coordina ng Council RFC ReliabilityFirst Corpora on SERC SERC Reliability Corpora on SPP RE Southwest Power Pool Regional En ty TRE Texas Reliability En ty WECC Western Electric Coordina ng Council Interconnec on Requirements for Variable Genera on - September 2012 [FERC] granted NERC the legal authority to enforce reliability standards with all U.S. users, owners, and operators of the bulk power system, and made compliance with those standards mandatory and enforceable. Reliability Standards are also mandatory and enforceable in Ontario and New Brunswick, and NERC is seeking to achieve comparable results in the other Canadian provinces. NERC will seek recognition in Mexico once necessary legislation is adopted. NERC = North American Electric Reliability Corporation
Motivation and challenges NERC Coordinating Councils Don’t forget FERC’s ISOs & RTOs!
Motivation and challenges Secondary Control Balancing Authority Figure 2 North American Balancing Authorities and Regions
Motivation and challenges Secondary Control Balancing Authority Figure 2 North American Balancing Authorities and Regions Transmission lines that join two areas are known as tie-lines.
Motivation and challenges Secondary Control Balancing Authority Transmission lines that join two areas are known as tie-lines. The net power out of an area is the sum of the flow on its tie-lines.
Motivation and challenges Secondary Control Balancing Authority Transmission lines that join two areas are known as tie-lines. The net power out of an area is the sum of the flow on its tie-lines. The flow out of an area is equal to Area 1 Area 2 G G G P P T 2 T 1 G G G G total gen - total load - total losses = tie-line flow Remember Economic Dispatch?
Motivation and challenges Primary and Secondary Control Loops Load Shedding (Secondary (emergency control) control loop) Balancing Authority (ACE calculation) Tie-Line Powers The Grid Automatic Loads Regulation Generation System Frequency: ω Transmission lines Control (AGC) Signals Other generators ω identical for synchronous machines in steady state Setpoint Turbine Valves or Power, Turbine Generator Calculation Governor Gates Frequency Speed Turbine Setpoint from Controller (Internal turbine power generation control loop) schedule and tertiary control (Primary control loop) ETH Dynamics 2012
Motivation and challenges Secondary Control Area Control Error Area Control Error: combination of: Deviation of frequency from nominal, and
Motivation and challenges Secondary Control Area Control Error Area Control Error: combination of: Deviation of frequency from nominal, and the difference between the actual tie-line flow, and the scheduled flow (from economic dispatch).
Motivation and challenges Secondary Control Area Control Error Area Control Error: combination of: Deviation of frequency from nominal, and the difference between the actual tie-line flow, and the scheduled flow (from economic dispatch). ACE = P actual − P scheduled + B ∆ ω B is the bias
Motivation and challenges Secondary Control Area Control Error Area Control Error: combination of: Deviation of frequency from nominal, and the difference between the actual tie-line flow, and the scheduled flow (from economic dispatch). ACE = P actual − P scheduled + B ∆ ω B is the bias Provides a measure of whether an area is producing more or less than it should to satisfy schedules and to contribute to controlling frequency. AGC: control signal designed to bring ACE to zero.
Motivation and challenges Secondary Control Area Control Error Area 1 Area 2 G G G P P T 2 T 1 G G G G ω 1 ω 2 AGC 1 AGC 2 ACE ACE 1 2 j P = P Tie line power for Area 1 Sum over all tie lines T 1 T 1 j 1 ETH Dynamics 2012
Motivation and challenges Secondary Control Regulation signals Regulation: on-line generation, responsive load and storage ... helps to maintain PJM RegD Measured interconnection frequency, manage differences between actual and scheduled 1 power flows between balancing areas, and Power match generation to load within the (kW) 0 balancing area. -1 Automatic Generation Control (AGC): 0 10 20 30 40 commands are typically sent about every Time (minute) four seconds.
Motivation and challenges Secondary Control Regulation signals Regulation: on-line generation, responsive load and storage ... helps to maintain PJM RegD Measured interconnection frequency, manage differences between actual and scheduled 1 power flows between balancing areas, and Power match generation to load within the (kW) 0 balancing area. -1 Automatic Generation Control (AGC): 0 10 20 30 40 commands are typically sent about every Time (minute) four seconds. Value is now recognized: FERC Orders
Motivation and challenges Secondary Control Regulation signals Regulation: on-line generation, responsive load and storage ... helps to maintain PJM RegD Measured interconnection frequency, manage differences between actual and scheduled 1 power flows between balancing areas, and Power match generation to load within the (kW) 0 balancing area. -1 Automatic Generation Control (AGC): 0 10 20 30 40 commands are typically sent about every Time (minute) four seconds. Value is now recognized: FERC Orders PJM regulation metrics: scores for correlation, precision, and performance. Faster and more accurate response is paid more.
Motivation and challenges Example: ACE and regulation signals at ERCOT 1000 Area Control Error 800 Deployed Regulation 600 400 Power (MW) 200 0 -200 -400 -600 -800 21:00 23:00 13:00 15:00 17:00 19:00 01:00 03:00 05:00 07:00 09:00 11:00
Motivation and challenges Refinement of our Control Loop Grid disturbance ρ ( t ) 1 + ref = 0 ACE Measured K K P GRID + I s - BA ACE = P actual − P scheduled + B ∆ ω
Motivation and challenges Refinement of our Control Loop Grid disturbance ρ ( t ) 1 + ref = 0 ACE Measured K K P GRID + I s - BA ACE = P actual − P scheduled + B ∆ ω The signal ACE is the sum of two errors.
Motivation and challenges Refinement of our Control Loop Grid disturbance ρ ( t ) 1 + ref = 0 ACE Measured K K P GRID + I s - BA ACE = P actual − P scheduled + B ∆ ω The signal ACE is the sum of two errors. The BA signal ρ ( t ) is obtained from filtering ACE through G c ≡ PI-compensator.
Motivation and challenges Does ACE Work? Generators are not always good actuators Fig. 10. Coal-fired generators do not follow regulation signals precisely.... Some do better than others 20 35 30 15 Gen 'A' Actual Regulation 25 Gen 'A' Requested Regulation 10 20 Regulation (MW) 5 Regulation (MW) 15 10 0 : 5 -5 0 -10 Gen 'B' Actual Regulation -5 -15 -10 Gen 'B' Requested Regulation -15 -20 6:00 7:00 8:00 9:00 6:00 7:00 8:00 9:00
Demand Dispatch Tracking Grid Signal with Residential Loads Example: 20 pools, 20 kW max load Each pool consumes 1kW when operating 12 hour cleaning cycle each 24 hours Power Deviation: 10 Output deviation Reference 20 pools kW 0 Input ζ t 3 0 -3 -10 0 20 40 60 80 100 120 140 160 t/hour Nearly Perfect Service from Pools Meyn, Barooah, B., Chen, Ehren 2015 [IEEE TAC] using an extension/reinterpretation of Todorov 2007 [NIPS] (linearly solvable MDPs)
Demand Dispatch Tracking Grid Signal with Residential Loads Example: 300,000 pools, 300 MW max load Each pool consumes 1kW when operating 12 hour cleaning cycle each 24 hours Power Deviation: Output deviation Reference 300,000 pools 100 50 MW 0 −50 Input ζ t 3 −100 0 -3 0 20 40 60 80 100 120 140 160 t/hour Nearly Perfect Service from Pools Meyn, Barooah, B., Chen, Ehren 2015 [IEEE TAC] using an extension/reinterpretation of Todorov 2007 [NIPS] (linearly solvable MDPs)
Demand Dispatch Demand Dispatch Frequency Decomposition Demand Dispatch: Power consumption from loads varies automatically and continuously to provide service to the grid , without impacting QoS to the consumer One Day at CAISO 2020 Frequency decomposition: Each class 25 Net Load Curve of flexible loads allocated to its own 20 Low pass bandwidth of service , based on QoS constraints and costs 15 GW Gas Turbine BP Coal Batteries BP 10 Control Water Pump Power Grid Σ BP The duck is a sum of a smooth energy signal, C LOAD H Voltage − BP Frequency and two zero-energy services BP C Phase A 5 Actuator feedback loop Mid pass 0 High pass Today: PJM regulation signal: -5 R = RegA + RegD 12am 3am 6am 9am 12pm 3pm 6pm 9pm 12am
Demand Dispatch Demand Dispatch Frequency Decomposition Demand Dispatch: Power consumption from loads varies automatically and continuously to provide service to the grid , without impacting QoS to the consumer Frequency decomposition: Each class of flexible loads allocated to its own bandwidth of service , based on QoS constraints and costs 4 G W ( t ) = Wind generation in BPA, Jan 2015 Goal: G W ( t ) + G r ( t ) ≡ 4 GW Gas Turbine 3 BP Coal Batteries BP Control Water Pump Power Grid Σ BP C LOAD H Voltage − GW BP 2 Frequency BP C Phase Ramps A Actuator feedback loop 1 0 Today: PJM regulation signal: R = RegA + RegD Jan 01 Jan 02 Jan 03 Jan 04 Jan 05 Jan 06
Demand Dispatch Demand Dispatch Frequency Decomposition Demand Dispatch: Power consumption from loads varies automatically and continuously to provide service to the grid , without impacting QoS to the consumer Frequency decomposition: Each class of flexible loads allocated to its own bandwidth of service , based on QoS constraints and costs 4 G r ( t ) Gas Turbine 3 BP Coal Scary Batteries BP Control Water Pump Power Grid Σ BP C LOAD H Voltage − GW BP 2 Frequency BP C Phase A Actuator feedback loop 1 Goal: G W ( t ) + G r ( t ) ≡ 4 GW G r ( t ) obtained from generation? 0 Today: PJM regulation signal: R = RegA + RegD Jan 01 Jan 02 Jan 03 Jan 04 Jan 05 Jan 06
Demand Dispatch Demand Dispatch Frequency Decomposition Demand Dispatch: Power consumption from loads varies automatically and continuously to provide service to the grid , without impacting QoS to the consumer Frequency decomposition: Each class of flexible loads allocated to its own bandwidth of service , based on QoS constraints and costs 4 G r ( t ) Gas Turbine 3 BP Coal Batteries BP Control Water Pump Power Grid Σ BP C LOAD H Voltage − GW G r = G 1 + G 2 + G 3 BP 2 Frequency BP C Phase A G 1 Traditional generation Actuator feedback loop G 2 DD: Chillers & Pool Pumps 1 G DD: HVAC Fans 3 0 Today: PJM regulation signal: R = RegA + RegD Jan 01 Jan 02 Jan 03 Jan 04 Jan 05 Jan 06
Demand Dispatch Demand Dispatch Frequency Decomposition Demand Dispatch: Power consumption from loads varies automatically and continuously to provide service to the grid , without impacting QoS to the consumer Frequency decomposition: Each class of flexible loads allocated to its own One Day at CAISO: 4GW loss at 6am bandwidth of service , based on QoS Net Load Curve constraints and costs 15 Gas Turbine BP Coal Batteries BP Control Water Pump Power Grid 10 Σ BP Low pass C LOAD H Voltage − BP Mid pass Frequency GW BP C Phase High pass A Higher pass Actuator feedback loop 5 Zero energy reg signals Today: PJM regulation signal: 0 R = RegA + RegD 12am 3am 6am 9am 12pm 3pm 6pm 9pm 12am
Demand Dispatch Demand Dispatch Responsive Regulation and desired QoS – A partial list of the needs of the grid operator, and the consumer High quality Ancillary Service? Customer QoS constraints satisfied?
Demand Dispatch Demand Dispatch Responsive Regulation and desired QoS – A partial list of the needs of the grid operator, and the consumer High quality Ancillary Service? Customer QoS constraints satisfied? Cost effective? Includes installation cost, communication cost, maintenance, and environmental.
Demand Dispatch Demand Dispatch Responsive Regulation and desired QoS – A partial list of the needs of the grid operator, and the consumer High quality Ancillary Service? Customer QoS constraints satisfied? Cost effective? Includes installation cost, communication cost, maintenance, and environmental. Reliable? Will AS be available each day? (may vary with time, but capacity must be predictable)
Demand Dispatch Demand Dispatch Responsive Regulation and desired QoS – A partial list of the needs of the grid operator, and the consumer High quality Ancillary Service? Customer QoS constraints satisfied? Cost effective? Includes installation cost, communication cost, maintenance, and environmental. Reliable? Will AS be available each day? (may vary with time, but capacity must be predictable) Is the incentive to the consumer reliable? If a consumer receives a $50 payment for one month, and only $1 the next, will there be an explanation that is clear to the consumer?
Demand Dispatch Demand Dispatch G r = G 1 + G 2 + G 3 G 1 G 3 ? G 2 G r
Demand Dispatch Demand Dispatch G r = G 1 + G 2 + G 3 G 1 Traditional generation G 2 G 3 G r
Demand Dispatch Demand Dispatch G r = G 1 + G 2 + G 3 G 1 Traditional generation G 2 Water pumping (e.g. pool pumps) Fans in commercial HVAC G 3 G r Demand Dispatch: Power consumption from loads varies automatically and continuously to provide service to the grid, without impacting QoS to the consumer
Demand Dispatch Demand Dispatch for Virtual Energy Storage Responsive Regulation and desired QoS Demand Dispatch: Power consumption from loads varies automatically and continuously to provide service to the grid, without impacting QoS to the consumer High quality Ancillary Service Reliable Cost effective Customer QoS constraints satisfied Virtual energy storage: achieve these goals simultaneously through distributed control
Control Architecture Control Goals and Architecture Macro control High-level control layer: BA or a load aggregator. The balancing challenges are of many different categories and time-scales: Automatic Generation Control (AGC); time scales of seconds to 20 minutes. Balancing reserves. In the Bonneville Power Authority, the balancing reserves include both AGC and balancing on timescales of many hours. Balancing on a slower time-scale is achieved through real time markets in some other regions of the U.S. Contingencies (e.g., a generator outage) Peak shaving Smoothing ramps from solar or wind generation
Control Architecture Control Goals and Architecture Local Control: decision rules designed to respect needs of load and grid Power deviation Local decision Grid signal U i Y i ζ t Local t t Load i Control X i t Local feedback loop Min. communication: each load monitors its state and a regulation signal from the grid. Aggregate must be controllable: randomized policies for finite-state loads.
Control Architecture Randomized Policies Power deviation Local decision Grid signal U i Y i ζ t Local t t Load i Control X i t Local feedback loop Local control architecture For the i th load: Y i t power, U i t load setpoint, X i t local state. Signal ζ t is from the grid operator – common to all loads of a certain class.
Control Architecture Randomized Policies Power deviation Local decision Grid signal U i Y i ζ t Local t t Load i Control X i t Local feedback loop Local control architecture For the i th load: Y i t power, U i t load setpoint, X i t local state. Signal ζ t is from the grid operator – common to all loads of a certain class. Policy: Decision rule that maps ( ζ t , X i t ) to the input U i t . Randomized Policy: Decision rule also depends on rand (an intelligent coin-flip)
Control Architecture Load Model Controlled Markovian Dynamics Load 1 Power Consumption (MW) BA Load 2 + ζ Reference (MW) G c ... r Load N Discrete time: i th load X i ( t ) evolves on finite state space X Each load is subject to common controlled Markovian dynamics. Signal ζ = { ζ t } is broadcast to all loads Controlled transition matrix { P ζ : ζ ∈ R } : t +1 = x ′ | X i P { X i t = x, ζ t = ζ } = P ζ ( x, x ′ ) Questions • How to analyze aggregate of similar loads? • Local control design?
Control Architecture Randomized Policies • How to analyze aggregate of similar loads? • How to design P ζ ? 600 400 200 MW 0 −200 −400 −600 Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7
Mean-field model How to analyze aggregate? Mean field model N loads running independently, each under the command ζ . Empirical Distributions: N t ( x ) = 1 � µ N I { X i ( t ) = x } , x ∈ X N i =1 U ( x ) power consumption in state x , N t = 1 � � y N U ( X i µ N t ) = t ( x ) U ( x ) N x i =1 Mean-field model: via Law of Large Numbers for martingales µ t +1 = µ t P ζ t , y t = � µ t , U � ζ t = f t ( y 0 , . . . , y t ) by design
Example: pool pumps Example: pool pumps How Pools Can Help Regulate The Grid 1,5KW 400V Needs of a single pool ⊲ Filtration system circulates and cleans: Average pool pump uses 1.3kW and runs 6-12 hours per day, 7 days per week ⊲ Pool owners are oblivious, until they see frogs and algae ⊲ Pool owners do not trust anyone: Privacy is a big concern
Example: pool pumps One Million Pools in Florida Pools Service the Grid Today On Call 1 : Utility controls residential pool pumps and other loads 1 Florida Power and Light , Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml
Example: pool pumps One Million Pools in Florida Pools Service the Grid Today On Call 1 : Utility controls residential pool pumps and other loads Contract for services: no price signals involved Used only in times of emergency — Activated only 3-4 times a year 1 Florida Power and Light , Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml
Example: pool pumps One Million Pools in Florida Pools Service the Grid Today On Call 1 : Utility controls residential pool pumps and other loads Contract for services: no price signals involved Used only in times of emergency — Activated only 3-4 times a year Opportunity: FP&L already has their hand on the switch of nearly one million pools! 1 Florida Power and Light , Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml
Example: pool pumps One Million Pools in Florida Pools Service the Grid Today On Call 1 : Utility controls residential pool pumps and other loads Contract for services: no price signals involved Used only in times of emergency — Activated only 3-4 times a year Opportunity: FP&L already has their hand on the switch of nearly one million pools! Surely pools can provide much more service to the grid 1 Florida Power and Light , Florida’s largest utility. www.fpl.com/residential/energysaving/programs/oncall.shtml
Example: pool pumps An Intelligent Pool 1 2 I − 1 I On . . . Local Control Architecture Off . . . I I − 1 1 2 Local control architecture For the i th load: Y i t = power: 1kW when running,
Example: pool pumps An Intelligent Pool 1 2 I − 1 I On . . . Local Control Architecture Off . . . I I − 1 1 2 Local control architecture For the i th load: Y i t = power: 1kW when running, U i t = 1 or 0 (pool pump is running or not)
Example: pool pumps An Intelligent Pool 1 2 I − 1 I On . . . Local Control Architecture Off . . . I I − 1 1 2 Local control architecture For the i th load: Y i t = power: 1kW when running, U i t = 1 or 0 (pool pump is running or not) X i t local state: ( U i t , I i t ) , with I i t the time in current power state.
Example: pool pumps An Intelligent Pool 1 2 I − 1 I On . . . Local Control Architecture Off . . . I I − 1 1 2 Local control architecture For the i th load: Y i t = power: 1kW when running, U i t = 1 or 0 (pool pump is running or not) X i t local state: ( U i t , I i t ) , with I i t the time in current power state. Randomized Policy: Decision rule that maps ( ζ t , X i t , rand t i ) to the input U i t .
Example: pool pumps An Intelligent Pool 1 2 I − 1 I On . . . Local Control Architecture Off . . . I I − 1 1 2 Local control architecture For the i th load: Y i t = power: 1kW when running, U i t = 1 or 0 (pool pump is running or not) X i t local state: ( U i t , I i t ) , with I i t the time in current power state. Randomized Policy: Decision rule that maps ( ζ t , X i t , rand t i ) to the input U i t . Randomized Policy: As ζ increases, probability of turning on increases: 1 ( ζ ) T ζ = 4 2 = ζ 0.5 Pool with nominal 12 hour cleaning cycle ζ = 0 - 2 4 - ζ = ζ = 0 0 12 24 hours
Example: pool pumps Pools in Florida Supply G 2 – BPA regulation signal ∗ Stochastic simulation using N = 10 6 pools 300 Reference Output deviation (MW) 200 100 0 −100 −200 −300 0 0 20 20 40 40 60 60 80 80 100 100 120 120 140 140 160 160 t/hour t = � t PI control: ζ t = 19 e t + 1 . 4 e I e t = r t − y t and e I t , k =0 e k Each pool pump turns on/off with probability depending on 1) its internal state, and 2) the BPA reg signal ∗ transmission.bpa.gov/Business/Operations/Wind/reserves.aspx
Example: pool pumps Mean Field Model Linearized Dynamics Mean-field model: µ t +1 = µ t P ζ t , y t = � µ t , U � ζ t = f t ( y 0 , . . . , y t ) Φ t +1 = A Φ t + Bζ t Linear state space model: γ t = C Φ t Interpretations: | ζ t | is small, and π denotes invariant measure for P 0 . • Φ t ∈ R | X | , a column vector with Φ t ( x ) ≈ µ t ( x ) − π ( x ) , x ∈ X • γ t ≈ y t − y 0 ; deviation from nominal steady-state • A = P T 0 , C = U T , and input dynamics linearized: T = d � B dζ πP ζ � � ζ =0
Example: pool pumps Transfer function The linearization at a particular value ζ is the state space model with transfer function, G ζ ( z ) = C [ Iz − A ] − 1 B (1) ζ , C i = � U ζ ( x i ) for each i , and in which A = P T � π ζ ( x ) E ζ ( x, x i ) , B i = 1 ≤ i ≤ d (2) x dζ P ζ � d E ζ = U ζ = U − ¯ U ζ , with ¯ U ζ = π ζ ( U ) .
Example: pool pumps Tracking Grid Signal with Residential Loads Example: 20 pools, 20 kW max load Each pool consumes 1kW when operating 12 hour cleaning cycle each 24 hours Power Deviation: 10 Output deviation Reference 20 pools kW 0 Input ζ t 3 0 -3 -10 0 20 40 60 80 100 120 140 160 t/hour Nearly Perfect Service from Pools Meyn et al. 2013 [CDC], Meyn et al. 2015 [IEEE TAC]
Example: pool pumps Tracking Grid Signal with Residential Loads Example: 300,000 pools, 300 MW max load Each pool consumes 1kW when operating 12 hour cleaning cycle each 24 hours Power Deviation: Output deviation Reference 300,000 pools 100 50 MW 0 −50 Input ζ t 3 −100 0 -3 0 20 40 60 80 100 120 140 160 t/hour Nearly Perfect Service from Pools Meyn et al. 2013 [CDC], Meyn et al. 2015 [IEEE TAC]
Example: pool pumps Range of services provided by pools Example: 10,000 pools, 10 MW max load 12 hr/day cycle Power Deviation Reference 2 MW 0 -2 -4 15 ζ 0 -15 0 20 40 60 80 100 120 140 160 t/hour
Local Control Design
Local Control Design Local Design Goal: Construct a family of transition matrices { P ζ : ζ ∈ R } Myopic Design: P myop ( x, x ′ ) := P 0 ( x, x ′ ) exp ζ U ( x ′ ) − Λ ζ ( x ) � � ζ �� �� � with Λ ζ ( x ) := log x ′ P 0 ( x, x ′ ) exp ζ U ( x ′ ) the normalizing constant.
Local Control Design Local Design Goal: Construct a family of transition matrices { P ζ : ζ ∈ R } Myopic Design: P myop ( x, x ′ ) := P 0 ( x, x ′ ) exp ζ U ( x ′ ) − Λ ζ ( x ) � � ζ �� �� � with Λ ζ ( x ) := log x ′ P 0 ( x, x ′ ) exp ζ U ( x ′ ) the normalizing constant. Exponential family design: P ζ ( x, x ′ ) := P 0 ( x, x ′ ) exp h ζ ( x, x ′ ) − Λ h ζ ( x ) � � with h ζ ( x, x ′ ) = ζH 0 ( x, x ′ ) . The choice of H 0 will typically correspond to the linearization of a more advanced design around the value ζ = 0 (or some other fixed value of ζ ).
Local Control Design Local Design Goal: Construct a family of transition matrices { P ζ : ζ ∈ R } Individual Perspective Design Consider a finite-time-horizon optimization problem: For a given terminal time T , let p 0 denote the pmf on strings of length T , T − 1 � p 0 ( x 1 , . . . , x T ) = P 0 ( x i , x i +1 ) , i =0 where x 0 ∈ X is assumed to be given. The scalar ζ ∈ R is interpreted as a weighting parameter in the following definition of total welfare. For any pmf p , � T � � W T ( p ) = ζ E p U ( X t ) − D ( p � p 0 ) t =1 where the expectation is with respect to p , and D denotes relative entropy: � p ( x 1 , . . . , x T ) � � D ( p � p 0 ) := log p ( x 1 , . . . , x T ) p 0 ( x 1 , . . . , x T ) x 1 ,...,x T
Local Control Design Local Design Goal: Construct a family of transition matrices { P ζ : ζ ∈ R } Myopic design is an optimizer for the horizon T = 1 , P myop ( x 0 , · ) ∈ arg max W 1 ( p ) . ζ p
Local Control Design Local Design Goal: Construct a family of transition matrices { P ζ : ζ ∈ R } Myopic design is an optimizer for the horizon T = 1 , P myop ( x 0 , · ) ∈ arg max W 1 ( p ) . ζ p The infinite-horizon mean welfare is denoted, 1 η ∗ T W T ( p ∗ ζ = lim T ) T →∞
Local Control Design Local Design Goal: Construct a family of transition matrices { P ζ : ζ ∈ R } Myopic design is an optimizer for the horizon T = 1 , P myop ( x 0 , · ) ∈ arg max W 1 ( p ) . ζ p The infinite-horizon mean welfare is denoted, 1 η ∗ T W T ( p ∗ ζ = lim T ) T →∞ Explicit construction via eigenvector problem: P ζ ( x, y ) = 1 v ( y ) ˆ P ζ ( x, y ) , x, y ∈ X , v ( x ) λ ˆ ˆ where P ζ v = λv, P ζ ( x, y ) = exp( ζ U ( x )) P 0 ( x, y ) for λ Perron-Frobenious eigenvalue. Extension/reinterpretation of [Todorov 2007] + [Kontoyiannis & Meyn 200X]
Local Control Design Local Design Extending local control design to include exogenous disturbances State space for a load model: X = X u × X n . Components X n are not subject to direct control (e.g. impact of the weather on the climate of a building).
Local Control Design Local Design Extending local control design to include exogenous disturbances State space for a load model: X = X u × X n . Components X n are not subject to direct control (e.g. impact of the weather on the climate of a building). Conditional-independence structure of the local transition matrix x ′ = ( x ′ P ( x, x ′ ) = R ( x, x ′ u ) Q 0 ( x, x ′ u , x ′ n ) , n ) Q 0 models uncontroled load dynamics and exogenous disturbances.
Recommend
More recommend
