Planning and Optimization D3. Abstractions: Additive Abstractions - PowerPoint PPT Presentation

Planning and Optimization D3. Abstractions: Additive Abstractions Malte Helmert and Thomas Keller Universit¨ at Basel October 30, 2019

Additivity Outlook Summary Content of this Course Foundations Logic Classical Heuristics Constraints Planning Explicit MDPs Probabilistic Factored MDPs

Additivity Outlook Summary Content of this Course: Heuristics Abstractions Delete Relaxation in General Heuristics Pattern Abstraction Databases Merge & Shrink Landmarks Network Constraints Flows Potential Heuristics

Additivity Outlook Summary Additivity

Additivity Outlook Summary Orthogonality of Abstractions Definition (Orthogonal) Let α 1 and α 2 be abstractions of transition system T . ℓ We say that α 1 and α 2 are orthogonal if for all transitions s − → t of T , we have α i ( s ) = α i ( t ) for at least one i ∈ { 1 , 2 } .

Additivity Outlook Summary Affecting Transition Labels Definition (Affecting Transition Labels) Let T be a transition system, and let ℓ be one of its labels. ℓ We say that ℓ affects T if T has a transition s − → t with s � = t . Theorem (Affecting Labels vs. Orthogonality) Let α 1 and α 2 be abstractions of transition system T . If no label of T affects both T α 1 and T α 2 , then α 1 and α 2 are orthogonal. (Easy proof omitted.)

Additivity Outlook Summary Orthogonal Abstractions: Example 2 6 9 12 5 7 14 13 3 4 1 11 15 10 8 Are the abstractions orthogonal?

Additivity Outlook Summary Orthogonal Abstractions: Example 2 6 9 12 5 7 14 13 3 4 1 11 15 10 8 Are the abstractions orthogonal?

Additivity Outlook Summary Orthogonality and Additivity Theorem (Additivity for Orthogonal Abstractions) Let h α 1 , . . . , h α n be abstraction heuristics of the same transition system such that α i and α j are orthogonal for all i � = j. Then � n i =1 h α i is a safe, goal-aware, admissible and consistent heuristic for Π .

Additivity Outlook Summary Orthogonality and Additivity: Example RLR RLL ILR RIL ILL RIR LLR LLL IIL IIR RRR RRL LIL IRR LIR IRL LRR LRL transition system T state variables: first package, second package, truck

Additivity Outlook Summary Orthogonality and Additivity: Example RLR RLL RLR RLL ILR ILR RIL RIL ILL RIR ILL RIR LLR LLR LLL LLL IIL IIL IIR IIR RRR RRR RRL RRL LIL LIL IRR IRR LIR IRL LIR IRL LRR LRL LRR LRL abstraction α 1 abstraction: only consider value of first package

Additivity Outlook Summary Orthogonality and Additivity: Example RLR RLL RLR RLL ILR ILR RIL RIL ILL RIR ILL RIR LLR LLR LLL LLL IIL IIL IIR IIR RRR RRR RRL RRL LIL LIL IRR IRR LIR IRL LIR IRL LRR LRL LRR LRL abstraction α 1 abstraction: only consider value of first package

Additivity Outlook Summary Orthogonality and Additivity: Example RLR RLL RLR RLL ILR ILR RIL RIL ILL RIR ILL RIR LLR LLR LLL LLL IIL IIL IIR IIR RRR RRR RRL RRL LIL LIL IRR IRR LIR IRL LIR IRL LRR LRL LRR LRL abstraction α 2 (orthogonal to α 1 ) abstraction: only consider value of second package

Additivity Outlook Summary Orthogonality and Additivity: Example RLR RLL RLR RLL ILR ILR RIL RIL ILL RIR ILL RIR LLR LLR LLL LLL IIL IIL IIR IIR RRR RRR RRL RRL LIL LIL IRR IRR LIR IRL LIR IRL LRR LRL LRR LRL abstraction α 2 (orthogonal to α 1 ) abstraction: only consider value of second package

Additivity Outlook Summary Orthogonality and Additivity: Proof (1) Proof. We prove goal-awareness and consistency; the other properties follow from these two. Let T = � S , L , c , T , s 0 , S ⋆ � be the concrete transition system. Let h = � n i =1 h α i . Goal-awareness: For goal states s ∈ S ⋆ , h ( s ) = � n i =1 h α i ( s ) = � n i =1 0 = 0 because all individual abstraction heuristics are goal-aware. . . .

Additivity Outlook Summary Orthogonality and Additivity: Proof (1) Proof. We prove goal-awareness and consistency; the other properties follow from these two. Let T = � S , L , c , T , s 0 , S ⋆ � be the concrete transition system. Let h = � n i =1 h α i . Goal-awareness: For goal states s ∈ S ⋆ , h ( s ) = � n i =1 h α i ( s ) = � n i =1 0 = 0 because all individual abstraction heuristics are goal-aware. . . .

Additivity Outlook Summary Orthogonality and Additivity: Proof (2) Proof (continued). o Consistency: Let s − → t ∈ T . We must prove h ( s ) ≤ c ( o ) + h ( t ). Because the abstractions are orthogonal, α i ( s ) � = α i ( t ) for at most one i ∈ { 1 , . . . , n } . Case 1: α i ( s ) = α i ( t ) for all i ∈ { 1 , . . . , n } . Then h ( s ) = � n i =1 h α i ( s ) = � n i =1 h ∗ T α i ( α i ( s )) = � n i =1 h ∗ T α i ( α i ( t )) = � n i =1 h α i ( t ) = h ( t ) ≤ c ( o ) + h ( t ) . . . .

Additivity Outlook Summary Orthogonality and Additivity: Proof (2) Proof (continued). o Consistency: Let s − → t ∈ T . We must prove h ( s ) ≤ c ( o ) + h ( t ). Because the abstractions are orthogonal, α i ( s ) � = α i ( t ) for at most one i ∈ { 1 , . . . , n } . Case 1: α i ( s ) = α i ( t ) for all i ∈ { 1 , . . . , n } . Then h ( s ) = � n i =1 h α i ( s ) = � n i =1 h ∗ T α i ( α i ( s )) = � n i =1 h ∗ T α i ( α i ( t )) = � n i =1 h α i ( t ) = h ( t ) ≤ c ( o ) + h ( t ) . . . .

Additivity Outlook Summary Orthogonality and Additivity: Proof (2) Proof (continued). o Consistency: Let s − → t ∈ T . We must prove h ( s ) ≤ c ( o ) + h ( t ). Because the abstractions are orthogonal, α i ( s ) � = α i ( t ) for at most one i ∈ { 1 , . . . , n } . Case 1: α i ( s ) = α i ( t ) for all i ∈ { 1 , . . . , n } . Then h ( s ) = � n i =1 h α i ( s ) = � n i =1 h ∗ T α i ( α i ( s )) = � n i =1 h ∗ T α i ( α i ( t )) = � n i =1 h α i ( t ) = h ( t ) ≤ c ( o ) + h ( t ) . . . .

Additivity Outlook Summary Orthogonality and Additivity: Proof (2) Proof (continued). o Consistency: Let s − → t ∈ T . We must prove h ( s ) ≤ c ( o ) + h ( t ). Because the abstractions are orthogonal, α i ( s ) � = α i ( t ) for at most one i ∈ { 1 , . . . , n } . Case 1: α i ( s ) = α i ( t ) for all i ∈ { 1 , . . . , n } . Then h ( s ) = � n i =1 h α i ( s ) = � n i =1 h ∗ T α i ( α i ( s )) = � n i =1 h ∗ T α i ( α i ( t )) = � n i =1 h α i ( t ) = h ( t ) ≤ c ( o ) + h ( t ) . . . .

Additivity Outlook Summary Orthogonality and Additivity: Proof (3) Proof (continued). Case 2: α i ( s ) � = α i ( t ) for exactly one i ∈ { 1 , . . . , n } . Let k ∈ { 1 , . . . , n } such that α k ( s ) � = α k ( t ). Then h ( s ) = � n i =1 h α i ( s ) i ∈{ 1 ,..., n }\{ k } h ∗ T α i ( α i ( s )) + h α k ( s ) = � i ∈{ 1 ,..., n }\{ k } h ∗ T α i ( α i ( t )) + c ( o ) + h α k ( t ) ≤ � = c ( o ) + � n i =1 h α i ( t ) = c ( o ) + h ( t ) , where the inequality holds because α i ( s ) = α i ( t ) for all i � = k and h α k is consistent.

Additivity Outlook Summary Orthogonality and Additivity: Proof (3) Proof (continued). Case 2: α i ( s ) � = α i ( t ) for exactly one i ∈ { 1 , . . . , n } . Let k ∈ { 1 , . . . , n } such that α k ( s ) � = α k ( t ). Then h ( s ) = � n i =1 h α i ( s ) i ∈{ 1 ,..., n }\{ k } h ∗ T α i ( α i ( s )) + h α k ( s ) = � i ∈{ 1 ,..., n }\{ k } h ∗ T α i ( α i ( t )) + c ( o ) + h α k ( t ) ≤ � = c ( o ) + � n i =1 h α i ( t ) = c ( o ) + h ( t ) , where the inequality holds because α i ( s ) = α i ( t ) for all i � = k and h α k is consistent.

Additivity Outlook Summary Outlook

Additivity Outlook Summary Using Abstraction Heuristics in Practice In practice, there are conflicting goals for abstractions: we want to obtain an informative heuristic, but want to keep its representation small. Abstractions have small representations if there are few abstract states and there is a succinct encoding for α .

Additivity Outlook Summary Counterexample: One-State Abstraction ALR ARL ALR ARL LLR RRL LLR RRL ALL ARR ALL ARR LRR LRR LLL LLL RRR RRR RLL RLL BLL BLL BRR BRR LRL LRL RLR RLR BRL BRL BLR BLR One-state abstraction: α ( s ) := const. + very few abstract states and succinct encoding for α − completely uninformative heuristic

Additivity Outlook Summary Counterexample: Identity Abstraction ALR ARL LLR RRL ALL ARR LRR LLL RRR RLL BLL BRR LRL RLR BRL BLR Identity abstraction: α ( s ) := s . + perfect heuristic and succinct encoding for α − too many abstract states

Additivity Outlook Summary Counterexample: Perfect Abstraction ALR ALR ARL ARL LLR LLR RRL RRL ALL ALL ARR ARR LLL RRR RLL LRR LLL RRR RLL BLL BRR BLL BRR LRL RLR LRL RLR BRL BRL BLR BLR Perfect abstraction: α ( s ) := h ∗ ( s ). + perfect heuristic and usually few abstract states − usually no succinct encoding for α

Additivity Outlook Summary Automatically Deriving Good Abstraction Heuristics Abstraction Heuristics for Planning: Main Research Problem Automatically derive effective abstraction heuristics for planning tasks. � we will study two state-of-the-art approaches in Chapters D4–D8

Additivity Outlook Summary Summary