1
play

1 Optimization in decision graphs Unfolding to decision tree - PDF document

Feb 07, 2024 •364 likes •467 views

Decision graphs II Influence Diagrams Advanced Herd Management Anders Ringgaard Kristensen Slide 1 Outline Optimization methods Decision tree Strong junction tree Single Policy Updating Decision node ordering Advantages and

  1. Decision graphs II Influence Diagrams Advanced Herd Management Anders Ringgaard Kristensen Slide 1 Outline Optimization methods • Decision tree • Strong junction tree • Single Policy Updating Decision node ordering Advantages and disadvantages of decision graphs Slide 2 1

  2. Optimization in decision graphs Unfolding to decision tree • Only option until Shachter (1986) Influence diagram with “no forgetting” (like the decision tree): • Famous article by Jensen, Jensen & Dittmer (1994): • Strict ordering of nodes • Creation of a “strong” junction tree • Implemented in the Hugin software system LImited Memory Influence Diagram (LIMID): • Described by Lauritzen & Nilsson (2001): • Decision nodes converted to chance nodes. • Implemented in the Esthauge LIMID software system Slide 3 The repeated milk test problem The reason for testing the milk from a particular cow is to decide whether or not to pour the milk into the bulk tank: • If the milk from an infected cow is poured into the bulk tank, the dairy will reduce the total payment by 10%. • If the milk from the cow is not poured into the bulk tank, the value of that milk is lost. • The farmer has 50 cows. • Under the action “Pour”: • The value of the milk (if not infected) is 1000 • The value of the milk with reduction is 900 • Under the action “Don’t pour”: • The value of the milk is 1000 × 49/50 = 980 • It doesn’t matter whether or not the milk is infected Slide 4 2

  3. As a decision graph The result of the test is known when the decision is made. Is that enough? Let’s try! Slide 5 Relevant past For a decision made at time t’ the values of all variables observed at time t ≤ t’ are in principle relevant. Moreover, all decisions made at previous time steps t ≤ t’ may be relevant. This observation is referred to, as a “no forgetting” assumption. Requires a strict ordering of the nodes! Slide 6 3

  4. Influence diagrams Jensen, Jensen & Dittmer (1994) “No forgetting” assumption: • The value of any previously observed variable is remembered. • Any decision made earlier is remembered. • Graphically, this means that we must insert numerous implicit edges into the net. • Implemented in the Hugin software system. Slide 7 The decision graph with no forgetting There are 13 edges into Pour7 ! Slide 8 4

  5. A comparison … Without implicit “no forgetting” edges. Implicit “no forgetting” edges visible. Slide 9 Consequences of “no forgetting” The decision strategy found is an optimal one. The optimal strategy gets very complex: • The optimal decision for Pour7 depends on the value 13 other variables. Optimization becomes very demanding from a computational point of view: • Even rather simple decision problems cannot be solved in practice. • The applicational experiences with influence diagrams have been disappointing. • Application to delivery policies in slaughter pigs failed. Slide 10 5

  6. LImited Memory Influence Diagrams Working title: “Demented Influence Diagrams”. Due to the disappointments with influence diagrams in herd management, a research initiative was initiated: • Dennis Nilsson as post doc at Aalborg University (later assistant professor at IHH) • Michael Höhle as PhD student at IHH The goal was to come up with better optimization methods for decision graphs by relaxing the “no forgetting” assumption. Slide 11 LIMIDs – the ideas behind Choice 1 Only one decision: • Try the alternatives one by one and select the best. Extend the idea to larger nets. Opened Choice 2 Gain True Slide 12 6

  7. Single Policy Updating in LIMIDs Pour1 Pour3 Pour2 Pour5 Pour7 Pour4 Pour6 Determine an optimization ordering (usually just backwards) Convert all decisions to chance nodes. Update the policy of each decision node one by one. Repeat until convergence. Slide 13 Single policy updating in LIMIDs Lauritzen & Nilsson (2001) Usually only near-optimal solutions. Never more complex than a Bayesian network. Not so computationally demanding as influence diagrams. The algorithm may be applied to influence diagrams if all implicit edges are added. Rather efficient even for influence diagrams. Implemented in the Esthauge LIMID Software System. Slide 14 7

  8. Soluble LIMIDs For some LIMIDs, the Single Policy Updating algorithm will provide us with an exact solution. Such LIMIDs are called soluble. All influence diagrams are soluble (i.e. if all implicit edges are added): • Some edges may be irrelevant. • The software system can automatically remove irrelevant information edges and find the so-called minimal reduction. • The software system can check whether the minimal reduction is soluble. • If it is soluble, a unique decision node ordering is automatically identified, and only one iteration is necessary. Slide 15 Check for solubility Slide 16 8

  9. Advantages of decision graphs State space representation: • Variable by variable (as opposed to dynamic programming). • Allow unobservable variables. • No forgetting – at least as an option (as opposed to dynamic programming). Slide 17 Disadvantages of decision graphs No forgetting: • Complexity – hard to solve (even though heavily improved with LIMIDs). Only suited for static decision problems: • Time steps must be explicitly modeled (as opposed to dynamic programming). Only suited for strictly symmetric decision problems (cf irregular decision trees) Slide 18 9

  10. Properties of methods for decision support Herd constraints Optimization Decision graphs Biological Functional variation limitations Uncertainty Dynamics Slide 19 10

Recommend

More recommend