06-10-2014 Institut for Produktionsdyr og Heste Optimal replacement - PDF document

06-10-2014 Institut for Produktionsdyr og Heste Optimal replacement policies for dairy cows based on daily yield measurements Katarina Nielsen Dominiak, Ph.D Student Advanced Quantitative Methods in Herd Management AQMHM 06-10-2014 Dias 1 Institut for Produktionsdyr og Heste Background In all types of life stock production the decision of when to replace or keep an animal is mulitifactorial For dairy cows some factors are Current milk yield • Expectations to future milk yield • Illness • Replacement heifers available • Reproduction • Goal of the farmer/producer • Market prices • Milk • Heifers • Feed • Etc • AQMHM 06-10-2014 Dias 2 Institut for Produktionsdyr og Heste Always a new situation Every day you know more - so when is the optimal time to make decisions? AQMHM 06-10-2014 Dias 3 1

06-10-2014 Institut for Produktionsdyr og Heste Markov Decision Process - MDP Observation of state -> Decision -> Reward and probabilistic evolution of the system to a new state Has been used for replacement policies for decades Often calculating economic impact of factors – used by academics and experts Stage lengths in previous models from 1 month to 1 year Not suitable for the farmer or for day-to-day decisions AQMHM 06-10-2014 Dias 4 Institut for Produktionsdyr og Heste Hierarchial Markov Decision Process - HMDP A series of MDP built together into one MDP Stages of the process expanded to a child process – multiple times if needed AMS with biosensors provide detailed information on a daily basis But no dairy cow replacement models based on daily time steps exist This paper describes the first step of developing such a model AQMHM 06-10-2014 Dias 5 Institut for Produktionsdyr og Heste Structure of the model AQMHM 06-10-2014 Dias 6 2

06-10-2014 Institut for Produktionsdyr og Heste Biological Functions used in the model Milk Yield • Growth • Feeding and Energy • Reproduction • Involuntary Culling (IC) • AQMHM 06-10-2014 Dias 7 Institut for Produktionsdyr og Heste Daily milk yield Daily milk yield measurements on individual level available t = day in lactation (measured as days from calving) j = lactation number µ = average daily milk yield (herd level) M = milk yield in kilograms (cow level) A = production potential (cow level) ~ N(0,σ 2 A ) X = local production effect (cow level) v = random error term ~ N(0, σ 2 v ) AQMHM 06-10-2014 Dias 8 Institut for Produktionsdyr og Heste Daily milk yield at lactation 3 AQMHM 06-10-2014 Dias 9 3

06-10-2014 Institut for Produktionsdyr og Heste Residual milk yield Substracting average daily milk yield on both sides gives the residual milk yield – how does the yield differ from expected: AQMHM 06-10-2014 Dias 10 Institut for Produktionsdyr og Heste Milk yield modeled as a State Space Model (SSM) SSM combines relevant prior knowledge and current information through an observation equation and a system equation Residual daily milk yield (observation equation) Can be written using matrix notation A is constant over time hence expressing a permanent trait for the cow AQMHM 06-10-2014 Dias 11 Institut for Produktionsdyr og Heste Milk yield modeled as a State Space Model (SSM) The system equation describes the relationship between variables A and X from time t – 1 to time t W is the covariance matrix to random variable w t,j and ρ is an autocorelation factor AQMHM 06-10-2014 Dias 12 4

06-10-2014 Institut for Produktionsdyr og Heste Milk yield modeled as a State Space Model (SSM) Assumption to is that the prior is normal distributed with mean and covariance matrix: AQMHM 06-10-2014 Dias 13 Institut for Produktionsdyr og Heste Milk yield modeled as a State Space Model (SSM) AQMHM 06-10-2014 Dias 14 Institut for Produktionsdyr og Heste Milk yield modeled as a State Space Model (SSM) For j > 1 the expected production potential is the same as estimated during the last lactation An estimation of the production potential of 5 means that the cow should produce 5 kg above herd average every day AQMHM 06-10-2014 Dias 15 5

06-10-2014 Institut for Produktionsdyr og Heste Growth Total body weight (BW) was estimated using Gompertz growth curve and a BCS curve yielding the BCS during lactation AQMHM 06-10-2014 Dias 16 Institut for Produktionsdyr og Heste Feeding and Energy Measured in the net energy of Scandinavian Feed Units (SFU) Total amount of energy needed during one cycle is the sum of energy needed for: Maintenance • Fetus growth • Milk yield • Standardized BW gain • AQMHM 06-10-2014 Dias 17 Institut for Produktionsdyr og Heste Reproduction A model simulating the estrus cycle was constructed based on a continuous Markov chain. The model could calculate the probability of a positive pregnancy test 35 days from calving till insemination 49 days from 40 days from drying off till insemination till calving pregnancy test 193 days lactation Repeatedly from positive inseminated till day pregnancy test till 250 if not tested drying off pregnant AQMHM 06-10-2014 Dias 18 6

06-10-2014 Institut for Produktionsdyr og Heste Involuntary culling (IC) Dead cows or cows slaughtered because of other reasons than milk yield or failure to concieve IC is typically influenced by factors like Lactation stage (max 10 in this model) • Disease incidence • Age of the cow • AQMHM 06-10-2014 Dias 19 Institut for Produktionsdyr og Heste Model structure AQMHM 06-10-2014 Dias 20 Institut for Produktionsdyr og Heste Stage length, States, and Decisions Level 0: State – dummy, representing insertion of new cow Stage – defined by child process, life span of a cow (level 1) Level 1: State – Expected production potential of the cow (Â t,j ) 13 levels Stage – defined by child process, lactation (level 2) maximum 10 stages Level 2: State – defined by a combination of and drying off week ŵ Moreover an IC state was added Stage – one day (except drying off week stage which was defined as one week) Action – keep or replace AQMHM 06-10-2014 Dias 21 7

06-10-2014 Institut for Produktionsdyr og Heste Transition Probabilities A child process at level 2 representing lactation j = a state at stage t = milk yield = drying off week is either described through or the IC state AQMHM 06-10-2014 Dias 22 Institut for Produktionsdyr og Heste Transition Probabilities Given state , unknown pregnancy status and decision keep the transition probabilities are = the probability of IC = the probability of a positive pregnancy test = the probability of a transition from to AQMHM 06-10-2014 Dias 23 Institut for Produktionsdyr og Heste Transition Probabilities Given state , known pregnancy status and decision keep the transition probabilities are AQMHM 06-10-2014 Dias 24 8

06-10-2014 Institut for Produktionsdyr og Heste Transition Probabilities If decision replace is made the process returned to level 0 with probability 1 If the cow entered the IC state the process returned to level 0 with probability 1 At the end of drying off week the process returned to level 1 (next lactation) AQMHM 06-10-2014 Dias 25 Institut for Produktionsdyr og Heste Rewards Rewards were equal to the expected economic net revenue (NPV) Revenues from Milk • Calves • Carcasses • Minus costs of Feed • New heifer • AQMHM 06-10-2014 Dias 26 Institut for Produktionsdyr og Heste Optimizing the model The objective was to maximize the NPV using a specific discount rate The optimizing was done by using a combination of Value iteration • and Policy iteration • AQMHM 06-10-2014 Dias 27 9

06-10-2014 Institut for Produktionsdyr og Heste Results AQMHM 06-10-2014 Dias 28 Institut for Produktionsdyr og Heste Results Retention pay off (RPO) = NPV ( keep ) – NPV ( replace ) AQMHM 06-10-2014 Dias 29 Institut for Produktionsdyr og Heste Results AQMHM 06-10-2014 Dias 30 10

06-10-2014 Institut for Produktionsdyr og Heste Results AQMHM 06-10-2014 Dias 31 Institut for Produktionsdyr og Heste Results AQMHM 06-10-2014 Dias 32 Institut for Produktionsdyr og Heste Summary The HMDP model updates on a daily basis Herd specific SSM parameters (lactation curves, revenues and expenditures) make the model useable for the farmer Real life validation is expensive and difficult All models are validated through model validation techniques (plots, input data validation, sensitivity analysis and comparison to other models) Seasonal effects are not included Decisions on insemination and treatment not included This HMDP model is a step in the direction of implementation in real herds AQMHM 06-10-2014 Dias 33 11