An An It Iterat erative ive Par Parameter ameter Est Estim - PowerPoint PPT Presentation

An An It Iterat erative ive Par Parameter ameter Est Estim imation ation Metho Me thod d fo for r Bio Biological logical Sy Syste stems ms Xian Yang, Yike Guo, Jeremy Bradley

Overview • Background • Problem Formulation • Parameter Estimation Methods • Experiments and Results • Discussion • Conclusion

Background • To interpret the time evolution of biological systems, scientists build up models, which are their simplified mathematical representations. • These models are essentially a set of equations whose solution describes the evolution, as a function of time, of the state of the system. • They include the following ingredients: • A phase space S • Time t • An evolution law • Example: • dx/dt = X(x) • x t+1 = X( x t )

Background • Mechanistic models of pathways describe the time evolution of molecules and give a detailed insight into pathway dynamics. • Some parameters, such as kinetic rates and initial concentrations, cannot be measured directly by biological experiments. • It is necessary to estimate unknown parameters from the observed system dynamics. • Given the specified structure of pathway and ranges of parameter values to be explored, parameter estimation methods search the parameter space to generate parameters for which the simulated model can exhibit the desired behaviour.

Overview • Background • Problem Formulation • Parameter Estimation Methods • Experiments and Results • Discussion • Conclusion

Problem Formulation • The biological process modelled by a system of differential equations is of the form: where contains the unknown parameters that we seek to estimate, represents the concentration levels of M species involved in the system, such as proteins and mRNA, at time t, and shows the initial concentration levels of these molecules.

Problem Formulation • We represent quantities, , that can be measured experimentally as: where h is the output function that relates the measurement with system state. Usually, can only be measured at discrete time instant where and corrupted with random noise .

Problem Formulation • A simple example: • Model equations: Where

Overview • Background • Problem Formulation • Parameter Estimation Methods • Other methods • Bayesian methods: ABC SMC • Integration of ABC SMC and windowing method • Experiments and Results • Discussion • Conclusion

Parameter Estimation methods – Other methods • Parameter inference methods combine experimental data with the knowledge about the underlying structure of a dynamical system to be investigated. • Optimization methods: • Formulate the parameter estimation problem as a nonlinear optimization problem. Its objective function is the difference between model prediction and the experimental data. • Kalman filters: • Parameter estimation is handled in the framework of control theory by using state observations. • They are recursive estimator that incorporates new information from experimental data.

Parameter Estimation methods – Bayesian Methods • Advantage: • They can infer the whole probability distributions of the parameters, rather than just a point estimate. • Bayes ’ ¡rule ¡for ¡parameter ¡estimation: • Suppose we get the measurable quantities , which reflect the system state , then the posterior distribution of parameters is: that is,

Parameter Estimation methods – Bayesian Methods • Approximate Bayesian computation (ABC): • For complex models, computing the likelihood is time consuming and intractable. • ABC method avoid explicit evaluation of the likelihood by considering distances between observvation and data simulated from a model with parameter . • The generic ABC approach to infer the posterior probability distribution of is:

Parameter Estimation methods – Bayesian Methods θ 2 X(t) t θ 1

Parameter Estimation methods – Bayesian Methods θ 2 X(t) t θ 1

Parameter Estimation methods – Bayesian Methods θ 2 X(t) t θ 1

Parameter Estimation methods – Bayesian Methods θ 2 X(t) t θ 1

Parameter Estimation methods – Bayesian Methods θ 2 X(t) t θ 1

Parameter Estimation methods – Bayesian Methods θ 2 X(t) t θ 1

Parameter Estimation methods – Bayesian Methods • Weakness of the ABC rejection sampler: • High rejection rate • ABC Sequential Monte Carlo (SMC) method: • The ABC SMC method is used to obtain posterior distribution via intermediate distributions , where and P is the total number of intermediate distributions. It should satisfy the condition that ϵ p+1 < ϵ p for all p. • Note that the first step of the ABC SMC method is the ABC rejection algorithm. • For the special case, when the prior distribution of parameter and the perturbation kernel are uniform, the sampling weight becomes 1/N.

Parameter Estimation methods – Bayesian Methods Population 1 Population 2 Population 3 ϵ 1 ϵ 2 ϵ 3 θ 2 θ 2 θ 2 Perturbing Random sampling θ 1 θ 1 θ 1

Parameter Estimation methods – In Inte tegrati gration on of of ABC ABC SMC SMC an and wi d wind ndowing owing me method thod • When the prior knowledge of parameters is poor and the parameter space to be explored is therefore large, the first run of the ABC SMC method, which corresponds to the ABC rejection algorithm, will take a long time to find the first intermediate distribution. • To solve this problem, we introduce a windowing method.

Parameter Estimation methods – In Inte tegrati gration on of of ABC ABC SMC SMC and wi an d wind ndowing owing me method thod • Assume two parameters, θ (1) and θ (2), are to be estimated. The prior distribution of these two parameters are Pr( θ (1))=U( ϴ 1 , ϴ 2 ) and Pr( θ (2))=U( ϴ 3 , ϴ 4 ) the error allowance is ϵ .

Parameter Estimation methods – In Inte tegrati gration on of of ABC ABC SMC SMC an and wi d wind ndowing owing me method thod • The parameter estimation method proposed in this paper is the combination of a windowing method and ABC SMC. • The windowing method is applied first to reduce the parameter space to be explored by ABC SMC. The process of this integrated method is as follows: 1. Using the windowing method to get a better prior knowledge of parameters. 2. By running the ABC SMC scheme, the target posterior distribution of parameters is approached by a set of intermediate distributions.

Parameter Estimation methods – In Inte tegrati gration on of of ABC ABC SMC SMC an and wi d wind ndowing owing me method thod • In this paper, the prior distribution of parameters is set to be uniform, and the perturbation kernel in SMC is also uniform. • Therefore, the sampling weight equals to 1/N for all p and n. • The ABC SMC method can be improved if we adjust the weight value corresponding to the distance.

Overview • Background • Problem Formulation • Parameter Estimation Methods • Experiments and Results • Model of a biological system • Results of the windowing method • Results of the ABC SMC method using equal sampling weight • Results of the ABC SMC method using adaptive sampling weight • Discussion • Conclusion

Experiments and Results – Mod Model el of of a bi a biologi ological cal sy system stem • In this paper, we infer the parameters of a standard repressilator model. • This model consists of six differential equations that represent the concentration levels of three mRNA (where i ϵ {lacl, tetR, cl}) and three repressor proteins (where j ϵ {cl, lacl, tetR}). The mathematical model that represents this system is: where

Experiments and Results – Settings of experiment • Settings of experiment: • Suppose we are able to measure the concentration level of mRNA at discrete time instants . • The noise in the measurement is assumed to give a standard derivation of 5% of the mean of the signal. • The initial concentrations of these six species are: • The model is simulated by MATLAB’s ¡ODE45 ¡solver . • In the simulation, the parameters that generate observations are: • The distance between the noisy observation and the solution of the system for is: • The prior distribution of each parameter is assumed to be uniform that:

Experiments and Results – Settings of experiment • The oscillatory dynamics of the repressilator system with original experimental settings. measured m lacl 80 measured m tetR 70 measured m cl m lacl 60 m tetR m cl Concentrations (arbitrary units) 50 40 30 20 10 0 -10 0 5 10 15 20 25 30 Time(min)

Experiments and Results – Results of the windowing method 1. Results of the windowing method: • Partition the whole large space into small regions whose central points are: • We set the error threshold to be 35000 initially which is much larger than the desired value. • Use the performance of the central point of each region to represent the whole region. • 62 parameter points are selected (acceptance rate is 0.22%) with the distance value smaller than 35000. • The ranges of each parameter after windowing method are : • After using the windowing method, the prior distributions of parameters which are the inputs of the ABC SMC method are: