a new method for bound constrained derivative free global
play

A New Method for Bound-Constrained Derivative-Free Global - PowerPoint PPT Presentation

Jul 24, 2023 •250 likes •1.45k views

A New Method for Bound-Constrained Derivative-Free Global Optimization and its Application to Parameter Estimation in Astrophysics A. Ismael F. Vaz 1 Lus Nunes Vicente 2 Joo Manuel Fernandes 3 2 Production an Systems Department, University of

  1. Particle swarm New position and velocity The new particle position is updated by Update particle x p ( t + 1) = x p ( t ) + v p ( t + 1) , where v p ( t + 1) is the new velocity given by Update velocity � � � � v p j ( t + 1) = ι ( t ) v p y p j ( t ) − x p y j ( t ) − x p j ( t ) + µω 1 j ( t ) j ( t ) + νω 2 j ( t ) ˆ j ( t ) , for j = 1 , . . . , n . ι ( t ) is the inertial factor µ is the cognitive parameter and ν is the social parameter ω 1 j ( t ) and ω 2 j ( t ) are random numbers drawn from the uniform (0 , 1) distribution. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 7 / 57

  2. Particle swarm New position and velocity The new particle position is updated by Update particle x p ( t + 1) = x p ( t ) + v p ( t + 1) , where v p ( t + 1) is the new velocity given by Update velocity � � � � v p j ( t + 1) = ι ( t ) v p y p j ( t ) − x p y j ( t ) − x p j ( t ) + µω 1 j ( t ) j ( t ) + νω 2 j ( t ) ˆ j ( t ) , for j = 1 , . . . , n . ι ( t ) is the inertial factor µ is the cognitive parameter and ν is the social parameter ω 1 j ( t ) and ω 2 j ( t ) are random numbers drawn from the uniform (0 , 1) distribution. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 7 / 57

  3. Particle swarm New position and velocity The new particle position is updated by Update particle x p ( t + 1) = x p ( t ) + v p ( t + 1) , where v p ( t + 1) is the new velocity given by Update velocity � � � � v p j ( t + 1) = ι ( t ) v p y p j ( t ) − x p y j ( t ) − x p j ( t ) + µω 1 j ( t ) j ( t ) + νω 2 j ( t ) ˆ j ( t ) , for j = 1 , . . . , n . ι ( t ) is the inertial factor µ is the cognitive parameter and ν is the social parameter ω 1 j ( t ) and ω 2 j ( t ) are random numbers drawn from the uniform (0 , 1) distribution. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 7 / 57

  4. Particle swarm Handling bound constraints In particle swarm, simple bound constraints are handled by a projection onto Ω = { x ∈ R n : ℓ ≤ x ≤ u } , for all particles i = 1 , . . . , s . Projection if x i  ℓ j j ( t ) < ℓ j ,  proj Ω ( x i if x i j ( t )) = u j j ( t ) > u j , x i j ( t ) otherwise,  for j = 1 , . . . , n . Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 8 / 57

  5. Particle swarm Example iter=1, best fx=−0.6836, nfx=36 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 9 / 57

  6. Particle swarm Example iter=11, best fx=−0.0131, nfx=396 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 10 / 57

  7. Particle swarm Example iter=21, best fx=−0.0131, nfx=756 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 11 / 57

  8. Particle swarm Example iter=31, best fx=−0.0074, nfx=1116 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 12 / 57

  9. Particle swarm Example iter=41, best fx=−0.0040, nfx=1476 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 13 / 57

  10. Particle swarm Example iter=51, best fx=−0.0040, nfx=1836 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 14 / 57

  11. Particle swarm Example iter=271, best fx=−0.0000, nfx=9756 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 15 / 57

  12. Particle swarm Example iter=871, best fx=−0.0000, nfx=31356 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 16 / 57

  13. Particle swarm Example iter=1181, best fx=−0.0000, nfx=42516 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 17 / 57

  14. Particle swarm Some properties Easy to implement. Easy to deal with discrete variables. Easy to parallelize. For a correct choice of parameters the algorithm terminates ( lim t → + ∞ v ( t ) = 0 ). Uses only objective function values. Convergence for a global optimum under strong assumptions (unpractical). High number of function evaluations. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 18 / 57

  15. Particle swarm Some properties Easy to implement. Easy to deal with discrete variables. Easy to parallelize. For a correct choice of parameters the algorithm terminates ( lim t → + ∞ v ( t ) = 0 ). Uses only objective function values. Convergence for a global optimum under strong assumptions (unpractical). High number of function evaluations. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 18 / 57

  16. Particle swarm Some properties Easy to implement. Easy to deal with discrete variables. Easy to parallelize. For a correct choice of parameters the algorithm terminates ( lim t → + ∞ v ( t ) = 0 ). Uses only objective function values. Convergence for a global optimum under strong assumptions (unpractical). High number of function evaluations. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 18 / 57

  17. Particle swarm Some properties Easy to implement. Easy to deal with discrete variables. Easy to parallelize. For a correct choice of parameters the algorithm terminates ( lim t → + ∞ v ( t ) = 0 ). Uses only objective function values. Convergence for a global optimum under strong assumptions (unpractical). High number of function evaluations. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 18 / 57

  18. Particle swarm Some properties Easy to implement. Easy to deal with discrete variables. Easy to parallelize. For a correct choice of parameters the algorithm terminates ( lim t → + ∞ v ( t ) = 0 ). Uses only objective function values. Convergence for a global optimum under strong assumptions (unpractical). High number of function evaluations. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 18 / 57

  19. Particle swarm Some properties Easy to implement. Easy to deal with discrete variables. Easy to parallelize. For a correct choice of parameters the algorithm terminates ( lim t → + ∞ v ( t ) = 0 ). Uses only objective function values. Convergence for a global optimum under strong assumptions (unpractical). High number of function evaluations. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 18 / 57

  20. Particle swarm Some properties Easy to implement. Easy to deal with discrete variables. Easy to parallelize. For a correct choice of parameters the algorithm terminates ( lim t → + ∞ v ( t ) = 0 ). Uses only objective function values. Convergence for a global optimum under strong assumptions (unpractical). High number of function evaluations. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 18 / 57

  21. Coordinate search Outline Introduction 1 Particle swarm 2 Coordinate search 3 The hybrid algorithm 4 Numerical results with a set of test problems 5 Parameter estimation in Astrophysics 6 Conclusions and future work 7 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 19 / 57

  22. Coordinate search Introduction to direct search methods Direct search methods are an important class of optimization methods that try to minimize a function by comparing objective function values at a finite number of points. Direct search methods do not use derivative information of the objective function nor try to approximate it. Coordinate search is a simple direct search method. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 20 / 57

  23. Coordinate search Introduction to direct search methods Direct search methods are an important class of optimization methods that try to minimize a function by comparing objective function values at a finite number of points. Direct search methods do not use derivative information of the objective function nor try to approximate it. Coordinate search is a simple direct search method. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 20 / 57

  24. Coordinate search Introduction to direct search methods Direct search methods are an important class of optimization methods that try to minimize a function by comparing objective function values at a finite number of points. Direct search methods do not use derivative information of the objective function nor try to approximate it. Coordinate search is a simple direct search method. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 20 / 57

  25. Coordinate search Some definitions Positive maximal basis Formed by the coordinate vectors and their negative counterparts: D ⊕ = { e 1 , . . . , e n , − e 1 , . . . , − e n } . D ⊕ spans R n with nonnegative coefficients. Coordinate search The direct search method based on D ⊕ is known as coordinate or compass search. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 21 / 57

  26. Coordinate search Some definitions Positive maximal basis Formed by the coordinate vectors and their negative counterparts: D ⊕ = { e 1 , . . . , e n , − e 1 , . . . , − e n } . D ⊕ spans R n with nonnegative coefficients. Coordinate search The direct search method based on D ⊕ is known as coordinate or compass search. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 21 / 57

  27. Coordinate search Some definitions Sets Given D ⊕ and the current point y ( t ) , two sets of points are defined: a grid M t and the poll set P t . The grid M t is given by � � y ( t ) + α ( t ) D ⊕ z, z ∈ N | D ⊕ | M t = , 0 where α ( t ) > 0 is the grid size parameter. The poll set is given by P t = { y ( t ) + α ( t ) d, d ∈ D ⊕ } . Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 22 / 57

  28. Coordinate search Example of M t and P t y(t)+ α (t)e 2 y(t)− α (t)e 1 y(t)+ α (t)e 1 y(t) y(t)− α (t)e 2 The grid M t and the set P t when D ⊕ = { e 1 , e 2 , − e 1 , − e 2 } Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 23 / 57

  29. Coordinate search Coordinate search The search step conducts a finite search on the grid M t . If no success is obtained in the search step then a poll step follows. The poll step evaluates the objective function at the elements of P t , searching for points which have a lower objective function value. If success is attained, the value of α ( t ) may be increased, otherwise it is reduced. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 24 / 57

  30. Coordinate search Coordinate search The search step conducts a finite search on the grid M t . If no success is obtained in the search step then a poll step follows. The poll step evaluates the objective function at the elements of P t , searching for points which have a lower objective function value. If success is attained, the value of α ( t ) may be increased, otherwise it is reduced. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 24 / 57

  31. Coordinate search Coordinate search The search step conducts a finite search on the grid M t . If no success is obtained in the search step then a poll step follows. The poll step evaluates the objective function at the elements of P t , searching for points which have a lower objective function value. If success is attained, the value of α ( t ) may be increased, otherwise it is reduced. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 24 / 57

  32. Coordinate search Coordinate search The search step conducts a finite search on the grid M t . If no success is obtained in the search step then a poll step follows. The poll step evaluates the objective function at the elements of P t , searching for points which have a lower objective function value. If success is attained, the value of α ( t ) may be increased, otherwise it is reduced. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 24 / 57

  33. Coordinate search Handling bound constraints For the coordinate search method it is sufficient to initialize the algorithm with a feasible initial guess ( y (0) ∈ Ω ) and to use ˆ f as the objective function. Penalty/Barrier function � f ( z ) if z ∈ Ω , ˆ f ( z ) = + ∞ otherwise. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 25 / 57

  34. The hybrid algorithm Outline Introduction 1 Particle swarm 2 Coordinate search 3 The hybrid algorithm 4 Numerical results with a set of test problems 5 Parameter estimation in Astrophysics 6 Conclusions and future work 7 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 26 / 57

  35. The hybrid algorithm Motivation for PSwarm Hybrid algorithm The hybrid algorithm tries to combine the best of both algorithms. From particle swarm The particle swarm ability of searching for the global optimum. From coordinate search The guarantee to obtain at least a stationary point. Some robustness. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 27 / 57

  36. The hybrid algorithm Motivation for PSwarm Hybrid algorithm The hybrid algorithm tries to combine the best of both algorithms. From particle swarm The particle swarm ability of searching for the global optimum. From coordinate search The guarantee to obtain at least a stationary point. Some robustness. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 27 / 57

  37. The hybrid algorithm Motivation for PSwarm Hybrid algorithm The hybrid algorithm tries to combine the best of both algorithms. From particle swarm The particle swarm ability of searching for the global optimum. From coordinate search The guarantee to obtain at least a stationary point. Some robustness. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 27 / 57

  38. The hybrid algorithm Motivation for PSwarm Central idea A particle swarm iteration is performed in the search step and if no progress is attained a poll step is taken. Key points In the first iterations the algorithm takes advantage of the particle swarm ability to find a global optimum (exploiting the search space), while in the last iterations the algorithm takes advantage of the pattern search robustness to find a stationary point. The number of particles in the swarm search can be decreased along the iterations (no need to have a large number of particles around a local optimum). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 28 / 57

  39. The hybrid algorithm Motivation for PSwarm Central idea A particle swarm iteration is performed in the search step and if no progress is attained a poll step is taken. Key points In the first iterations the algorithm takes advantage of the particle swarm ability to find a global optimum (exploiting the search space), while in the last iterations the algorithm takes advantage of the pattern search robustness to find a stationary point. The number of particles in the swarm search can be decreased along the iterations (no need to have a large number of particles around a local optimum). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 28 / 57

  40. The hybrid algorithm Motivation for PSwarm Central idea A particle swarm iteration is performed in the search step and if no progress is attained a poll step is taken. Key points In the first iterations the algorithm takes advantage of the particle swarm ability to find a global optimum (exploiting the search space), while in the last iterations the algorithm takes advantage of the pattern search robustness to find a stationary point. The number of particles in the swarm search can be decreased along the iterations (no need to have a large number of particles around a local optimum). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 28 / 57

  41. The hybrid algorithm Motivation for PSwarm Central idea A particle swarm iteration is performed in the search step and if no progress is attained a poll step is taken. Key points In the first iterations the algorithm takes advantage of the particle swarm ability to find a global optimum (exploiting the search space), while in the last iterations the algorithm takes advantage of the pattern search robustness to find a stationary point. The number of particles in the swarm search can be decreased along the iterations (no need to have a large number of particles around a local optimum). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 28 / 57

  42. The hybrid algorithm Example iter=1, best fx=12.3615, pollsteps=0, suc=0, delta=0.81920000 nfx=20 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 29 / 57

  43. The hybrid algorithm Example iter=2, best fx=2.2032, pollsteps=1, suc=1, delta=0.81920000 nfx=43 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 30 / 57

  44. The hybrid algorithm Example iter=3, best fx=1.2456, pollsteps=1, suc=1, delta=0.81920000 nfx=60 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 31 / 57

  45. The hybrid algorithm Example iter=4, best fx=0.3038, pollsteps=1, suc=1, delta=0.81920000 nfx=70 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 32 / 57

  46. The hybrid algorithm Example iter=5, best fx=0.3038, pollsteps=2, suc=1, delta=0.40960000 nfx=81 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 33 / 57

  47. The hybrid algorithm Example iter=6, best fx=0.3038, pollsteps=3, suc=1, delta=0.20480000 nfx=90 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 34 / 57

  48. The hybrid algorithm Example iter=61, best fx=0.0543, pollsteps=58, suc=42, delta=0.00160000 nfx=400 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 35 / 57

  49. The hybrid algorithm Example iter=511, best fx=0.0264, pollsteps=211, suc=149, delta=0.40960000 nfx=1206 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 36 / 57

  50. The hybrid algorithm Example iter=6506, best fx=0.0018, pollsteps=1120, suc=499, delta=0.81920000 nfx=9997 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2 −1.5 −1 −0.5 0 0.5 1 1.5 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 37 / 57

  51. Numerical results with a set of test problems Outline Introduction 1 Particle swarm 2 Coordinate search 3 The hybrid algorithm 4 Numerical results with a set of test problems 5 Parameter estimation in Astrophysics 6 Conclusions and future work 7 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 38 / 57

  52. Numerical results with a set of test problems Test problems 122 problems were collected from the global optimization literature. 12 problems of large dimension (between 100 and 300 variables). The others are small ( < 10 ) and medium size ( < 30 ). Majority of objective functions are differentiable, but non-convex. All problems have simple bounds on the variables (needed for the search step — particle swarm). The test problems were coded in AMPL ( A Modeling Language for Mathematical Programming ). Test problems available on http://www.norg.uminho.pt/aivaz (under software ). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 39 / 57

  53. Numerical results with a set of test problems Test problems 122 problems were collected from the global optimization literature. 12 problems of large dimension (between 100 and 300 variables). The others are small ( < 10 ) and medium size ( < 30 ). Majority of objective functions are differentiable, but non-convex. All problems have simple bounds on the variables (needed for the search step — particle swarm). The test problems were coded in AMPL ( A Modeling Language for Mathematical Programming ). Test problems available on http://www.norg.uminho.pt/aivaz (under software ). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 39 / 57

  54. Numerical results with a set of test problems Test problems 122 problems were collected from the global optimization literature. 12 problems of large dimension (between 100 and 300 variables). The others are small ( < 10 ) and medium size ( < 30 ). Majority of objective functions are differentiable, but non-convex. All problems have simple bounds on the variables (needed for the search step — particle swarm). The test problems were coded in AMPL ( A Modeling Language for Mathematical Programming ). Test problems available on http://www.norg.uminho.pt/aivaz (under software ). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 39 / 57

  55. Numerical results with a set of test problems Test problems 122 problems were collected from the global optimization literature. 12 problems of large dimension (between 100 and 300 variables). The others are small ( < 10 ) and medium size ( < 30 ). Majority of objective functions are differentiable, but non-convex. All problems have simple bounds on the variables (needed for the search step — particle swarm). The test problems were coded in AMPL ( A Modeling Language for Mathematical Programming ). Test problems available on http://www.norg.uminho.pt/aivaz (under software ). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 39 / 57

  56. Numerical results with a set of test problems Test problems 122 problems were collected from the global optimization literature. 12 problems of large dimension (between 100 and 300 variables). The others are small ( < 10 ) and medium size ( < 30 ). Majority of objective functions are differentiable, but non-convex. All problems have simple bounds on the variables (needed for the search step — particle swarm). The test problems were coded in AMPL ( A Modeling Language for Mathematical Programming ). Test problems available on http://www.norg.uminho.pt/aivaz (under software ). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 39 / 57

  57. Numerical results with a set of test problems Test problems 122 problems were collected from the global optimization literature. 12 problems of large dimension (between 100 and 300 variables). The others are small ( < 10 ) and medium size ( < 30 ). Majority of objective functions are differentiable, but non-convex. All problems have simple bounds on the variables (needed for the search step — particle swarm). The test problems were coded in AMPL ( A Modeling Language for Mathematical Programming ). Test problems available on http://www.norg.uminho.pt/aivaz (under software ). Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 39 / 57

  58. Numerical results with a set of test problems Average objective value Average objective value of 30 runs with maxf=1000 (7500) 1 1 0.9 0.95 0.8 0.9 0.7 0.85 0.6 ρ 0.5 0.8 ρ 0.4 0.75 0.3 0.7 0.2 ASA PSwarm 0.65 PGAPack 0.1 Direct MCS 0 0.6 1 2 3 4 5 6 7 8 9 10 200 400 600 τ τ Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 40 / 57

  59. Numerical results with a set of test problems Average of objective function evaluations Average objective evaluation of 30 runs with maxf=1000 1 1 0.9 0.95 0.8 0.9 0.7 0.85 0.6 ρ 0.5 0.8 ρ 0.4 0.75 0.3 0.7 0.2 ASA PSwarm 0.65 PGAPack 0.1 Direct MCS 0 0.6 5 10 15 20 25 30 35 200 400 600 τ τ Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 41 / 57

  60. Numerical results with a set of test problems Average number of objective function evaluations maxf ASA PGAPack PSwarm Direct MCS 1009 ∗ 1107 ∗ 1837 ∗ 1000 857 686 10009 ∗ 11517 ∗ 10000 5047 3603 4469 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 42 / 57

  61. Numerical results with a set of test problems Coordinate search vs Particle swarm vs PSwarm Average objective value of 30 runs with maxf=1000 (7500) 1 1 0.98 0.9 0.96 0.94 0.8 0.92 ρ 0.7 0.9 ρ 0.88 0.6 0.86 0.84 0.5 CS 0.82 PSwarm PSOA 0.4 0.8 1 2 3 4 5 6 7 8 9 10 200 400 600 τ τ Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 43 / 57

  62. Parameter estimation in Astrophysics Outline Introduction 1 Particle swarm 2 Coordinate search 3 The hybrid algorithm 4 Numerical results with a set of test problems 5 Parameter estimation in Astrophysics 6 Conclusions and future work 7 Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 44 / 57

  63. Parameter estimation in Astrophysics The problem Objective To determine a set of stellar parameters (that define the star internal structure and evolution) from observable information. Set of parameters to be determined M — stellar mass (relative to Sun mass M ⊙ ). X — abundance of hydrogen ( % ). Y — abundance of helium ( % ). Z — abundance of other elements ( Z = 100% − X − Y ). t — star age (in Gyr = 1000 million years). two other parameters. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 45 / 57

  64. Parameter estimation in Astrophysics The problem Objective To determine a set of stellar parameters (that define the star internal structure and evolution) from observable information. Set of parameters to be determined M — stellar mass (relative to Sun mass M ⊙ ). X — abundance of hydrogen ( % ). Y — abundance of helium ( % ). Z — abundance of other elements ( Z = 100% − X − Y ). t — star age (in Gyr = 1000 million years). two other parameters. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 45 / 57

  65. Parameter estimation in Astrophysics The problem Objective To determine a set of stellar parameters (that define the star internal structure and evolution) from observable information. Set of parameters to be determined M — stellar mass (relative to Sun mass M ⊙ ). X — abundance of hydrogen ( % ). Y — abundance of helium ( % ). Z — abundance of other elements ( Z = 100% − X − Y ). t — star age (in Gyr = 1000 million years). two other parameters. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 45 / 57

  66. Parameter estimation in Astrophysics The problem Objective To determine a set of stellar parameters (that define the star internal structure and evolution) from observable information. Set of parameters to be determined M — stellar mass (relative to Sun mass M ⊙ ). X — abundance of hydrogen ( % ). Y — abundance of helium ( % ). Z — abundance of other elements ( Z = 100% − X − Y ). t — star age (in Gyr = 1000 million years). two other parameters. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 45 / 57

  67. Parameter estimation in Astrophysics The problem Objective To determine a set of stellar parameters (that define the star internal structure and evolution) from observable information. Set of parameters to be determined M — stellar mass (relative to Sun mass M ⊙ ). X — abundance of hydrogen ( % ). Y — abundance of helium ( % ). Z — abundance of other elements ( Z = 100% − X − Y ). t — star age (in Gyr = 1000 million years). two other parameters. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 45 / 57

  68. Parameter estimation in Astrophysics The problem Objective To determine a set of stellar parameters (that define the star internal structure and evolution) from observable information. Set of parameters to be determined M — stellar mass (relative to Sun mass M ⊙ ). X — abundance of hydrogen ( % ). Y — abundance of helium ( % ). Z — abundance of other elements ( Z = 100% − X − Y ). t — star age (in Gyr = 1000 million years). two other parameters. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 45 / 57

  69. Parameter estimation in Astrophysics The problem Objective To determine a set of stellar parameters (that define the star internal structure and evolution) from observable information. Set of parameters to be determined M — stellar mass (relative to Sun mass M ⊙ ). X — abundance of hydrogen ( % ). Y — abundance of helium ( % ). Z — abundance of other elements ( Z = 100% − X − Y ). t — star age (in Gyr = 1000 million years). two other parameters. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 45 / 57

  70. Parameter estimation in Astrophysics The problem Objective To determine a set of stellar parameters (that define the star internal structure and evolution) from observable information. Set of parameters to be determined M — stellar mass (relative to Sun mass M ⊙ ). X — abundance of hydrogen ( % ). Y — abundance of helium ( % ). Z — abundance of other elements ( Z = 100% − X − Y ). t — star age (in Gyr = 1000 million years). two other parameters. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 45 / 57

  71. Parameter estimation in Astrophysics The problem Observable data from spectrum analysis t eff — stellar surface temperature. lum — total stellar luminosity. � Z � — relation between the abundance of other elements and X hydrogen. g — surface gravity (less accurate). Parameters and observable data for Sun M = 1 and t = 4 . 6 Gyr, with t eff = 5777 , lum = 1 and Z/X = 0 . 0245 . This information is only available for Sun. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 46 / 57

  72. Parameter estimation in Astrophysics The problem Observable data from spectrum analysis t eff — stellar surface temperature. lum — total stellar luminosity. � Z � — relation between the abundance of other elements and X hydrogen. g — surface gravity (less accurate). Parameters and observable data for Sun M = 1 and t = 4 . 6 Gyr, with t eff = 5777 , lum = 1 and Z/X = 0 . 0245 . This information is only available for Sun. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 46 / 57

  73. Parameter estimation in Astrophysics The problem Observable data from spectrum analysis t eff — stellar surface temperature. lum — total stellar luminosity. � Z � — relation between the abundance of other elements and X hydrogen. g — surface gravity (less accurate). Parameters and observable data for Sun M = 1 and t = 4 . 6 Gyr, with t eff = 5777 , lum = 1 and Z/X = 0 . 0245 . This information is only available for Sun. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 46 / 57

  74. Parameter estimation in Astrophysics The problem Observable data from spectrum analysis t eff — stellar surface temperature. lum — total stellar luminosity. � Z � — relation between the abundance of other elements and X hydrogen. g — surface gravity (less accurate). Parameters and observable data for Sun M = 1 and t = 4 . 6 Gyr, with t eff = 5777 , lum = 1 and Z/X = 0 . 0245 . This information is only available for Sun. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 46 / 57

  75. Parameter estimation in Astrophysics The problem Observable data from spectrum analysis t eff — stellar surface temperature. lum — total stellar luminosity. � Z � — relation between the abundance of other elements and X hydrogen. g — surface gravity (less accurate). Parameters and observable data for Sun M = 1 and t = 4 . 6 Gyr, with t eff = 5777 , lum = 1 and Z/X = 0 . 0245 . This information is only available for Sun. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 46 / 57

  76. Parameter estimation in Astrophysics The problem Observable data from spectrum analysis t eff — stellar surface temperature. lum — total stellar luminosity. � Z � — relation between the abundance of other elements and X hydrogen. g — surface gravity (less accurate). Parameters and observable data for Sun M = 1 and t = 4 . 6 Gyr, with t eff = 5777 , lum = 1 and Z/X = 0 . 0245 . This information is only available for Sun. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 46 / 57

  77. Parameter estimation in Astrophysics The optimization problem The optimization problem � 2 � 2 � t eff − t eff,obs � lum − lum obs min + δt eff,obs δlum obs M,t,X,Y � 1 − X − Y � Z � 2 � � 2 − � g − g obs X X obs + + � Z � δg obs δ X obs Given M , t and fixing X , Y ( α and ov ) the parameters t eff , lum and g are computed by simulating (CESAM code) a system of differentiable equations. The equations of internal structure are five: conservation of mass and energy, hydrostatic equilibrium, energy transport, production and destruction of chemical elements by thermonuclear reactions. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 47 / 57

  78. Parameter estimation in Astrophysics The optimization problem The optimization problem � 2 � 2 � t eff − t eff,obs � lum − lum obs min + δt eff,obs δlum obs M,t,X,Y � 1 − X − Y � Z � 2 � � 2 − � g − g obs X X obs + + � Z � δg obs δ X obs Given M , t and fixing X , Y ( α and ov ) the parameters t eff , lum and g are computed by simulating (CESAM code) a system of differentiable equations. The equations of internal structure are five: conservation of mass and energy, hydrostatic equilibrium, energy transport, production and destruction of chemical elements by thermonuclear reactions. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 47 / 57

  79. Parameter estimation in Astrophysics The optimization problem The optimization problem � 2 � 2 � t eff − t eff,obs � lum − lum obs min + δt eff,obs δlum obs M,t,X,Y � 1 − X − Y � Z � 2 � � 2 − � g − g obs X X obs + + � Z � δg obs δ X obs Given M , t and fixing X , Y ( α and ov ) the parameters t eff , lum and g are computed by simulating (CESAM code) a system of differentiable equations. The equations of internal structure are five: conservation of mass and energy, hydrostatic equilibrium, energy transport, production and destruction of chemical elements by thermonuclear reactions. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 47 / 57

  80. Parameter estimation in Astrophysics Numerical results Getting t eff , lum and g – CESAM t eff , lum and g are computed by CESAM (Fortran 77 code), which is viewed as a black box function for the optimization process. Optimization solver – PSwarm PSwarm (C code). Solver used with default options. Linking PSwarm and CESAM Optimization solver communicates with CESAM by input and output files. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 48 / 57

  81. Parameter estimation in Astrophysics Numerical results Getting t eff , lum and g – CESAM t eff , lum and g are computed by CESAM (Fortran 77 code), which is viewed as a black box function for the optimization process. Optimization solver – PSwarm PSwarm (C code). Solver used with default options. Linking PSwarm and CESAM Optimization solver communicates with CESAM by input and output files. Vaz, Vicente, Fernandes (Opt 2007) Global optimization in Astrophysics July 22-25, 2007 48 / 57

Recommend

Extension of PSwarm to Linearly Constrained Derivative-Free Global Optimization A. Ismael F. Vaz 1

Extension of PSwarm to Linearly Constrained Derivative-Free Global Optimization A. Ismael F. Vaz 1

Extension of PSwarm to Linearly Constrained Derivative-Free Global Optimization A. Ismael F. Vaz 1 and Lus Nunes Vicente 2 1 University of Minho - Portugal aivaz@dps.uminho.pt 2 University of Coimbra - Portugal lnv@mat.uc.pt SIAM Conference on

1.2k views • 81 slides
Bound-free pair production in heavy-ion collisions at high energies Rainer Schicker Phys. Inst.,

Bound-free pair production in heavy-ion collisions at high energies Rainer Schicker Phys. Inst.,

Secondary beams Bound-free pair prod. Double bound-free pair prod. A trigger Constrained luminosity Physics interest Bound-free pair production in heavy-ion collisions at high energies Rainer Schicker Phys. Inst., Heidelberg Juni 19, 2015

240 views • 8 slides
11. Equality constrained minimization equality constrained minimization eliminating

11. Equality constrained minimization equality constrained minimization eliminating

Convex Optimization Boyd &amp; Vandenberghe 11. Equality constrained minimization equality constrained minimization eliminating equality constraints Newtons method with equality constraints infeasible start Newton method

445 views • 19 slides
The Probabilistic Method Techniques Union bound Argument from expectation Alterations The

The Probabilistic Method Techniques Union bound Argument from expectation Alterations The

The Probabilistic Method Techniques Union bound Argument from expectation Alterations The second moment method The (Lovasz) Local Lemma And much more Alon and Spencer, The Probabilistic Method Bolobas, Random Graphs Hung Q.

258 views • 9 slides
A high-order unstaggered constrained transport method for the 3D ideal magnetohydrodynamic

A high-order unstaggered constrained transport method for the 3D ideal magnetohydrodynamic

A high-order unstaggered constrained transport method for the 3D ideal magnetohydrodynamic equations based on the method of lines Bertram Taetz (joint work with C. Helzel &amp; J.A. Rossmanith) Department of Numerical Mathematics

486 views • 37 slides
Finding max/min under constraint The behaviour of economic actors is often constrained by the

Finding max/min under constraint The behaviour of economic actors is often constrained by the

Topic 9: Constrained Optimisation Reading: Jacques: Chapter 5.5, 5.6 1. Finding max/min under constraint: Substitution Method Lagrange Multiplier Method 2. Application I: Utility 3. Application II: Production Finding max/min

239 views • 12 slides
MATHEMATICS 1 CONTENTS Unconstrained optimization Constrained optimization Lagrange method

MATHEMATICS 1 CONTENTS Unconstrained optimization Constrained optimization Lagrange method

Constrained optimization BUSINESS MATHEMATICS 1 CONTENTS Unconstrained optimization Constrained optimization Lagrange method Old exam question Further study 2 UNCONSTRAINED OPTIMIZATION Recall extreme values in two dimensions

599 views • 25 slides
Branch-and-Bound Math 482, Lecture 33 Misha Lavrov April 27, 2020 Branch-and-bound methods

Branch-and-Bound Math 482, Lecture 33 Misha Lavrov April 27, 2020 Branch-and-bound methods

Branch-and-bound methods Example The general method Branch-and-Bound Math 482, Lecture 33 Misha Lavrov April 27, 2020 Branch-and-bound methods Example The general method Branch-and-bound The branch-and-bound method is a general strategy

768 views • 49 slides
First-order logic: Free and bound variables. Scope of a quantifier. Substitution of terms for

First-order logic: Free and bound variables. Scope of a quantifier. Substitution of terms for

First-order logic: Free and bound variables. Scope of a quantifier. Substitution of terms for variables. Capture. Variable renaming. Valentin Goranko DTU Informatics September 2010 V Goranko Free and bound variables Two essentially

143 views • 13 slides
JUST THE MATHS SLIDES NUMBER 14.11 PARTIAL DIFFERENTIATION 11 (Constrained maxima and

JUST THE MATHS SLIDES NUMBER 14.11 PARTIAL DIFFERENTIATION 11 (Constrained maxima and

JUST THE MATHS SLIDES NUMBER 14.11 PARTIAL DIFFERENTIATION 11 (Constrained maxima and minima) by A.J.Hobson 14.11.1 The substitution method 14.11.2 The method of Lagrange multipliers UNIT 14.11 PARTIAL DIFFERENTIATION 11 CONSTRAINED

666 views • 14 slides
Efficient Derivative Pricing by the Extended Method of Moments Patrick GAGLIARDINI University of

Efficient Derivative Pricing by the Extended Method of Moments Patrick GAGLIARDINI University of

Efficient Derivative Pricing by the Extended Method of Moments Patrick GAGLIARDINI University of Lugano and Swiss Finance Institute Joint work with Christian GOURIEROUX 1 and Eric RENAULT 2 1 CREST (Paris), CEPREMAP and University of Toronto 2

339 views • 20 slides
Optimization Unconstrained optimization Constrained optimization Newton with equality

Optimization Unconstrained optimization Constrained optimization Newton with equality

Optimization Unconstrained optimization Constrained optimization Newton with equality One-dimensional constraints Multi-dimensional Active-set method Newtons method Descent methods Basic Newton Gradient Simplex method Gauss-

795 views • 55 slides
Integer Programming Part 2 Assoc. Prof. Dr. Arslan M. RNEK 1 9.3. Branch-and-Bound Method

Integer Programming Part 2 Assoc. Prof. Dr. Arslan M. RNEK 1 9.3. Branch-and-Bound Method

Chapter 9 Integer Programming Part 2 Assoc. Prof. Dr. Arslan M. RNEK 1 9.3. Branch-and-Bound Method (Pure IP) Branch-and-Bound methods find the optimal solution to an IP by efficiently enumerating the points in a subproblems feasible

539 views • 31 slides
Ensemble Kalman Inversion Derivative-Free Optimization Andrew Stuart Computing and Mathematical

Ensemble Kalman Inversion Derivative-Free Optimization Andrew Stuart Computing and Mathematical

Ensemble Kalman Inversion Derivative-Free Optimization Andrew Stuart Computing and Mathematical Sciences California Institute of Technology Allen Philanthropies, Mission Control for Earth, NSF, ONR, Schmidt Futures. ISDA 2019, Kobe, Japan

365 views • 33 slides
Ghost-free vector superfield actions in supersymmetric higher-derivative theories Ryo Yokokura

Ghost-free vector superfield actions in supersymmetric higher-derivative theories Ryo Yokokura

Ghost-free vector superfield actions in supersymmetric higher-derivative theories Ryo Yokokura (Keio) 2018. 8. 2 Strings and fields @ YITP with T. Fujimori, M. Nitta, K. Ohashi, Y. Yamada based on JHEP 1709 (2017)143, [arXiv:1708.05129] What

289 views • 17 slides
An asynchronous parallel derivative-free algorithm for handling general constraints Josh Griffin

An asynchronous parallel derivative-free algorithm for handling general constraints Josh Griffin

An asynchronous parallel derivative-free algorithm for handling general constraints Josh Griffin Computational Sciences and Mathematics Research Sandia National Laboratories Livermore, California USA Second International Congress on

805 views • 67 slides
2. Theory of the Derivative 2.1 Tangent Lines 2.2 Definition of Derivative 2.3 Rates of Change

2. Theory of the Derivative 2.1 Tangent Lines 2.2 Definition of Derivative 2.3 Rates of Change

2. Theory of the Derivative 2.1 Tangent Lines 2.2 Definition of Derivative 2.3 Rates of Change 2.4 Derivative Rules 2.5 Higher Order Derivatives 2.6 Implicit Differentiation 2.7 LHpitals Rule 2.8 Some Classic Theoretical Results

1.08k views • 105 slides
Derivative-Free Robust Optimization by Outer Approximations Stefan Wild Mathematics and Computer

Derivative-Free Robust Optimization by Outer Approximations Stefan Wild Mathematics and Computer

Derivative-Free Robust Optimization by Outer Approximations Stefan Wild Mathematics and Computer Science Division Argonne National Laboratory Joint work with Goldfarb grandson Matt Menickelly (Argonne) + Sven Leyffer, Todd Munson, Charlie

531 views • 42 slides
Securities Board of India Guest Lecture Convergence of Derivative and Cash Markets Andrew Sheng

Securities Board of India Guest Lecture Convergence of Derivative and Cash Markets Andrew Sheng

Securities Board of India Guest Lecture Convergence of Derivative and Cash Markets Andrew Sheng Chairman Securities and Futures Commission Hong Kong 26 May 2005 Table of Contents 1. Global Trend in Derivative Markets 2. Derivative Markets

462 views • 32 slides
A generalized MBO diffusion generated method for constrained harmonic maps Braxton Osting

A generalized MBO diffusion generated method for constrained harmonic maps Braxton Osting

A generalized MBO diffusion generated method for constrained harmonic maps Braxton Osting University of Utah February 10, 2018 Inverse Problems and Machine Learning Based on joint work with Dong Wang and Ryan Viertel 1/ 28 Motion by mean

579 views • 28 slides
MATHEMATICS 1 CONTENTS More than two variables More than one constraint Lagrange method The

MATHEMATICS 1 CONTENTS More than two variables More than one constraint Lagrange method The

Multiple constrained optimization BUSINESS MATHEMATICS 1 CONTENTS More than two variables More than one constraint Lagrange method The Lagrange multiplier Old exam question Further study 2 MORE THAN TWO VARIABLES Recall the constrained

398 views • 21 slides
MATH529 Fundamentals of Optimization Fundamentals of Constrained Optimization V: Linear

MATH529 Fundamentals of Optimization Fundamentals of Constrained Optimization V: Linear

MATH529 Fundamentals of Optimization Fundamentals of Constrained Optimization V: Linear Programming and the Simplex Method Marco A. Montes de Oca Mathematical Sciences, University of Delaware, USA 1 / 61 Methods for Constrained

2.43k views • 61 slides
Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods Jorge Nocedal

Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods Jorge Nocedal

Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods Jorge Nocedal Northwestern University Huatulco, Jan 2018 1 Collaborators Albert Berahas Richard Byrd Northwestern University University of Colorado 2 Discussion

226 views • 20 slides
u t = u t u t d u = u u + u u d t = u A Vectors Derivative is

u t = u t u t d u = u u + u u d t = u A Vectors Derivative is

Vector Derivative expressed in terms of its smoothly varying (time dependent) magnitude &amp; direction (unit vector) functions. u t = u t u t d u = u u + u u d t = u A Vectors Derivative is determined by its current

662 views • 11 slides

More recommend