HEURISTIC OPTIMIZATION Various Topics Outline 1. Dynamic (time-varying) Optimization Problems 2. Stochastic Optimization Problems 3. Continuous (real-parameter) Optimization Problems 4. SLS Algorithms Engineering Heuristic Optimization 2016 2
Dynamic (time-varying) optimization problems I in many problems data, objectives, or constraints change over time I as a result, a candidate solution to a problem may (need to) adapt while implementing it I in dynamic optimization problems, a dynamic (i.e. time-varying) problem is solved online I large variety of di ff erent problem characteristics depending on how and when changes are considered Heuristic Optimization 2016 3 DOPs: classification I time-linkage: does future behavior of the problem depend on current solution? I predictability: are changes predictable? I detectability: are changes visible or detectable? I recurrency: are changes cyclic / recurrent? I changes: which are the problem data / information that changes? (objectives? number, domain, type of decision variables? constraints? instance data?) Heuristic Optimization 2016 4
DOP example: dynamic travelling salesman problem (DTSP) I various DTSP formulations are possible I time-varying travel costs I edge weights may change e.g. mimicking tra ffi c jams etc. I time-varying customers (nodes) I occasionally some nodes disappear / appear and, thus, modified instances are obtained I instances parameterized by frequency and amount of changes Heuristic Optimization 2016 5 Tackling DOPs I detecting changes? I restarting algorithms after changes? I easy, straightforward choice I may be e ff ective if change is very strong I however, it may (i) waste computation resources, (ii) may lead to very di ff erent solutions after change (even if change is small) I other approaches to adapt algorithms to specificities of the dynamic problems I uses of memory to store useful information / promising solution components I adaptation of parameters or neighborhoods I increasing diversity by ewn solutions (e.g. random immigrants) I prediction of changes and pro-active actions Heuristic Optimization 2016 6
Tackling DOPs .. cont’d I periodic reoptimization I periodically, a static problem instance is solved either when available data changes or at fixed intervals of time I can rely on known e ff ective algorithms for static problems I but requires optimization before updating solutions I continuous reoptimization I perform optimization throughout the day I maximizes computational capacity I however, solutions may change at any time Heuristic Optimization 2016 7 Performance evaluation for DOPs I two main aspects I convergence speed I quality of obtained solutions I a large number of performance measures w.r.t. measuring quality and convergence speed related behavior have been proposed I unification possible e.g. by using hypervolume of dominated time/cost tradeo ff Heuristic Optimization 2016 8
Stochastic optimization problems I stochastic optimization concerns the study and solution of optimization problems that involve uncertainty I part of the information about problem data is partially unknown I knowledge about the probability distribiton of unknown is assumed Heuristic Optimization 2016 9 Modeling approaches to uncertainty I how is uncertainty modeled? I prefect knowledge of data 7! classical deterministic optimization I by means of random variables with known distributions I fuzzy sets / quantities I interval values without known distribution I no knowledge 7! online optimization Heuristic Optimization 2016 10
Modeling approaches to uncertainty I dynamicity of the model? I i.e. time when uncertain information is revealed w.r.t. time when decision needs to be taken I distinguish time before actual realization of random variables and time after random variables are revealed I a priori optimization versus decision in stages I two-stage optimization problems: first stage decision is done ( a priori solution) and corrective actions can be made once random realizations are known I also known as simple recourse model Heuristic Optimization 2016 11 Domain of stochastic (combinatorial) optimization Heuristic Optimization 2016 12
Formalization of stochastic combinatorial optimization problem I problems that can be described as ⇥ ⇤ Min F ( x ) = E f ( x , ω ) , subject to x 2 S , I x is a solution I S is the set of feasible solutions I E is the mathematical expectation I f is the cost function I ω is a multi-variate random variable, hence f ( x , ω ) makes the cost function a random variable Heuristic Optimization 2016 13 Probabilistic TSP (PTSP) I complete graph G = ( V , A , C , P ) with I set of nodes V I set of edges A I C cost-matrix for travel costs between pairs of nodes I probability vector P that for each node i specifies its probability p i of requiring visit. I i.e. ω here is a n -variate Bernoulli distribution I realization: a binary vector of size n I 1: node requires visit I 0: node is to be skipped (no visit) I homogeneous PTSP: p i = p : 8 i 2 V I heterogenous PTSP: otherwise Heuristic Optimization 2016 14
Probabilistic TSP (2) a priori optimization I Stage 1: determine permutation of all nodes a priori solution I . . . realization of random variable becomes available . . . I Stage 2: determine actual tour by skipping nodes not to be visited a posteriori solution Heuristic Optimization 2016 15 Applying metaheuristics to SOPs I typically involves computation / approximation of expected value of the objective function I three main possibilities I closed-form expressions available to compute exact expected value I ad hoc and fast approximation if computation is too expensive I estimation of expected values by simulation excursion: e ffi cient local search for PTSP Heuristic Optimization 2016 16
Stochastic vs. dynamic problems I many problems domains where stochastic problems arise can also be modeled as dynamic problems I advantage of stochastic problems is that assumed distribution of data may be useful to generate realistic solutions that are more easily adapted to practical situations I in a sense, stochastic information is used to define “policies” I however, computation of objective function is more demanding and there is a necessity to assess probability distributions from data or (subjective) expertise Heuristic Optimization 2016 17 Continuous (real-parameter) optimization problems see excerpt of slides from Anne Auger of her course on derivative-free optimization Heuristic Optimization 2016 18
SLS algorithms I among most successful techniques for tackling hard problems I prominent in computing science, operations research and engineering I range from simple constructive and iterative improvement algorithms to general-purpose methods (“metaheuristics”) I widely studied, thousands of publications I sub-areas have become established fields (evolutionary algorithms, swarm intelligence) SLS algorithms are by now a well established tool for solving theoretically and practically relevant search problems Heuristic Optimization 2016 19 Heuristic Optimization 2016 20
Heuristic Optimization 2016 21 Heuristic Optimization 2016 22
Heuristic Optimization 2016 23 SLS algorithms Current deficiencies I few general guidelines of how to design e ffi cient SLS algorithms; application often considered an art I high development times and expert knowledge required I shortcomings in experimental methodology I relationship between problem / instance characteristics and performance not well understood I enormous gap between theory and practice Heuristic Optimization 2016 24
Which metaheuristic for which problem? Metaheuristics network I collaborative research project among four academic and one industry partner I initial structure of research I work on a common set of problems I each lab implements its favorite metaheuristic and one more I compare performance of SLS algorithms to allow insights into which metaheuristic strategies are the most successful for specific problems I ideal case: matching between problems / instance classes and success of metaheuristics Heuristic Optimization 2016 25 Which metaheuristic for which problem? Insights from Metaheuristics Network I success with SLS algorithms rather due to I level of expertise of developer and implementer I time invested in designing and tuning the SLS algorithm I creative use of insights into algorithm behaviour and interplay with problem characteristics I fundamental are issues like choice of underlying neighbourhoods, e ffi cient data structures, creative use of algorithm components; to a less extent strictly following the templates of specific SLS methods (“metaheuristics”) Heuristic Optimization 2016 26
Recommend
More recommend
