Emergent Optimization: Design and Applications in Telecommunications - PowerPoint PPT Presentation

Emergent Optimization: Design and Applications in Telecommunications and Bioinformatics PhD Thesis Dissertation Author: José Manuel García-Nieto Advisor: Dr. Enrique Alba PhD Thesis Dissertation – José Manuel García-Nieto 1 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Objectives | Organization Objectives Part I Work hypothesis: I.H: Particle Swarm Optimization is a first class base-line optimizer able of the best performance in modern benchmarking, as well as in present real-world optimization problems An ideal approach should have: Design and analysis of new PSO proposals and their validation on standard benchmarks Application to real world problems in different areas of engineering PhD Thesis Dissertation – José Manuel García-Nieto 2 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Objectives | Organization Organization Part I • DEPSO, RPSO-vm, SMPSO Design of • PSO6, PSO6-Mtsls , PMSO Algorithms Part I • Introduction • Fundamentals • Scalability, Speedup, Analysis of • Evolvability , Performance Properties • Design of algorithms Part II • Analysis of properties • CEC’05, BBOB’09, DTLZ, Benchmarks • Benchmarking & validation • SOCO’10 , Statistical Tests & Validation • Application to real world Part III • Algorithmic analysis • Gene selection in Cancer Genomic & • Problem domain solution Bioinformatic DNA Microarrays analysis • Communication Protocol Telecommu- nications Optimization in VANETs Part IV • Conclusions • Future Work • Signal Light Timing Traffic Management Programs PhD Thesis Dissertation – José Manuel García-Nieto 3 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Part I Part I Introduction Fundamentals PhD Thesis Dissertation – José Manuel García-Nieto 4 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Optimization Problem Part I An optimization problem can be defined as a pair: P = (S,f) where: S is the set of possible solutions (a.k.a. solution space) f: S → R is an objective function we wish to maximize or minimize • Global Maximum • Local Maximum In the case of minimization the • Local Minimum objective is to find s’  S | f(s’) ≤ f(s),  s  S • Global Minimum PhD Thesis Dissertation – José Manuel García-Nieto 5 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Definition and Classification Part I Optimization algorithms A metaheuristic is a top-level structured strategy that guides underlying heuristics to solve a given problem Swarm Intelligence Nature inspired techniques based on swarm dynamics and search strategies PhD Thesis Dissertation – José Manuel García-Nieto 6 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Particle Swarm Optimization (PSO) Part I Features Designed in 1995 by Kennedy and Eberhart Inspired on the Nature: Swarm of birds and fish schooling, modeling movements and reactions Solutions are encoded as particles that are moved using a velocity equation local the velocity depends on the position neighbor of other particles Popular metaheuristic nowadays global Fast convergence Easy to understand and implement PhD Thesis Dissertation – José Manuel García-Nieto 7 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Particle Swarm Optimization (PSO) Part I Learning procedure Linear combination of vectors with random components Movement dynamics 𝑢+1 = 𝑦 𝑢 + 𝑤 𝑢+1 𝑦 𝑗 𝑗 𝑗 𝑢+1 = 𝜕 ∙ 𝑤 𝑗 𝑢 + 𝑉 𝑢 [0, 𝜍] ∙ (𝑞 𝑢 − 𝑦 𝑢 − 𝑦 𝑢 ) + 𝑉 𝑢 [0, 𝜍] ∙ (𝑐 𝑗 𝑢 ) 𝑤 𝑗 𝑗 𝑗 𝑗 Social factor Individual factor PhD Thesis Dissertation – José Manuel García-Nieto 8 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Particle Swarm Optimization (PSO) Part I Major achievements in the State Of the Art (S.O.A.) Standards Prominent versions in S.O.A. Canonical PSO (Kennedy & Eberhart) Other PhD Thesis in S.O.A. Binary (Kennedy) Constriction factor (Clerc) Proposed in this PhD Thesis Discrete FIPS (Mendes et al.) Standard 2011 SMPSO Bare Bones (Yoshida) PhD Thesis Standard 2006 (Kennedy) OLPSO (Mendes) Standard 2007 RPSO-vm (Zhang et al.) MOPSO Geometric MOPSO CLPSO (Parsopoulos (Moraglio ) IPSO (Moore & (Liang et al.) & Vrahatis) UPSO (Montes de Oca et al.) Chapman) (Parsopoulos CCPSO & Vrahatis) (Li & Yao) SLPSO (Li et al.) DMS-PSO Binary PSO PhD Thesis (Zhao et al.) Toolbox PMSO PSO6 (Van Den Bergh) (Clerc) DEPSO PSO6-Mtsls 1995 1998 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 PhD Thesis Dissertation – José Manuel García-Nieto 9 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Particle Swarm Optimization (PSO) Part I From the literature we can conclude that: Benchmarking (7 standard benchmarks) CEC’ 05, CEC’ 08, SOCO’ 10 BBOB’ 09 ZDT, DTLZ, WFG Real world applications (more than 28 domains) 1 Communications … 15 Bioinformatics (Data Mining) 18 Traffic management 19 Vehicular networks … 28 Chemical processes Source: work done in this thesis based in more than 3000 papers, using IEEExplore, DBLP ACM Digital Library, MIT Press… PhD Thesis Dissertation – José Manuel García-Nieto 10 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Experimental Procedure for our Studies Part I Benchmarking Given a problem to solve 1º Select algorithms 2º Execute multiple 3º Perform to compare with independent runs statistical analyses Given an algorithm to evaluate Real world applications PhD Thesis Dissertation – José Manuel García-Nieto 11 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Reference Algorithms Part I Benchmarking G-CMA-ES, DE (rand/1), CHC,K-PCX FIPS, FIPS-Square IPSO, IACO, MOS-DE, GaDE CLPSO OMOPSO, NSGA-II Real world applications PSO Standards 2006, 2007, and 2011 DE (rand/1) GA, SA, ES Random Search Deterministic SGCP PhD Thesis Dissertation – José Manuel García-Nieto 12 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work Optimization | Metaheuristics | PSO | Methodology Statistical Validation Part I Null-hypothesis: equality of distributions with a confidence level of 95% (Statistical differences can be found if tests result with p-value<0.05) [GMLH09] S. García, D. Molina, M. Lozano, and F. Herrera, A study on the use of nonparametric tests for analyzing the evolutionary algorithms’ behavior: a case study on the CEC’2005 , Journal of Heuristics 15 (2009), no. 6, 617 – 644. PhD Thesis Dissertation – José Manuel García-Nieto 13 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work DEPSO | RPSO-vm | SMPSO Part II Part II Design of Analysis of Algorithms Properties PhD Thesis Dissertation – José Manuel García-Nieto 14 / 53 February 22, 2013

Algorithm Proposals Real World Conclusions & Introduction Fundamentals and Validation Applications Future Work DEPSO | RPSO-vm | SMPSO Design Issues Part II In PSO, modify the learning procedure to induce an improved performance usually means a reformulation to have a new velocity vector equation We have opted for several mechanisms: Mechanism Description Proposals Hybridization DEPSO Using differential evolution operators Velocity modulation RPSO-vm Constraining velocity to the search range Using velocity modulation and leader Multi-objective SMPSO selection from non-dominated set Neighborhood topology Discovering a quasi-optimal information PSO 𝟕 ± 𝟑 & number of informants scheme Interdependency of Hybridizing with local search & range PSO6-Mtsls variables decisions Parallel swarm PMSO Structuring swarms in parallel PhD Thesis Dissertation – José Manuel García-Nieto 15 / 53 February 22, 2013