Automating la large -scale simulation and data analysis with - PowerPoint PPT Presentation

Automating la large -scale simulation and data analysis with OMNeT++ ++ Antonio Virdis (Carlo Vallati, Giovanni Nardini) University of Pisa – Italy OMNeT++ Summit 2016 OMNeT++ Summit 2016 Antonio Virdis 1

OUTLINE • Simulation Phases • Factors vs Parameters • Five main topics OMNeT++ Summit 2016 2 Antonio Virdis

Panelists Red Corner Blue Corner • Laura Marie Feeney • Andras Varga • Rudolf Hornig (Uppsala University, Sweden) • Kyeong Soo (Joseph) Kim (OMNeT++ Team) (Xi'an Jiaotong-Liverpool University, Suzhou, China) OMNeT++ Summit 2016 3 Antonio Virdis

Simulation Phases We are here Documentation idea Validation/ Modeling Development Simulation verification • Modeling , development and validation/verification are completed. • We have a pretty good idea on what to test. • We have a pretty good idea on what to measure. OMNeT++ Summit 2016 4 Antonio Virdis

Simulation Phases Result Result Scenario Execution Management Analysis Generation • What to simulate . • How to perform simulation (single PC? Multi PC? How many in parallel?) • How we write results? How we read them? • Statistical analysis and result presentation . OMNeT++ Summit 2016 5 Antonio Virdis

Factors vs Parameters • Non varying parameters **.packets_second = 50 **.mobility_type = “linear” parameters • Varying parameters **.size= ${ 50 , 100 } **.speed = ${ 1 , 2 } ${iteration vars} factors OMNeT++ Summit 2016 6 Antonio Virdis

Factors and Simulations repeat = 3 ID size speed repetition 0 50 1 0 1 50 1 1 ID size speed 2 100 1 0 0 50 1 1 100 1 3 100 1 1 2 50 2 4 50 2 0 3 100 2 5 50 2 1 6 100 2 0 7 100 2 1 OMNeT++ Summit 2016 7 Antonio Virdis

Architecture Factors Scenario Generator Result Parser values Analyzer Launcher Parameters files OMNeT++ Summit 2016 8 Antonio Virdis

Topic 0: Large Scale • When does a simulation become “ large ”? Size of the single simulation run • Lots of modules • Lots of metrics • Lots of factors Number of simulation runs OMNeT++ Summit 2016 9 Antonio Virdis

Result Result Scenario Execution Management Generation Analysis Topic 1: Scenario Generation • Are factors that important? • Naming: ID based vs Factor Based ID size speed repetition 0 50 1 0 1 50 1 1 2 100 1 0 **.size= ${ 50 , 75 , 100 } 3 100 1 1 4 50 2 0 5 50 2 1 6 100 2 0 7 100 2 1 OMNeT++ Summit 2016 10 Antonio Virdis

Result Result Scenario Execution Management Generation Analysis Topic 2: Simulation execution Single • Available: opp_run and opp_runall configuration • How to deal with a large number of runs (possibly on multiple cores)? • Is AKAROA your favorite son (still)? • Need for dynamic stop criterion ? (e.g. statistical confidence reached) OMNeT++ Summit 2016 11 Antonio Virdis

Result Result Scenario Execution Management Generation Analysis Topic 3: Writing/Reading Results • Available: scavetool + GUI interface (parsing) • Work on files using regular expressions . • Results are fully loaded into memory . • Alternatives? • Implementing a new writers ? • Connecting results to factors? output-scalar-file = ${configname}-${runnumber}-${iterationvars}-${repetition}.sca OMNeT++ Summit 2016 12 Antonio Virdis

Result Result Scenario Execution Management Generation Analysis Topic 4: Result Analysis • Built-in in OMNeT via GUI • Connection with R, Octave, Matlab … • Data representation: gnuplot interface anyone? OMNeT++ Summit 2016 13 Antonio Virdis

Topic 5: Unified Framework Factors Scenario Generator Result Parser values Analyzer Launcher Parameters files OMNeT++ Summit 2016 14 Antonio Virdis