Optimisation Zhiwei Cao, Renhao Huang, Junning Fan Background - PowerPoint PPT Presentation

Optimisation Zhiwei Cao, Renhao Huang, Junning Fan

❏ Background Knowledge Introduction ❏ What is this optimisation about? ❏ NP-complete Problem ❏ NP-complete problem in the real world ❏ MetaHeuristic Algorithm

What is this optimisation about? ❏ Optimisation of software and hardware ❏ Find the balance between power consumption and speed

NP-complete Problem ❏ NP problem stands for problems that run in Non-deterministic Polynomial time and can be verified in polynomial time. ❏ A problem p in NP is NP- complete if every other problem in NP can be transformed (or reduced) into p in polynomial time. ❏ Time Complexity ❏ Deterministic polynomial time: O(n a ) ❏ Non-Deterministic polynomial time: O(a n ) ❏ O(n!), O(nW)

Hamiltonian Path Problem ❏ A Hamiltonian path is a path that visits each vertex of the graph exactly once. ❏ Time complexity ❏ Find the solution: O(2 n n 2 ) ❏ Verification: O(1)

Processor: Hi3559A Hardware dual-core 1.8 CPU Memory: 1GB NP- complete problem in System: Software Embedded linux the real world Structural Recogisation in 100 ms Functionalities: Broader Crossing detection

MetaHeuristic Algorithm

Simulated Annealing Renhao Huang

Ex: Travel Salesman Problem(TSP) A salesman wants to: B 1. Start from a city A 2. Travel through cities {B,C,D,E} A C 3. Go back to city A Assume: Edges are bidirectional ● E D Each edge has a weight (distance) ● Must go through all cities ● Aim: Find the shortest path? However, these accurate algorithms cannot handle large scales of Possible solution: problems. 1. Brute force: Enumeration (O(n!)) 2. Dynamic Programming (O(n 2 2 n )) 3. Cutting-plane method Solution: Simulated Annealing Algorithm

● A heuristic algorithm imitating the process of annealing ● Invented by S. Kirkpatrick, D. Gelatt et. in 1983 Simulated ● To find an optimal solution in NP- Annealing completes cases ● Popular in applications such as VLSI, Deep Learning, Image processing, TSP

What is Annealing A Physical technology used in Metallurgy and material ● science. Cooling slowly to find the lowest energy state (Crystallization) ● Steps: ● Heating Maintain Cooling With the temperature decrease, With the temperature Under the constant the thermal motion of molecules increases, the motion of temperature, the free energy decreases and the energy level the molecules increases. goes to minimum and the system goes to minimum. becomes equilibrium

Start Initialize T, Final Tend and Cooling Rate 0< δ <1 Algorithm Generate a New Get Initial Solution A Solution B Find the cost Δ E = E(A) - E(B) T Find Possibility Δ E > 0 P( Δ E) ● Temperature T is just a value used to control the F probability P( Δ E) Accept New T > 0.5 Solution ● δ is similar to the learning rate in deep F learning(or deceleration of the temperature ） Reaches the Preserve Previous F no. of Solution ○ iterations If δ is too small: less accuracy ○ If δ is too large: time consuming T Cooling T = δ * T F T = Tend T END

Probability Function ∆" : The change of the internal energy : Boltzmann constant With temperature goes down, ● probability goes to zero The probability becomes low if there is ● a big change of internal energy The internal energy of a system also means the evaluation of the cost of the system. In most cases, we want to minimize the energy. Energy in TSP = the total distance

Travel Salesman Problem: SA Parameter: T = 50, Tend = 0.01, δ = 0.95 B Step 1: Initialize first path: Path A = A -> B -> C -> D -> E -> A Step 2: Generate a new path (by swapping) A C Path B = A -> E -> C -> D -> B -> A Step 3: Evaluate two paths: Δ distance =distance(Path A) - distance(Path B) E D Step 4.1: If Δ distance > 0, B win. Accept Path B Step 4.2: If Δ distance < 0, A win. Calculate P( Δ distance) to decide if Path B stays Step 5: After several loops from step 1 to step 4, update T ← T x δ and repeat.

Travel Salesman Problem: SA

Why Simulated Annealing is Good Local minima A tradeoff between global greedy and dynamic minima programming Simulated Annealing vs Vanilla Greedy Simulated Annealing vs Dynamic Programming ● Greedy Algorithm may be stuck at the local minimum ● DP will reach the global minima but slow ● Simulated Annealing has an opportunity to jump out the local minimum ● Simulated Annealing can find a solution that is close to the optimal

How to use Simulated Annealing Algorithms? Simulated Annealing algorithm can be used on most of optimization ● problems Find a solution with minimum cost or maximum benefit ○ But too heavy to use dynamic programming ○ It is an NP-complete problem ○ Perfect solution is not required ○ First define the energy function , e.g.: ● In TSP function: Total distance = ∑ distance between a pair of cities ○ In Knapsack problem: Total values of all items ○ List all constraints : ● Have to avoid solutions outside the limitations ○ Finally, follow the steps of simulated annealing algorithm ● Current solution ○ Another possible solution ○ Compare and decide which one to be selected ○

Ex: Problem Definition on FPGA Floorplanning using SA ● Description: ○ An NP-complete problem ○ FPGA consists of CLBs connected with each other by routing. ○ Long wires will lead to the delay. ○ Good floor planning will increase the efficiency of the FPGA ● Define: ○ Wire Length Cost IO between I/O connections ○ Internal Wire Length Cost IC ○ Extra cost Ex such as the power cost from the logic gates or the area demand. ○ Total Cost = sum of {Ex, IO, IC} ● Constraints: ○ Block placement cannot be overlapped, etc ● Object : ○ Find the floor plan with minimum total cost

Ex: Problem Definition on FPGA Floorplanning using SA Delay on wires x10 x20

Some Disadvantages of SA ● Selection of initial temperature is essential. High initial temperature -> high accuracy low efficiency ○ Low initial temperature -> low accuracy high efficiency ○ ● Need to set a suitable cooling rate δ High δ -> high accuracy low efficiency ○ Low δ -> low accuracy high efficiency ○ ● Need to control the number of iterations between each cooling Search under a same temperature is important ○ But too many iterations will be time consuming ○ ● Still has possibility to stuck at local minimum

Genetic Algorithm in HW/SW partitioning Edward Fan

Challenge: • For SoC like Zynq or more complicated embedded system, there’re various Hardware- computation resources to be selected for software tasks. partitioning – CPUs can execute software code – FPGAs can be reconfigured to accelerate various tasks – ASIC components (DSPs) can accelerate certain computations like multiplication • We have limitation on both cost, power and timing for executing a task. • Question: How do we plan and assign tasks to these resources to meet the limitations?

Difference • Experiment shows: between – Software implementations are the slowest, but consumes least power. resources – FPGAs are fast, and are reconfigurable to execute other tasks, but they are power hungry. – ASIC are the fastest, but they lack flexibility to be executed other tasks while being power hungry. • How to plan these resources to meet multiple objects of power, timing and cost? – Use of genetic Algorithm is purposed to achieve these multiple objectives.

Genetic • A stochastic search based algorithm. (expect some randomness) Algorithm • It performed well on solving combinatorial problem (like assignment of a discrete, finite set of objects that satisfies given conditions) • Good at avoiding local optimal in the solution space • A genius algorithm inspired by the genetic inheritance, exchange and mutation of organisms.

Elements of Genetic Algorithm • Chromosome – Chromosome of genetic algorithm H F S H represent a certain solution to the A chromosomic representation of problem. (e.g. which task is executed by scheduling first task to hardware, second to hardware, third to FPGA, software/hardware/FPGA) forth to software • Fitness – Fitness is a function that evaluates how a specific chromosome meets the F( ) = 2.2 H H F S requirements of the problem.

Operations of Genetic Algorithm • Selection • Crossover • Mutation • Reproduction

H H F S Selection F F S H Fittest 2 pairs • For a certain generation, select the fittest H H H S N pairs of parents. These parents will have the change to pass their gene to the next H S F S generation. H F S S S F S S S S S S F F F F

Crossover • When pair of parents is selected for creating offspring, the parent may exchange part of chromosome. – A crossover point is randomly selected – Offspring have chromosome from both parents

Mutation • When a offspring is formed, their gene can be subjected to random mutation, result in minor alternation of their gene. – Mutation introduces diversity and randomness to the population