Population-Based Search 2-3-16 Reading Quiz Question 1: Which of - PowerPoint PPT Presentation

Population-Based Search 2-3-16

Reading Quiz Question 1: Which of the following attributes do stochastic beam search and genetic algorithms NOT share. a) a temperature parameter b) a population of states c) probabilistic survival of individuals d) exhaustive generation of neighbors

Simulated Annealing state = get_candidate() best_state = state temp = INIT_TEMP for round = 1:MAX_ITERS: neighbor = random_step(state) if cost(neighbor) < cost(best_state) best_state = neighbor if accept(state, neighbor, temp) state = neighbor temp *= DECAY function accept(state, neighbor, temp): return best_state delta = cost(state) - cost(neighbor) r ~ U[0,1] return r < e^(delta / temp)

function accept(state, neighbor, temp): delta = cost(state) - cost(neighbor) Exercise r ~ U[0,1] return r < e^(delta / temp) INIT_TEMP = 20; DECAY = .95; round = 4 Given the following state: ● cost(DC, Baltimore, Boston, Philly, NY) = 16.6 If the following neighbor is generated, what is the probability that it is accepted? ● cost(Boston, Baltimore, DC, Philly, NY) = 13.8 If the following neighbor is generated, what is the probability that it is accepted? ● cost(Philly, Baltimore, Boston, DC, NY) = 19.7

Choosing a temperature

Choosing a decay rate

Beam Search population = [POP_SIZE random states] temp = INIT_TEMP for i = 1:MAX_ITERS: candidates = [] for individual in population: add all neighbors of individual to candidates population = [POP_SIZE lowest-cost candidates] temp *= DECAY return best state encountered

Beam Search Illustrated

Stochastic Beam Search population = [POP_SIZE random states] temp = INIT_TEMP for i = 1:MAX_ITERS: candidates = [] for individual in population: add all neighbors of individual to candidates population = gibbs_samples(candidates, temp, POP_SIZE) temp *= DECAY return best state encountered

Stochastic Beam Search - helper function function gibbs_samples(candidates, temp, POP_SIZE): weights = [e^(-cost(n)/temp) for each candidate n] distribution = [weights[n] / sum(weights) for each candidate n] population = [POP_SIZE random draws from distribution] return population Note that the samples are drawn independently, which means that some states may be repeated in the new population.

Exercise: construct the distribution INIT_TEMP = 20; DECAY = .95; round = 4 Suppose we have the following candidates: cost(Boston, DC, Philly, NY) = 13.8 w = [e^(-cost(n)/temp) for cost(Philly, Boston, DC, NY) = 16.6 each candidate n] distr = [w[n] / sum(w) for cost(NY, Boston, Philly, DC) = 13.8 each candidate n] cost(DC, Philly, Boston, NY) = 13.8 cost(DC, NY, Philly, Boston) = 16.6 cost(DC, Boston, NY, Philly) = 13.8 What is the probability distribution from which the next population will be drawn?

Genetic Algorithms population = [POP_SIZE random states] for i = 1:MAX_ITERS: new_population = [] for j = 1:POP_SIZE: parent1, parent2 = select(population) add offspring(parent1, parent2) to new_population temp *= DECAY return best state encountered

Selection There are many ways this could be done, but a good one is Gibbs sampling. Call the function from before with POP_SIZE = 2. This means we need to add a temperature parameter. function gibbs_samples(candidates, temp, POP_SIZE): weights = [e^(-cost(n)/temp) for each candidate n] distribution = [weights[n] / sum(weights) for each candidate n] population = [POP_SIZE random draws from distribution] return population

Reproduction: crossover and mutation function offspring(parent1, parent2) cross_point = random location in state representation child = parent1[start:cross_point] + parent2[cross_point:end] for each variable in child: if (small mutation probability): give it a random new value return child

Q Reproduction in N-Queens Q Q parent1 = (1,2,4,2,2,5), parent2 = (2,6,4,6,5,0) Q Q cross_point = random integer in [0,5] # suppose it’s 4 Q offspring after crossover = (1,2,4,2,5,0) each element mutated with probability .02 # suppose 1 is mutated mutated elements get random new values # suppose it’s 0 offspring returned = (1,0,4,2,5,0)

Discussion: state representation for GAs What effect does crossover have on the state representation for these problems? Is there an alternative state representation that would work better?