Optimized Q-learning Model for Distributing Traffic in On-Chip - PowerPoint PPT Presentation

Optimized Q-learning Model for Distributing Traffic in On-Chip Networks Fahimeh Farahnakian, Masoumeh Ebrahimi ,Masoud Daneshtalab, Pasi Liljeberg, and Juha Plosila University of Turku, Finland

Outline q Intruduction • 2D mesh NoC • Routing algorithm q Background • Q-routing • C-routing q Clustered Q-routing • Routing table • Packets format • Routing algorithm q Results 2

A Mesh Network on-Chip (NoC) Topology q Each core connect to a router by a local network interface q Each router connect to its neighboring routers through bidirectional links q Cores communicate with each other using packets 3

Routing Algorithm Routing policy Deterministic Adaptive • In deterministic routing algorithms a transfer path is completely determined by the source and destination addresses, like XY . • In adaptive routing algorithms each packet’s transfer path determines based on the current network conditions, for example DyXY. 4

Motivation(1/3) An intelligent adaptive routing algorithm which is able to find minimum latency path from a source to a destination using Q-routing. 5

Q-routing(1/2) • Q-routing is an adaptive routing method based on the Q-learning model in a communication network. • Each router stores a routing table (Q-table) to maintain information about the routing cost (Q-value) from itself to the possible destination nodes. 6

Q-routing(2/2) y z x d s Q x (y,d) i j q y = waiting time in the packet queue of node y δ =transmission delay over the link from node x to y Q y (z ; d) = the time it would take for node y to send this packet to its Destination via any of node y 's neighbors (z ) ( ) ( ) ( ) ( ) ( ) Q y , d Q y , d Q z , d q Q y , d = + γ + + δ − x x y y x new old old 7

C-routing The C-routing algorithm is a combination of a deterministic routing algorithm (XY) and a Q-routing. Depending on the location of source and destination switches, one of the routing algorithms is invoked.

Motivation(2/3) • Each router maintains a Q-table with n × m entries in n × n 2D mesh. The area occupied by the Q-tables: n : Number of routers in the network m :Number of neighboring routers 9

Motivation(3/3) A clustering approach in order to • Reduce the area overhead • Improve the network performance Clustered Q-routing (CQ-routing) 10

CQ-table • A network into C clusters • CQ-table is maintained for each cluster instead of each switch. • The area occupied by the Q-tables: D : Number of routers within each cluster. 11

Area Reduction AU AU No. of No. of C-routing Q-routing Mesh Size Clusters Tables (%) (%) 75% 8 × 8 16 16 94% 83% 16 × 16 32 32 98% 93% 32 × 32 64 64 99% 93% 64 × 64 128 128 99% 12

CQ-routing Algorithm (1/2) Learning packet Data packet 2 ¡bits 4 ¡bits 4 ¡bits 2 bits 4 bits New ¡ Upstream Destiantion ¡ Receiving ¡ Estimated ... QTime Cluster ID Cluter ¡ID Cluster ¡ID ¡Latency 13

CQ-routing Algorithm (2/2) C12 C13 C14 Cd C12 C13 C14 Cd C12 C13 C14 Cd C8 C9 C10 C11 C8 C9 C10 C11 C8 C9 C10 C11 C4 C4 C4 C5 C6 C7 C5 C6 C7 C5 C6 C7 8 9 10 11 10 11 10 11 C2 C3 C1 C3 C1 C3 Cs C1 Cs Cs C2 C2 0 1 2 3 2 3 2 3 East West North South East West North South East West North South 1 . . . (a) 10 -1 4 -1 6 -1 8 -1 15 5 -1 8 -1 (b) (a) (c) ( ) ( ) Q 1 , 5 5 0 . 5 7 5 Destination Receiving New = + − Learning Packet new Cluster_ID Cluster_ID EstimatedLatency Data Packet Learning Packet from C1 to C0 0 15 7 14

Results Random Transpose Hotspot Performance different traffic models in 8 × 8 2D-mesh 15

Results Random Transpose Hotspot Performance different traffic models in 14 × 14 2D-mesh 16

Thank you!