Q-learning based Congestion-aware Routing Algorithm for On-Chip - PowerPoint PPT Presentation

Q-learning based Congestion-aware Routing Algorithm for On-Chip Network Fahime Farahnakian, Masoumeh Ebrahimi, Masoud Daneshtalab, Pasi Liljeberg, and Juha Plosila Department of Information Technology, University of Turku, Finland Email: fahfar@utu.fi

Outline q Motivation q Related work q Q-learning based Congestion-aware Algorithm (QCA) q Implemention q Experimental Result

Motivation A congestion-aware adaptive routing algorithm based on the Q-learning for network-on-chip(NoC).

Related work-(1/2) Q-learning output of Q-learning ---------> Q-table q Q-table ü Each time, this algorithm starts with an initial state and reaches a goal state by executing a sequence of actions and receiving rewards. Action Space ¡ Q-values ¡ Current-State Next-State Space ¡ Space ¡ ¡ ¡ ¡ ¡

Related work- (2/2 ) Q-routing q y= waiting time in the packet queue of node y δ = transmission delay over the link from node x to y Q y (z ; d) = the time it would take for node y to send this packet to its destination via any of node y 's neighbors (z )

Q-learning based Congestion-aware Algorithm ü NoC-based adjusted Q-table ü Learning Packet Format ü QCA algorithm

NoC-based adjusted Q-table Current Next Router Output port latency Destination router Node 4 1 3 South West Node 0 Node 4 1 - South - - Node 1 Node 4 1 Node 2 5 South East Node 4 3 Node 3 - West - - Node 4 4 Node 4 - Local - - Node 4 5 Node 5 - East - - Node 4 3 Node 6 7 West North Node 4 7 Node 7 - North Node 4 5 7 East North Node 8

Learning Packet 4 ¡bits 8 ¡bits 4 ¡bits 2 ¡bits Receiving ¡ Local Global ¡ Destiantion ¡ Node ¡ID ¡Latency Latency Node ¡ID q y Q y (z,d) d X

QCA algorithm

Experimental Result Uniform Random Traffic(Throughput) Average packet latency Throughput

Experimental Result Hotspot Traffic Average packet latency Throughput

Hardware Cost Components Area(mm 2 ) router in QCA method 0.1683 router in DyXY method 0.1503

Thank you!