Scalable Interconnection Networks Chapter 10 1 Adaptado dos slides - PDF document

Scalable Interconnection Networks Chapter 10 1 Adaptado dos slides da editora por M Mario Côrtes – IC/Unicamp – 2009s2

10.1 Scalable, High Performance Network Mario Côrtes – IC/Unicamp – 2009s2 At Core of Parallel Computer Architecture Requirements and trade-offs at many levels • Elegant mathematical structure • Deep relationships to algorithm structure • Managing many traffic flows • Electrical / Optical link properties • Electrical / Optical link properties Adaptado dos slides da editora por M Little consensus Scalable Interconnection Network • interactions across levels • Performance metrics? • Cost metrics? network • Workload? interface CA CA => need holistic understanding P P M M (p. 749) 2

Requirements from Above Mario Côrtes – IC/Unicamp – 2009s2 Communication-to-computation ratio => bandwidth that must be sustained for given computational rate • traffic localized or dispersed? • bursty or uniform? Programming Model • protocol • protocol Adaptado dos slides da editora por M • granularity of transfer • degree of overlap (slackness) => job of a parallel machine network is to transfer information from source node to dest. node in support of network transactions that realize the programming model (p. 750) 3

Goals Mario Côrtes – IC/Unicamp – 2009s2 Latency as small as possible As many concurrent transfers as possible • operation bandwidth • data bandwidth Cost as low as possible Adaptado dos slides da editora por M (p. 751) 4

Outline Mario Côrtes – IC/Unicamp – 2009s2 Introduction 10.1-2 Basic concepts, definitions, performance perspective 10.3 Organizational structure 10.4 Topologies 10.6 Routing and switch design Adaptado dos slides da editora por M 5

Basic Definitions Mario Côrtes – IC/Unicamp – 2009s2 Network interface Links • bundle of wires or fibers that carries a signal Switches • connects fixed number of input channels to fixed number of output channels channels Adaptado dos slides da editora por M (p. 751-752) 6

Links and Channels Mario Côrtes – IC/Unicamp – 2009s2 ...ABC123 => ...QR67 => Receiver Transmitter transmitter converts stream of digital symbols into signal that is driven down the link down the link Adaptado dos slides da editora por M receiver converts it back • tran/rcv share physical protocol trans + link + rcv form Channel for digital info flow between switches link-level protocol segments stream of symbols into larger units: packets or messages (framing) node-level protocol embeds commands for dest communication assist within packet (p. 751-752) 7

Formalism Mario Côrtes – IC/Unicamp – 2009s2 network is a graph V = {switches and nodes} connected by communication channels C ⊆ V × V Channel has width w and signaling rate f = 1/τ • channel bandwidth b = wf • phit (physical unit) data transferred per cycle • flit - basic unit of flow-control Adaptado dos slides da editora por M Number of input (output) channels is switch degree Sequence of switches and links followed by a message is a route Think streets and intersections (p. 752) 8

What characterizes a network? Topology (what) Mario Côrtes – IC/Unicamp – 2009s2 • physical interconnection structure of the network graph • direct: node connected to every switch • indirect: nodes connected to specific subset of switches Routing Algorithm (which routes) • restricts the set of paths that msgs may follow • many algorithms with different properties Adaptado dos slides da editora por M – gridlock avoidance? Switching Strategy (how) • how data in a msg traverses a route • circuit switching vs. packet switching Flow Control Mechanism (when) • when a msg or portions of it traverse a route • what happens when traffic is encountered? • (ver definição de flit e exemplo p. 753) (p. 752-753) 9

Interplay of all of these aspects of the design What determines performance 10 Adaptado dos slides da editora por M Mario Côrtes – IC/Unicamp – 2009s2

Topological Properties Mario Côrtes – IC/Unicamp – 2009s2 Routing Distance - number of links on route Shortest Path – menor routing distance entre dois nós Diameter - maximum shortest path entre quaisquer dois nós Average Distance – média das routing distances de todos pares de nós A network is partitioned by a set of links if their removal A network is partitioned by a set of links if their removal Adaptado dos slides da editora por M disconnects the graph (p. 753-754) 11

Typical Packet Format Mario Côrtes – IC/Unicamp – 2009s2 Header Control and Routing Code Error Trailer Payload Data digital symbol Sequence of symbols transmitted over a channel Adaptado dos slides da editora por M (Packet switching é mais usado que circuit switching) Componentes: Header (roteamento e controle), Trailer (ECC), Payload Two basic mechanisms for abstraction • Encapsulation: carrega informação de protocolo de alto nível dentro do pacote • Fragmentation: fragmenta as informações de protocolo de alto nível em uma sequência de mensagens (p. 754) 12

10.2 Communication Perf: Latency Mario Côrtes – IC/Unicamp – 2009s2 • Time(n) s-d = overhead + routing delay + channel occupancy + contention delay • Occupancy = (n + n e ) / b • n= tamanho payload; n e = tamanho do envelope • b= bandwidth • visto nos capítulos passados; mas agora, visão do ponto de Adaptado dos slides da editora por M vista da rede • Routing delay? • depende do Nº de links (routing distance) e do delay de cada switch ( ∆ ∆ ) ∆ ∆ • Contention? (p. 756) 13

Store&Forward vs Cut-Through Routing Mario Côrtes – IC/Unicamp – 2009s2 C ut-Through R outing Store & Forward R outing S ource Dest Dest 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 2 1 0 3 3 2 2 1 0 1 0 3 3 2 2 1 0 1 0 Adaptado dos slides da editora por M 3 2 1 0 3 2 1 0 3 2 1 0 Tim e h(n/b + ∆ ) n/b + h ∆ vs what if message is fragmented? (ver texto; semelh. ct) ver texto: packet switching e circuit switching wormhole vs virtual cut-through h = routing dist; ∆ =delay/hop (p. 756-757) 14

Contention Mario Côrtes – IC/Unicamp – 2009s2 Two packets trying to use the same link at same time Adaptado dos slides da editora por M • limited buffering • drop? Most parallel mach. networks block in place • link-level flow control • (back pressure) • tree saturation Closed system - offered load depends on delivered (p. 759) 15

10.2.2 Bandwidth Mario Côrtes – IC/Unicamp – 2009s2 What affects local bandwidth? • packet density b . n/( n + n e ) b . n / ( n + n e + w ∆ ∆ ) ∆ ∆ • routing delay (derated) • contention w ∆ = oportunidade – endpoints perdida devido a link – within the network bloqueado Aggregate bandwidth Aggregate bandwidth Adaptado dos slides da editora por M • bisection bandwidth – sum of bandwidth of smallest set of links that partition the network (duas metades iguais) • total bandwidth of all the channels: Cb (Nº canais * b / canal) • suppose N hosts issue packet every M cycles with ave dist h – each msg occupies h channels for � = n/w cycles each – C/N channels available per node – link utilization ρ = MC/N h �� < 1 (na realidade << 1) (p. 761-762) 16

Saturation 0,8 80 Mario Côrtes – IC/Unicamp – 2009s2 0,7 Delivered Bandwidth 70 0,6 Saturation 60 0,5 Latency 50 0,4 40 Saturation 0,3 30 0,2 20 0,1 10 0 Adaptado dos slides da editora por M 0 0,5 1 1,5 0 Offered Bandwidth 0 0,2 0,4 0,6 0,8 1 Delivered Bandwidth • Delivered bandwidth: valor real “entregue” pela rede • saturação: latência cresce exponencialmente • Offered bandwidth: demanda colocada pelas aplicações • ok p/ baixas demandas; atinge a saturação (p. 762-763) 17

Outline Mario Côrtes – IC/Unicamp – 2009s2 Introduction 10.1-2 Basic concepts, definitions, performance perspective 10.3 Organizational structure 10.4 Topologies 10.6 Routing and switch design Adaptado dos slides da editora por M 18

10.3 Organizational Structure Mario Côrtes – IC/Unicamp – 2009s2 Processors • datapath + control logic + memory interface • datapath: ALU, register file, pipeline latches ….. • control logic determined by examining register transfers in the datapath • conexões: via de dados (curtas e rápidas); controle (longas e lentas); escalamento ?? escalamento ?? Adaptado dos slides da editora por M Networks (composta dos componentes) • links • switches • network interfaces (p. 764) 19

10.3.1 Link Design/Engineering Space Mario Côrtes – IC/Unicamp – 2009s2 • Cable of one or more wires/fibers with connectors at the ends attached to switches or interfaces • características importantes: length, width, clocking Synchronous: Narrow: - source & dest on same - control, data and timing clock multiplexed on wire Adaptado dos slides da editora por M Short: Long: - single logical - stream of logical value at a time values at a time Asynchronous: Wide: - source encodes clock in - control, data and timing signal on separate wires (p. 764) 20