interconnect technologies for clusters interconnect
play

Interconnect Technologies for Clusters Interconnect approaches - PowerPoint PPT Presentation

Dec 29, 2022 •376 likes •759 views

Cluster Computing Interconnect Technologies for Clusters Interconnect approaches Cluster Computing WAN infinite distance LAN Few kilometers SAN Few meters Backplane Not scalable Physical Cluster

  1. Cluster Computing Interconnect Technologies for Clusters

  2. Interconnect approaches Cluster Computing • WAN – ’infinite distance’ • LAN – Few kilometers • SAN – Few meters • Backplane – Not scalable

  3. Physical Cluster Interconnects Cluster Computing • FastEther • Gigabit EtherNet • 10 Gigabit EtherNet • ATM • cLan • Myrinet • Memory Channel • SCI • Atoll • ServerNet

  4. Switch technologies Cluster Computing • Switch design – Fully interconnected – Omega • Package handling – Store and forward – Cut-through routing (worm-hole routing)

  5. Implications of switch technologies Cluster Computing • Switch design – Affects the constant associated with routing • Package handling – Affects the overall routing latency in a major may

  6. Store-and-fwd vs. Worm- hole one step Cluster Computing • T ( ν ) = Overhead+Channel+Time Routing Delay • Cut through: • Store ’n fw:

  7. Store-and-fwd vs. Worm- hole ten steps Cluster Computing • T ( ν ) = Overhead+Channel+Time Routing Delay • Cut through: • Store ’n fw:

  8. FastEther Cluster Computing • 100 Mbit/sec + Generally supported + Extremely cheap - Limited bandwidth - Not really that standard - Not all implementations support zero-copy protocols

  9. Gigabit EtherNet Cluster Computing • Ethernet is hype-only at this stage • Bandwidth really is 1Gb/sec • Latency is only slightly improved – Down to 20us from 22us in 100Mb • Current standard – But NICs are as different as with FE

  10. 10 Gigabit EtherNet Cluster Computing • Target applications not really defined – But clusters are not the most likely customers – Perhaps as backbone for large clusters • Optical interconnects only – Copper currently being proposed

  11. ATM Cluster Computing • Used to be the holy grail in cluster computing • Turns out to be poorly suited for clusters – High price – Tiny packages – Designed for throughput not reliability

  12. cLAN Cluster Computing • Virtual Interface Architecture • API standard not HW standard • 1.2 Gbit/sec

  13. Myrinet Cluster Computing • Long time ’defacto-standard’ • LAN and SAN architectures • Switch-based • Extremely programmable

  14. Myrinet Cluster Computing • Very high bandwidth – 0.64Gb + 0.64 Gb in gen 1 (1994) – 1.28Gb + 1.28 Gb in gen 2 (1997) – 2.0Gb + 2.0 Gb in gen 3 (2000) – (10.0Gb + 10Gb in gen4 (2005)) ether • 18 bit parallel wires • Error-rate at 1bit per 24 hours • Very limited physical distance

  15. Myrinet Interface Cluster Computing • Hosts a fast RISC processor – 132 MHz in newest version • Large memory onboard – 2,4 or 8MB in newest version • Memory is used as both send and recieve buffers and run at CPU speed – 7.5ns in newest version

  16. Myrinet-switch Cluster Computing • Worm-hole routed – 5 ns route time • Process to process – 9us (133 MHz LANai) – 7us (200 MHz LANai)

  17. Myrinet Cluster Computing

  18. Myrinet Prices Cluster Computing • PCI/SAN interface – $495, $595, $795 • SAN Switch – 8 port $4,050 – 16 port $5,625 – 128 port $51,200 • 10 ft. cable $75

  19. Memory Channel Cluster Computing • Digital Equipment Corporation product • Raw performance: – Latency 2.9 us – Bandwidth 64 MB/s • MPI performance – Latency 7 us – Bandwidth 61 MB/s

  20. Memory Channel Cluster Computing

  21. Memory Channel Cluster Computing

  22. SCI Cluster Computing • Scalable Coherent Interface • IEEE standard • Not widely implemented • Coherency protocol is very complex – 29 stable states – An enourmous amount of transient states

  23. SCI Cluster Computing

  24. SCI Coherency Cluster Computing • States – Hom e: no remote cache in the system contains a copy of the block – Fres h: one or more remote caches may have a read-only copy, and the copy in memory is valid. – Gon e: another remote cache contains a writeable copy. There is no valid copy on the local node.

  25. SCI Coherency Cluster Computing • State is named by two components – ONLY – HEAD – TAIL – MID – Dirty: modified and writable – Clean: unmodified (same as memory) but writable – Fresh:data may be read, but not written until memory is informed – Copy: unmodified and readable

  26. SCI Coherency Cluster Computing • List constructio n: adding a new node (sharer) to the head of a list • Rollou t: removing a node from a sharing list, which requires that a node communicate with its upstream and downstream neighbors informing them of their new neighbors so they can update their pointers • Purging (invalidation ): the node at the head may purge or invalidate all other nodes, thus resulting in a single-element list. Only the head node can issue a purge.

  27. Atoll Cluster Computing • University research project • Should be very fast and very cheap • Keeps comming ’very soon now’ • I have stopped waiting

  28. Atoll Cluster Computing • Grid architecture • 250 MB/sec bidirectional links – 9 bit – 250MHz clock

  29. Atoll Cluster Computing

  30. Atoll Cluster Computing

  31. Atoll Cluster Computing

  32. Servernet-II Cluster Computing • Supports 64-bit, 66-MHz PCI • Bidirectional links – 1.25+1.25Gbit/sec • VIA compatible

  33. Servernet II Cluster Computing

  34. Servernet-II Cluster Computing

  35. Infiniband Cluster Computing • New standard • An extension of PCI-X – 1x = 2.5Gbps – 4x = 10Gbps – current standard – 12x = 30Gbps

  36. InfiniBand Price / Performance Cluster Computing Myrinet InfiniBand 10GigE GigE Myrinet D PCI-Express E 900MB/s 100MB/s 245MB/s 495MB/ Data Bandwidth 950MB/s s (Large Messages) MPI Latency 5us 50us 50us 6.5us 5.7us (Small Messages) HCA Cost $550 $2K-$5K Free $535 $880 (Street Price) $100- Switch Port $250 $2K-$6K $400 $400 $300 Cable Cost $100 $100 $25 $175 $175 (3m Street Price) • Myrinet pricing data from Myricom Web Site (Dec 2004) ** InfiniBand pricing data based on Topspin avg. sales price (Dec 2004) *** Myrinet, GigE, and IB performance data from June 2004 OSU study • Note: MPI Processor to Processor latency – switch latency is less

  37. InfiniBand Cabling Cluster Computing • CX4 Copper (15m) • Flexible 30-Gauge Copper (3m) • Fiber Optics up to 150m

  38. Cluster Computing The InfiniBand Driver Architecture APPLICATION INFINIBAND SAN NETWORK NFS-RDMA User BSD Sockets BSD Sockets FS API UDAPL Kernel TCP FILE SYSTEM SDP SDP DAT TS TS IP SCSI API SRP IPoIB Drivers FCP VERBS ETHER INFINIBAND HCA FC INFINIBAND SWITCH ETHER FC SWITCH ETH GW FC GW SWITCH E SAN LAN/WAN SERVER FABRIC

Recommend

Molded Interconnect Device Molded Interconnect Device (MID) Device (MID) A 3

Molded Interconnect Device Molded Interconnect Device (MID) Device (MID) A 3

MID Manufacturing: Advances in Metallization Plating Technologies Metallization Plating Technologies Leading to Improved Yields 9 th International Congress Molded Interconnect Devices MID 2010 September 30, 2010 Richard C. Retallick Senior

471 views • 25 slides
RF-Interconnect RF-Interconnect and its Applications to and its Applications to NoC Design NoC

RF-Interconnect RF-Interconnect and its Applications to and its Applications to NoC Design NoC

RF-Interconnect RF-Interconnect and its Applications to and its Applications to NoC Design NoC Design NOCS Tutorial Course, NOC Tutorial Course, May 10 2009 y 10 2009 San Diego, California San Diego, Californ Frank Chang, Jason Cong and

426 views • 41 slides
COOL Interconnect COOL Interconnect Low Power Interconnection Technology Low Power

COOL Interconnect COOL Interconnect Low Power Interconnection Technology Low Power

COOL Interconnect COOL Interconnect Low Power Interconnection Technology Low Power Interconnection Technology for Scalable 3D LSI Design for Scalable 3D LSI Design Marco Chacin, Hiroyuki Uchida, Michiya Hagimoto, Takashi Miyazaki, Takeshi

267 views • 15 slides
Robust Interconnect Robust Interconnect Communication Capacity Algorithm Communication Capacity

Robust Interconnect Robust Interconnect Communication Capacity Algorithm Communication Capacity

Robust Interconnect Robust Interconnect Communication Capacity Algorithm Communication Capacity Algorithm by Geometric Programming by Geometric Programming Jifeng Chen, Jin Sun and Janet Wang Department of Electrical and Computer Engineering

413 views • 25 slides
CS184c: Computer Architecture [Parallel and Multithreaded] Day 15: May 29, 2001 Interconnect

CS184c: Computer Architecture [Parallel and Multithreaded] Day 15: May 29, 2001 Interconnect

CS184c: Computer Architecture [Parallel and Multithreaded] Day 15: May 29, 2001 Interconnect CALTECH cs184c Spring2001 -- DeHon Previously CS184a: Day 11--14 interconnect needs and requirements basic topology

606 views • 24 slides
The Interconnect Verification Challenge Franois Cerisier and Mike Bartley Test and

The Interconnect Verification Challenge Franois Cerisier and Mike Bartley Test and

The Interconnect Verification Challenge Franois Cerisier and Mike Bartley Test and Verification Solutions IP-SOC 2012 Grenoble, 5 Dec 2012 The Interconnect Verification Challenge Whats an interconnect Interconnects

182 views • 16 slides
Connectors: HiLo Interconnect Wednesday, October 04, 2017 Company Overview March 12, 2015

Connectors: HiLo Interconnect Wednesday, October 04, 2017 Company Overview March 12, 2015

Next Level Integration Next Level Integration Connectors: HiLo Interconnect Wednesday, October 04, 2017 Company Overview March 12, 2015 HiLo INTERCONNECT OVERVIEW BGA/LGA Socketing Module Interconnect Highs Lows Flexibility High

254 views • 23 slides
Interconnects Outline Interconnect scaling issues Aluminum technology Copper

Interconnects Outline Interconnect scaling issues Aluminum technology Copper

Interconnects Outline Interconnect scaling issues Aluminum technology Copper technology araswat tanford University EE311 / Al Interconnects 1 Properties of Interconnect Materials Material Thin film Melting resistivity point

438 views • 15 slides
Outline Interconnect scaling issues Aluminum technology Copper technology araswat

Outline Interconnect scaling issues Aluminum technology Copper technology araswat

Outline Interconnect scaling issues Aluminum technology Copper technology araswat tanford University 1 EE311/ Cu Interconnect Wire Half Pitch vs Technology Node ITRS 2002 Narrow line effects Ref: J. Gambino, IEDM, 2003 araswat

417 views • 18 slides
Power/Performance Issues on Interconnection Network (IN) Is interconnect power problem?

Power/Performance Issues on Interconnection Network (IN) Is interconnect power problem?

NsimPower: Interconnect Simulator for Power and Performance Prediction Koji Inoue Kyushu University, Japan 1 Power/Performance Issues on Interconnection Network (IN) Is interconnect power problem? Roughly 10 to 30 % of

381 views • 10 slides
Geometrically Parameterized Interconnect Geometrically Parameterized Interconnect Performance

Geometrically Parameterized Interconnect Geometrically Parameterized Interconnect Performance

Geometrically Parameterized Interconnect Geometrically Parameterized Interconnect Performance Models for Interconnect Performance Models for Interconnect Synthesis Synthesis Luca Daniel, UC Berkeley Luca Daniel, UC Berkeley C. S. Ong, S. C.

417 views • 18 slides
An Interconnect-Centric Design Flow for Nanometer Technologies Jason Cong UCLA Computer Science

An Interconnect-Centric Design Flow for Nanometer Technologies Jason Cong UCLA Computer Science

An Interconnect-Centric Design Flow for Nanometer Technologies Jason Cong UCLA Computer Science Department Email: cong@cs.ucla.edu Tel: 310-206-2775 http://cadlab.cs.ucla.edu/~cong Outline Global interconnects in nanometer technologies

496 views • 20 slides
TRILL DC Interconnect

TRILL DC Interconnect

TRILL DC Interconnect dra0-balaji-l2vpn-trill-over-ip-mul;-level-02.txt Balaji Venkat Venkataswami Ramasubramani Mahadevan Shivakumar Sundaram Narayana

107 views • 8 slides
Interconnect Product Assembly AS YOUR PARTNER IN ESTONIA Since 1999 www.ipa.ee Company Profile

Interconnect Product Assembly AS YOUR PARTNER IN ESTONIA Since 1999 www.ipa.ee Company Profile

Interconnect Product Assembly AS YOUR PARTNER IN ESTONIA Since 1999 www.ipa.ee Company Profile Established in 1999 Interconnect Product Assembly is an independent privately owned company specialising in the contract manufacture of

472 views • 20 slides
Development of Optical Interconnect PCBs for High-Speed Electronic Systems Fabricators View

Development of Optical Interconnect PCBs for High-Speed Electronic Systems Fabricators View

Development of Optical Interconnect PCBs for High-Speed Electronic Systems Fabricators View 2011 IBM Printed Circuit Board Symposium Raleigh, NC, USA November 16 th 2011, Time: 10:00-10:30am Speaker: Marika Immonen TTM Technologies

482 views • 20 slides
Scaling the Cascades Interconnect-aware FPGA implementation of Machine Learning problems Anand

Scaling the Cascades Interconnect-aware FPGA implementation of Machine Learning problems Anand

BRAM DSP URAM Scaling the Cascades Interconnect-aware FPGA implementation of Machine Learning problems Anand Samajdar, Tushar Garg, Tushar Krishna, Nachiket Kapre nachiket@uwaterloo.ca Claim Hard FPGA interconnect (cascades) e ffi ciently

993 views • 82 slides
? VM VM VM VM VM VM VM VM VM VM VM VM How to interconnect two OpenStack deployments ?

? VM VM VM VM VM VM VM VM VM VM VM VM How to interconnect two OpenStack deployments ?

Neutron-Neutron interconnections Thomas Morin ? VM VM VM VM VM VM VM VM VM VM VM VM How to interconnect two OpenStack deployments ? ( two or more OpenStack ? two regions ?) Between datacenters or between NFV POPs, you may

203 views • 7 slides
QCD$Library$for$GPU$Cluster$with$ Proprietary$Interconnect$for$ GPU$Direct$Communica<on

QCD$Library$for$GPU$Cluster$with$ Proprietary$Interconnect$for$ GPU$Direct$Communica<on

QCD$Library$for$GPU$Cluster$with$ Proprietary$Interconnect$for$ GPU$Direct$Communica<on 1 , Hisafumi Fujii 1 , Toshihiro Hanawa 2 , Yuetsu Kodama 3 , ! Norihisa Fujita 4 1,3 , Yoshinobu Kuramashi 3 , Mike Clark Taisuke

690 views • 49 slides
At-Speed BIST for Board-Level Interconnect Artur Jutman artur@pld.ttu.ee Tallinn University of

At-Speed BIST for Board-Level Interconnect Artur Jutman artur@pld.ttu.ee Tallinn University of

EBTW05 EBTW05 At-Speed BIST for Board-Level Interconnect Artur Jutman artur@pld.ttu.ee Tallinn University of Technology, ESTONIA EBTW 2005, Tallinn, Estonia Slide 1 Outline EBTW05 EBTW05 1. Introduction 2. Interconnect Faults: Models

809 views • 54 slides
with Proprietary Interconnect TCA for GPU Direct Communication Kazuya MATSUMOTO 1 , Toshihiro

with Proprietary Interconnect TCA for GPU Direct Communication Kazuya MATSUMOTO 1 , Toshihiro

Implementation of CG Method on GPU Cluster with Proprietary Interconnect TCA for GPU Direct Communication Kazuya MATSUMOTO 1 , Toshihiro HANAWA 2 , Yuetsu KODAMA 4 , Hisafumi FUJII 5 , Taisuke BOKU 1,3 1 Center for Computational

600 views • 32 slides
Interconnect Delay Aware RTL Verilog Bus Architecture Generation for an SoC Kyeong Ryu,

Interconnect Delay Aware RTL Verilog Bus Architecture Generation for an SoC Kyeong Ryu,

Interconnect Delay Aware RTL Verilog Bus Architecture Generation for an SoC Kyeong Ryu, Alexandru Talpasanu, Vincent Mooney and Jeffrey Davis School of Electrical and Computer Engineering Georgia Institute of Technology August 2004 Outline

472 views • 21 slides
Interconnect Gate delay Wire delay The delay in VLSI circuits have two components Gate delay (

Interconnect Gate delay Wire delay The delay in VLSI circuits have two components Gate delay (

Advanced Digital IC-Design Propagation Delay Lumped C p Distributed model C model Interconnect Gate delay Wire delay The delay in VLSI circuits have two components Gate delay ( t pHL / t pLH ) Wire (interconnect) delay ( t w ) The Wire

492 views • 20 slides
Free-Space Optical Interconnect: an Alternative Optical Domain Electrical Electrical Domain

Free-Space Optical Interconnect: an Alternative Optical Domain Electrical Electrical Domain

12/5/10 Initial Results of Prototyping a 3-D Integrated Intra-Chip Free-Space Optical Interconnect Berkehan Ciftcioglu, Rebecca Berman, Jian Zhang, Zach Darling, Alok Garg , Jianyun Hu, Manish Jain, Peng Liu, Ioannis Savidis, Shang Wang, Jing

607 views • 12 slides
Characterization of Electromagnetic Properties of Molded Interconnect Devices Materials M ld d I

Characterization of Electromagnetic Properties of Molded Interconnect Devices Materials M ld d I

Institut fr Hochfrequenztechnik und Funksysteme d F k t Characterization of Electromagnetic Properties of Molded Interconnect Devices Materials M ld d I t t D i M t i l and their Effect on Radio Frequency Applications - 9 th

189 views • 18 slides

More recommend