Stata Center Network Infrastructure Outline Introduction Physical - PowerPoint PPT Presentation

Stata Center Network Infrastructure Outline Introduction Physical topology overview Network design strategy Summary of specific requirements Proposal expectations/timeline Q&A

Stata Center Network Infrastructure Introduction The Ray and Maria Stata Center is a new building which will be a new anchor point on the MIT campus. It will house these departments: Laboratory for Computer Science Artificial Intelligence Laboratory Laboratory for Information and Decision Systems Department of Linguistics and Philosophy Each one is a recognized world leader in its field. The Net32 committee is chartered to design, specify, and validate a world-class data network to serve the needs of our world-class faculty, researchers, and students.

Stata Center Network Infrastructure Introduction (II) From Claude Shannon and Bob Fano’s pioneering work in information theory to Bob Metcalfe’s Ethernet; from Ron Rivest, Adi Shamir, and Len Adleman’s discovery of practical public-key encryption to Tim Berners-Lee and the World Wide Web; from Dave Clark and John Wroclawski’s differentiated services model of packet forwarding to Vincent Chan’s all-optical Next Generation Internet... We invented the technologies at the core of your business. (And that’s not even mentioning our seminal contributions in machine vision, machine learning, speech processing, natural language analysis and understanding, operating system design, complexity theory, medical informatics, synthetic imagery, computer architecture, economics, semantics, biological computation, and many other fields.)

Stata Center Network Infrastructure Physical topology (I) The building itself: Designed by world-reknowned architect Frank O. Gehry (see, e.g., the Guggenheim at Bilbao) Two five-story towers (Dreyfoos and Gates) Two-story high-ceiling "warehouse" Instructional facilities (N.I.C.) M/E/P and Parking facilities below grade

Stata Center Network Infrastructure Physical topology (II) Three riser columns per tower. Risers converge on 4th floor. Main data center to be located on 3rd floor, in two sections. Satellite machine rooms on G5 and D4. Capacity for >1000 machines in machine rooms. Includes multiple HPC clusters. One riser continues down to "mezzanine" in basement for telco entrance facilities, but the space is suboptimal for active electronics.

Stata Center Network Infrastructure Physical topology (III) Wiring to be supplied and installed by others. Each office will have four 10/100/1000 (Cat6) drops; lab and open spaces receive similar density. Thirty-seven "collapse points" for horizontal wiring with 96 ports each. Collapse points wired back to data center over SMF backbone. Most collapse points located in free-standing enclosures due to lack of closet space; some located in machine rooms.

Stata Center Network Infrastructure Network design strategy (I) 10:1 oversubscription for office drops; 5:1 for machine rooms. This is primarily a technology and cost limitation; upgradability is essential. (LCS’s current network is designed for 5:1 oversubscription; other departments vary.) Heavy use of 802.1Q VLANs for software management of network port assignment. "Any network, any drop" philosophy makes more efficient use of staff time by avoiding unnecessary trips to closets for moves, adds, changes. Clear segregation of switch function: Edge switches do layer 2 only Backbone switches do layer 2 and internal-only layer 3 Border routers do heavy IP lifting

Stata Center Network Infrastructure Security is fundamental (I) This is an open network: No firewalls Limited packet filtering System administrators responsible for host security Popular target for attackers Implications: Switches must be able to protect themselves. Proprietary, unauditable management technologies unacceptable. Network infrastructure must still function even under extreme loads ... especially in the obscure cases that nobody thought to optimize.

Stata Center Network Infrastructure Security is fundamental (II) Management security is an essential part of network security. That means: All switches must have out-of-band management (i.e., serial consoles) SNMPv3/USM/VACM Remote console access via SSHv2 Administrative access defined by user/role/view, not password Administrative access auditable More on these points in the requirements document.

Stata Center Network Infrastructure Network design strategy (II) Redundancy is a significant part of the network design: Each edge switch must have two paths to network core. Redundant paths may not be counted towards subscription ratio. Redundant paths need not be counted against subscription ratio. Every switch must have redundant power. More important switches should have redundant management/processor. Proposal should include on-site spares for all essential parts. This includes hot and cold spares, but not both for the same part.

Stata Center Network Infrastructure Requirements: Scale 3,552 10/100/1000 ports on edge switches serving offices 1,160 10/100/1000 ports on edge switches serving machine rooms (4,712 gigabits total) Approx. 70 10G ports at core

Stata Center Network Infrastructure Requirements: Redundancy Level and "temperature" of redundancy proportional to number of users affected by failure of a component. Hot spares for core infrastructure. Cold spares for edge switches. Uplink redundancy must be preserved in single-point failure.

Stata Center Network Infrastructure Requirements: Protocol Support -- Layer 2 IEEE 802.1 Standards: 802.1D Spanning Tree, enabled per-port 802.1s and 802.1w enhancements to Spanning Tree 802.1p encapsulation 802.1p priority labels configurably ignored at ingress 802.1Q VLANs 802.1D GVRP for one-way VLAN subscription (edge -> core) Random Early Detection on interface queues IGMP snooping (or equivalent) to limit multicast delivery

Stata Center Network Infrastructure Requirements: All Layer-3 Devices IPv4 and IPv6 No non-IP protocol support necessary. Equal-Cost Multi-path routing All traffic forwarded at wire speed, regardless of protocol ICMP responses configurable PIM-SM

Stata Center Network Infrastructure Requirements: Border Routers All core requirements plus: BGP4 Inter-domain multicast routing (MBGP, etc.) Multiple FIBs for policy routing Multiple 1-Gbit interfaces (internal and external) (See also under Monitoring.)

Stata Center Network Infrastructure Requirements: Configuration Management Human-readable ASCII text configuration files All configuration parameters available in files and console UI Secure remote configuration up/download; i.e.: SFTP SCP Secure remote firmware download Must be usable on a real terminal (e.g., VT420)

Stata Center Network Infrastructure Requirements: Security Management Administratively defined users and views Configurable views for SNMPv1/v2c clients Flexible wire-speed filtering at both L2 and L3/4 Non-password-based network authentication for remote console access

Stata Center Network Infrastructure Requirements: Monitoring Syslog SNMP MIBs for anything even remotely interesting Port mirroring Per-VLAN (per-RIF) interface statistics Complete bridge table Flow data export (at border) Traffic accounting per-interface (at border)

Stata Center Network Infrastructure What we expect in your proposals (I) N.B.: We do not expect that every vendor will have a complete solution. We will review the proposals for these areas independently: Edge switches Core switches Border routers Wireless (separate solicitation) Optional extras Your proposal should assume that we may buy one segment from you and another segment from your competitor. Proven interoperability counts!

Stata Center Network Infrastructure What we expect in your proposals (II) If more competitive pricing is available for a complete solution, you are welcome to include it in your proposal, in addition to the individual segments. However, even if your proposal or some component of it is offered gratis, we are not obligated to accept it ... ... unless our evaluation determines it to be an appropriate fit to our needs and requirements.

Stata Center Network Infrastructure What we expect in your proposals (III) Each proposal should include: Complete bill of materials with itemized prices Cost of NBD service contract Cost summary for the areas mentioned previously Availability/shipping time estimate Complete specifications for all products shown on BOM Equipment for evaluation Your proposal may include: Installation (but note Project Labor Agreement) Optional software and services Competitive evaluation results from independent test labs

Stata Center Network Infrastructure Some things to think about Lots of people in this area will be looking at what we’re doing and how well we succeed. This is a great opportunity for your companies to help us build a world-class network that can showcase your products. Many naming opportunities remain; contact Industrial Liaison Office for details. Our budget is more limited than we would like. If we are unable to meet our stated requirements, we will be forced to compromise to make budget. With your help, that won’t be an issue. Many of us are computer-science researchers. We expect a great deal from our data networks, and we use them in ways that commercial shops do not.