TF-NOC flash presentation DFN TF-NOC Feb 2012 Dubrovnik Thomas Schmid, schmid@dfn.de
DFN (1) • Head office in Berlin – Management, Project Management, Planning, Ordering, Services (Lambda Service, Roaming, PKI, Federations, LIR, IP ressources, VoIP, secondary DNS, netnews, PoP facility management, …) … – ca. 35 people • Branch Office in Stuttgart: 2 Teams – NOC/Ops/Engineering/PERT (5 FTE, 2 PTE) – IP, Mcast, VPNs, MPLS, Troubleshooting, Consulting, Testing, Planning, Projects, Migraeions, staging, … – Normal office hours, 24x7 for escalation. 1 of 4 people on-call duty. – Service Operation (VC, Mail, Spam-marking, Web, DSL, Serverhosting, PKI, AAI, Federations, Domain, …) (8 FTE)
DFN (2) • DFN CERT GmbH – Hamburg, 30+ FTE/PTE • WiN-Labor Erlangen – Statistics, Accounting, Hades, Perfsonar, Quality control, Geant Projects ... (ca. 8 FTE/PTE, DFN, Univ. of Erlangen) • Ixus GmbH – Spin-off in Berlin, 2 FTE • CNM-Team – Located at LRZ Munich, working also for Geant Projects, 2 FTE.
IP Topology mix of 10G and parallel 2 x 10G links Routerplatform Cisco 4 x CRS-1 55 x 7609 major topology change this year tender for 100G routers ongoing managed by DFN local loops to member institutes ordered and managed by DFN ca. 770 institutes redundant connection to 2 PoPs in sum 1000+ links Serving Universities, Research Institutes, PoPs at universities (except Frankfurt) polytechnical Universities, government agencies, commercial research labs, medical institutes, … primary/secondary schools via DSL and webhosting (few)
DWDM Huawei BWS 1600G system static lambdas, lead time 8 weeks for new lambdas managed by Huawei NOC has access to T2000 management client tender ongoing Equipment will be replaced until end of year ROADM + OTN switching? new platform: for 18 months managed by supplier. Depending on experiences then perhaps managed in-house?
Fiber country-wide leased fiber managed by fiber providers leased lambdas in few places
Service Desk Outsourced to Dimension Data 24x7x365 1st level active monitoring of routers, core and access links core: monitor interface status access: monitor BGP session to member or active link monitoring with ping coordinates troubleshooting in case of link failures between e.g. Huawei/fiber carriers or local loop suppliers Coordinates emergency hardware replacements dedicated users can open tickets communication by phone, mail, web-interface escalation 24x7 to NOC proprietary ticket system Dimension Data also takes care of PoP maintenance and is our Cisco contractor.
Tools: CNM Customer view on network weathermap individual usage statistics developed at LRZ Munich and part of PerfSonar project
Tools: PoP hardware documentation database home-made racks cabling power climate routers switches patches room plans PoP access ISDN/phone numbers … Used by NOC and Service Desk
Tools: GIS The most important Database/Tool in DFN! Topology database customer/user database IP addressing database (core, users) hardware database services database (DSL, VC, VoIP, …) statistics database billing contracts ressource-service mapping ordering procedures maintenance management process tracking data export to other tools/projects … Developed and maintained by Ixus GmbH, spin-off from former DFN interns
Tools: HPOV Mainly used during business hours. Probably replaced soon by e.g. Icinga.
Tools: MRTG frontend home-made basic monitoring and alarming function integrated Top-N List generation for numerous parameters
Tools: Accounting Traffic Matrix and user Accounting developed by Win-Labor Erlangen Operated by NOC used for planning
Tool: Mail instead of internal ticketing system home-made
Tools, misc • Tons of scripts that have been developed over the last 17 years – Yes, we still use e.g. one monitoring script that was written in 1993! – Monitoring, network planning, logging, router management, config management, alarming, process tracking, ... – pre-nagios/cacti/RT/… origin – Nagios, OTRS, … for services
Inter-NOC communication • Internal communication via VC, phone, mail, Jabber – Weekly VCs with Berlin – periodic phone conferences/VCs for ongoing projects – 2-day meetings outside Berlin for special topics • NOC-NOC communication via phone, mail(lists), IRC (commercial ISPs)
Misc • no written SLAs with customers • additional documentation in internal Wiki • MediaWiki, will be replaced by FosWiki • better document integration • “Betriebshandbuch” • documents operational procedures with Dimension Data, Carriers
Recommend
More recommend
