PSNC NOC migration Szymon Trocha, PSNC 13.12.2012
Facts and figures PIONIER POZMAN Regional PSU, Servers Services generators Air Security CCTV conditioning 3 engineers Fire 24 / 7 / 1 operator detection
Rationales for change • New services, new responsibilities • More people, more space • Room refresh • Redesign for public viewing • Equipment detached from operation room • Better visibility • Power stability • Stable network connectivity • Dispose old workstations • Accommodate new workstations
Plan… • Accommodation of new space and room renovation • New and extended infrastructure – Fitted for new equipment and space organization • Replace old monitoring stations with new hardware and software – Integrate as much as possible • Deploy new TTS • Install videowall for large display of network and services status • Deploy new workstations to control videowall • Increase NOC team with additional engineers – Cover new services – Operator 24/7 • Integrate CCTV from multiple locations • Improve and extend telephone system to fit new room organization and NOC needs
… results • Started Dec 2011, still in progress… • New room • Dedicated infrastructure • Videowall • New workstations (but not main NMS) • New applications • Good support from management
Space • Previously – ~34 m 2 – Old wall painting – Equipment rack in the same room – Not sufficient lights – Insufficient visibility • Now – ~63 m 2 – Separated room for racks – Painted interior – New floor finish – New lights – Air-conditioning – Operator’s console
Infrastructure • New power circuits • Rack Automatic Transfer Switch • New power sockets, sufficient amount – Power switch protected for critical workstations – UPS protected • Cable ducts for signal cables along the main wall • Electromechanical lock for doors – Remotely controlled
Hardware and network • New workstations – Fujitsu-Siemens • E.g. 2 CPUs, 64 GB RAM, RAID • Wireless keyboards – Windows 7 – Specialized graphic cards – To fully migrate from SUN soon • New, sufficient network structured cabling – Minimise long connections and free laying cables • Used for phone lines too • Protection and redundancy assured – Dedicated LAN switch • Cold standby switch to replace – VRRP
Software iSpy RIM FEDERICA map (CCTV) (Juniper) Nfsen (NetFlow) ADVA NMS MRTG (SNMP) Nagios Request Tracker WiFi Poznań
Software ctd. • Also using – Dude • Regional network monitoring – What’s Up Gold • Commodity Internet – HP OpenView • Outdated (sw, hw) • Changing to new system part of the NOC migration • Candidates – HP OpenView – IBM Netcool • Targeting integration of different event sources under one umbrella – Minimise applications used – Better root cause analysis
Visualization
Visualization ctd. • Videowall – 2 x 6 NEC 55” Full HD • Space allocated for 2 x 8 • Very thin bezel • LAN control – Dedicated specialized flat panel CHIEF mounts • Precise regulation • NVIDIA Quadro NVS450 graphic cards – Multi-display graphics solution • 4 x DisplayPort (FullHD) – 2 cards in a single workstation (2x), 1 card in a single workstation (2x) – Control panel to control displays and place application windows – Very silent • No active cooling – Performance impact when scaling to multiple displays
Still to do • NMS migration (Jan 2013) – Change for software, OS, hardware, integration – Failover solution • Additional furniture to accommodate more persons (Jan 2013) • New CCTV system (1Q2013) – Central application – Large screen display • Telephony – Additional lines – Mobility • Integration of new services to monitor (1Q2013) – Enhanced monitoring – Minimise number of events – New dedicated engineers • Improve and automate reporting
Lessons learnt • Decision process is slow – Even if money exists – Take it into account when planning • Plan your room infrastructure carefully – There is always a need for additional socket • Tools must fit the infrastructure – Suitable for videowall display – Multi-screen display – Integration • Make sure NOC team follows changes in tools – Communicate and train • Keep you desk clean – there are visiting groups around the corner
noc.pionier.gov.pl noc.man.poznan.pl
Recommend
More recommend
