Development of perfSONAR-Measurement Archive (MA) Capability based - PowerPoint PPT Presentation

perfSONAR workshop, 29th APAN Meeting 2010.2.11 Development of perfSONAR-Measurement Archive (MA) Capability based on PRESTA 10G NTT Network Innovation Labs. Kenji Shimizu, Katsuhiro Sebayashi 1 1 This work was partially supported by the National Institute of Information and Communications Technology.

Background PRESTA 10G PRESTA 10G � High-speed streaming application Network Interface Card and perfSONAR workshop, 29th APAN Meeting 2010.2.11 Network Interface Card and � Production quality Network Monitoring Platform Network Monitoring Platform � Uncompressed HD video stream (i-Visto) � Over one gigabit per second per stream � Network monitoring � Guarantee QoS for video stream � Required micro-second order accuracy and resolution i ‐ Visto eXmedia server i ‐ Visto gateway (XG ‐ 1) Optical IP Networks 6 ‐ Gb/s Uncompressed 4K video stream i ‐ Visto gateway (XG ‐ 2) 2 1.5 ‐ Gb/s Uncompressed HD video stream

Features of “PRESTA 10G Network Interface Card” 10-Gb/s network interface card perfSONAR workshop, 29th APAN Meeting 2010.2.11 with highly accurate traffic measurement capabilities � Supports 3 protocols for 10-Gb/s Ethernet LAN-PHY/WAN-PHY and OC-192c POS � 10-Gb/s wire-rate packet capturing and generating capabilities � 100-ns time-stamping for sending and receiving packets using external timing source � Hardware-assisted inter-packet gap control function for packet generating � Flexible traffic shaping function Timing source input (10 MHz, 1PPS) PCI ‐ X/DMA Packet processing engine 3 Winter 2010 ESCC/Internet2 Joint Techs, Feb. 2nd, 2010

Features of “PRESTA 10G Platform” Consists of general purpose PC and PRESTA 10G network interface card � perfSONAR workshop, 29th APAN Meeting 2010.2.11 Can be applied for passive monitoring, active measurement and traffic � generator � Easy development of 10-Gb/s network monitoring software � Easy migration of existing/open-source network monitoring tools into 10-Gb/s networks using libpcap-compatible API library High ‐ end, advanced traffic • Open ‐ source monitoring tools monitoring tools • Existing monitoring software •Non ‐ sampling packet capturing, •In ‐ service QoS monitoring •NetFlow probe, traffic, statistics reporting •Pre ‐ service network testing Libpcap ‐ compatible API library PRESTA API library provides raw software interface PRESTA NIC device driver Linux Kernel 4 PRESTA 10G Platform includes PRESTA network interface card

TARGET perfSONAR workshop, 29th APAN Meeting 2010.2.11 � perfSONAR as a sharing infrastructure of our precise and high-resolution measurement data taken from PRESTA 10G. � Currently, PRESTA 10G provides following measurement capability. One-way delay : micro-second accuracy. I. � Using probe packets or application’s packets. Jitter : micro-second accuracy. II. Inter-packet gap : 100-nano-second resolution. III. Available bandwidth estimation : by using packet train IV. generation well-controlled by a hardware-assisted function. Flow records : using NetFlow/PSAMP protocols V. Traffic bit-rate : 100-micro second resolution. VI. .... etc. 5

FIRST STEP: IMPLEMENTATION OF MA perfSONAR workshop, 29th APAN Meeting 2010.2.11 perfSONAR-MA on PRESTA 10G to show every- � micro-second traffic bit-rate . cf.) Existing perfSONAR-SNMP MA, ABW based on RRD archives provides every-second/minute bit-rate We needs higher-resolution (up-to micro-second) measurement extensions to existing perfSONAR. Long-tern Short-tern 6 Exinsting RRD-based perfSONAR-MA high-resolution perfSONAR-MA on PRESTA 10G

POINTS: MODIFICATION INTO PERFSONAR-PS_HRAMA � To take advantage of PRESTA 10G, we need to modify perfSONAR-PS to perfSONAR workshop, 29th APAN Meeting 2010.2.11 support 1. high-resolution timestamps (100-nanosecond resolution.) 2. accelerating to calculate a large amount of data from 10-Gbps high- speed networks. � HRA (High Resolution Archive) file formats � Meta-data to accelerate the calculation of bit-rate. � Fixed-length index. � Timestamps formats � sub-second expression: ISO-8601, UNIX timestamps. � Resolution definition in XML messages. � Unit of millisecond (“m”) and microsecond (“u”) � Compatibility with SNMP MA � RRDtools and backend HRA processor. 7 � The requests switches based on the resolution.

Existing HRA Viewer perfSONAR- OVERVIEWS UI perfSONAR workshop, 29th APAN Meeting 2010.2.11 � High-resolution archive (HRA) patches A daemon captures and stores in the � HRA formats by using PRESTA 10G . Res. <= 1s Res. > 1s A processor reads the stored data and � request. request. calculate traffic bit-rate in requested Existing perfSONAR-PS resolution. SNMP MA(ver. 3.1.8) + perfSONAR requests are switches based � HRA patch on the resolution. Switch based on requested resolution Res. <= 1s Res. > 1s What we made. HRA IF SNMP IF Existing codes. HRA HRA daemon RRDtools processor PRESTA-enabled libpcap HRA archive or RRD 8 8 pcap archive archive PRESTA 10G provides: •10-Gbps wire-rate capturing. •Accurate and high-resolution timestamps.

DEMONSTRATION perfSONAR workshop, 29th APAN Meeting 2010.2.11 HRA viewer Apache TCPmon (*) Host-only virtual network Virtual Machine (CentOS 5.4 x86_64) HRA-enabled perfSONAR MA HRA archives local IF (captured with (lo) PRESTA 10G) 9 (*) http://ws.apache.org/commons/tcpmon/

HRA Viewer It can make the bit rate visible perfSONAR workshop, 29th APAN Meeting 2010.2.11 by a high resolution as if the loupe. 350Mbps 250Mbps 160Mbps 140Mbps 125Mbps 150Mbps 0bps bit-rate calculated every 100 microseconds bit-rate calculated every 10 milliseconds 10 bit-rate calculated every 1 seconds

POINTS perfSONAR workshop, 29th APAN Meeting 2010.2.11 in SetupDataRequest. sub-second timestamp expression is supported by ISO-8601, UNIX timestamps. microsecond unit (“u”) is new definition. 11

CONCLUSION perfSONAR workshop, 29th APAN Meeting 2010.2.11 � We introduced PRESTA 10G platform for high-accuracy 10-Gb/s network monitoring. � We showed our motivation and implementation of perfSONAR measurement archive (MA) capability of PRESTA 10G. � Our perfSONAR-PS HRAMA takes advantage of our PRESTA 10G’s high-resolution accurate timestamps and 10-Gbps wire-rate traffic capturing capability. � We needs comments on � Any pointers to high-resolution perfSONAR (ABW) ? � GUI? (Should I merge to other UI?) � Next steps (perfSONAR-PS BUOY?), we are planning to 12 share the results of hardware-assisted available bandwidth estimation.