Reliable Multicast in the STOW RTI Prototype 97S-SIW-119 Harry - PowerPoint PPT Presentation

Reliable Multicast in the STOW RTI Prototype 97S-SIW-119 Harry Wolfson <HarryWolfson@LL.MIT.EDU> Steve Boswell <Boswell@LL.MIT.EDU> Daniel J. Van Hook <dvanhook@LL.MIT.EDU> Steve McGarry <smcgarry@LL.MIT.EDU> MIT Lincoln Laboratory March 6, 1997 Wolfson 1 97S-SIW-119

Outline • Introduction • Message Transport Services • Reliable Multicast • Discovery Service • Emulation of Reliable Multicast in RTI-s • Performance Measurements • Summary Wolfson 2 97S-SIW-119

Federation using the HLA / RTI Fe de ra te Fe de ra te Fe d e ra te Fe de ra te Fe de ra te Fe d e ra te RTI RTI RTI RTI RTI RTI C o mm C o mm Co mm Co m m C o mm Co mm Wolfson 3 97S-SIW-119

RTI-s Sponsors • Defense Modeling and Simulation Office (DMSO) – Part of larger DoD 1.0 RTI development effort – Initial implementation of Data Distribution Management Services – Support initial use of the HLA in a large scale, real-time system • DARPA – Synthetic Theater of War (STOW) program » STOW is an Advanced Concept Technology Demonstration (ACTD) » Customer is Atlantic Command (ACOM) » An HLA based simulation system Wolfson 4 97S-SIW-119

RTI-s Development • STOW Requirements (RTI perspective) – ~ 25 Federate classes on 350-450 host computers – 10 – 20 sites (LANs connected via a WAN) – 10,000 objects (scaleable to 100,000) • MIT Lincoln Laboratory is developing the prototype RTI-s for use by STOW – Full RTI functionality except for Time and Ownership Management – Focus on real-time performance and very large exercise scalability » Low latency, High throughput, Low bandwidth overhead – Early availability of RTI (15 Oct 96) Wolfson 5 97S-SIW-119

RTI-s Block Diagram RTI Ambassador Remove Obj, S ubscribe, P ublish, Control Create, Fe d Create, Update Attr, Refle ct Attr, Remove Obj, Set Trans p Provide Attr Assoc Regions Updates Delete Join, etc. S end Interaction Re c Interaction P rovide Attr Mode Obj { oid, attribute_values or, class , param_values } Subs cription P rincipa l Object Transport Ma nager Ma nager S ubscriptions , Streams Publications , of Interest Resources { oid, attribute_value s Federa tion or, class , param_values } Local Manager Consistency S ubscription Mana ger Agent Minimum Be st S tate Reliable Ra te Consistent Effort S trea m create Mappings delete Reques t Buffe r update { stream, data, count} Manager Consistency Discovery P rotocol P rotocol Distribution Res ource s Databas e Stream Manager RTI Initializa tion Data Communication UDP TCP Rel Distrib (FED + RID) Managers Wolfson 6 97S-SIW-119

Message Transport Services • Best Effort – Based on UDP Multicast » Permits scaling to large exercises – No mechanism for reliability or message ordering • Minimum Rate – Comparable to DIS Transport mechanism • State Consistent – Ensures delivery of the latest attribute value – No guarantee of intermediate message delivery • Reliable – Guaranteed delivery of messages – Messages delivered in order Wolfson 7 97S-SIW-119

Reliable Multicast • Reliable Multicast Protocols – RAMP, RMP, SOM (MTP-2) – Based on UDP to support multicast – Some provide reliability based on NACKs – May be optimized for single sender (one-to-many) – Some are optimized for a fixed set of senders and receivers • Emulation of Reliable Multicast in RTI-s – Dynamic message routing based on streams – Reliability provided by use of TCP protocol – Does not consume UDP multicast groups – Multicast emulated via use of hierarchical topology of Reliable Distributors (exploders) Wolfson 8 97S-SIW-119

Discovery Services • General purpose mechanism to facilitate contact between distributed service providers and clients – Predates similar capability in CORBA – Supports multiple providers of same service type • Provider advertises its service – Service type – Qualifier: discoverer can screen for desired value – Invariant data: constant provider info (eg. socket addr) – Variant data: time-varying provider status info • Client discovers available providers – Only receives advertisements from appropriate providers – Initiates connection with provider based on information in advertisement (eg. Invariant data) • Clients may monitor status of providers – Fault detection – Load balancing Wolfson 9 97S-SIW-119

Discovery Service Manager (DSM) Fe d e rat e Fe d e rat e RTI Se rvice s DS M Fe d e rat e RTI advert ms gs RTI S erv ices DS M Se rvice s DS M Advert Dis cov er Monitor C A • • • RD advert ms gs Fe d e rat e s end lis ten and callb ack Reques tor RTI • Periodic Timer • • S e rvice s DS M advert ms gs adv ert ms gs advert ms gs Wolfson 10 97S-SIW-119

Implementation of Reliable Multicast in RTI-s • Based on TCP point-to-point connections • Reliable Distributor services a number of clients that wish to send and / or receive reliable message traffic • Clients connect to appropriate Reliable Distributor using information provided by the Discovery Service Wolfson 11 97S-SIW-119

Sample Network Topology for Reliable Servers / Clients LAN # 1 Fe de ra te Fe de ra te RTI RD tcp RTI tcp Fe de ra te RTI tcp RD Fe de ra te RTI Fe d e ra te RD tcp Fe de ra te Fe de ra te RTI tcp Fe de ra te RTI RTI tc p tcp LAN # 3 RTI tcp LAN # 2 • STOW is using fully connected mesh – Could configure Server connections as a-cyclic tree Wolfson 12 97S-SIW-119

Features of RTI-s Reliable Multicast • Robust – Outgoing message queue (accommodates bursty traffic) – Re-establishing connectivity (fault tolerance) • Provides atomic message delivery within a stream based protocol (TCP) – Framing of received messages – Reassembly of fragmented messages – Fragmentation of large outgoing messages • Configurable Server hierarchy – Auto-configuration for small Federation Execution, or in a small scale testing environment – Configurable server / client topology for larger Federations that span LANs and WANs – Stand-alone or integrated servers Wolfson 13 97S-SIW-119

Performance Measurements • Latency – Limits the ultimate responsiveness of a system – Defined as elapsed time between invocation of the updateAttributeValues in one Federate, and the corresponding invocation of the reflectAttributeValues method in the second Federate • Throughput – Defined as the number of transactions per second that can be processed by a system on a sustained basis Wolfson 14 97S-SIW-119

Latency Performance Reliable Transport Service in RTI-s 2 0 0 RD RD 1 5 0 3 hop la te ncy (mS e c) Latency (mSec) 2 hop la te ncy (mS e c) Rcv 1 0 0 Snd 3 Sun Ultra 1/140’s and 1 5 0 SGI Onyx, using 10Mbps Ethernet Time: Snd updates, Rcv 0 reflects, Rcv updates, Snd 1 2 3 4 5 1 0 10 1 0 1 0 1 0 reflects; divide result by 2 Me s s a g e s iz e (b yte s ) • Dominated by Ethernet bit rate Wolfson 15 97S-SIW-119

Latency Performance with Receiver “Fan Out” Reliable Transport Service in RTI-s 5 0 0 RD Rcv 1 4 0 0 Rcv 2 pre dicte d: 10 rcvr pre dicte d: 5 rcvr Latency (mSec) 3 0 0 pre dicte d: 3 rcvr pre dicte d: 2 rcvr Snd Rcv N me a s ure d: 1 rcvr 2 0 0 1 0 0 Predicted Latency for N th 0 Receiver: 1 2 3 4 5 10 10 1 0 1 0 1 0 L R N = L R 1 / 2 + ( N * L R 1 / 2 ) Me s s a g e s iz e ( b yte s ) Wolfson 16 97S-SIW-119

Throughput Performance Reliable Transport Service in RTI-s 4000 RD 3000 messages per second Ethernet-limited Rcv 2000 Snd operating region 3 Sun Ultra 1/140’s, using 1000 10Mbps Ethernet Snd updates, Rcv reflects 0 Rates are sustained, with 10 5 1 10 100 1000 10000 no packet loss m e s s a g e s iz e (byte s ) Wolfson 17 97S-SIW-119

Future Work • Continue evaluation of Reliable multicast performance in the context of STOW • Deliver rti-s/C to STOW, 24 March 97 • Merge rti-s with RTI 1.0 to create RTI 1.1 • Re-visit evaluation of available reliable multicast protocols Wolfson 18 97S-SIW-119

Summary • Reliable Message Transport Service in RTI-s emulates reliable multicast protocol – Based on TCP point-to-point connections – Reliable Distributors service clients and interconnect with other RDs – Provides adequate performance for STOW • Discovery Service provides mechanism for Service providers to find each other, and for clients to find servers – Provides connection information – Allows monitoring of providers status Wolfson 19 97S-SIW-119