fsnsg
play

fsnsg Presented by: Bartlomiej Polot and Matthias Wachs Beyond - PowerPoint PPT Presentation

Jul 31, 2023 •392 likes •639 views

Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Nathan S. Evans Christian Grothoff Technische Universit at M unchen August 8, 2011 fsnsg Presented by: Bartlomiej Polot and Matthias Wachs Beyond Simulation:

  1. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Nathan S. Evans Christian Grothoff Technische Universit¨ at M¨ unchen August 8, 2011 fsnsg Presented by: Bartlomiej Polot and Matthias Wachs

  2. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Systems Research — Simulation Bug Implementation Bug Interpret Abstract Simulation Ideal use of simulation for security testing. Simulation N. Evans, C. Grothoff (TUM) August 8, 2011 2

  3. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Systems Research — Simulation Implementation Bug Interpret Abstract Simulation Ideal use of simulation for security testing. Simulation N. Evans, C. Grothoff (TUM) August 8, 2011 3

  4. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Systems Research — Simulation Simulation ??? Bug Design Implementation Exploit Reality for most simulation usage. Simulation N. Evans, C. Grothoff (TUM) August 8, 2011 4

  5. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Systems Research — Emulation APP APP APP APP APP APP OS OS OS OS OS OS VLAN VLAN VLAN OS OS OS DETER LAN/WAN Emulation N. Evans, C. Grothoff (TUM) August 8, 2011 5

  6. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Our Emulation Approach Test Control App APP APP APP APP APP APP APP APP OS OS OS LAN/WAN Emulation N. Evans, C. Grothoff (TUM) August 8, 2011 6

  7. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols GNUnet Architecture Peer A Your App P2P framework Focus on security Written in C Portable & extendable Core Multi-process architecture & IPC Peerinfo Extensive utility library Transport GNUnet N. Evans, C. Grothoff (TUM) August 8, 2011 7

  8. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols The Transport Service Transport API Low-level P2P connectivity GNUnet Transport Service host transport selection Transport plugins: send/receive provide many connection options TCP UDP HTTP WLAN Unix domain sockets GNUnet GNUnet GNUnet GNUnet Blacklisting & whitelisting GNUnet N. Evans, C. Grothoff (TUM) August 8, 2011 8

  9. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols P2P Emulation Steps Design P2P application Implement as GNUnet service Use built-in statistics or design logging facility Create test control application Links against emulation library Peer group startup/shutdown Utilizes API to access service GNUnet N. Evans, C. Grothoff (TUM) August 8, 2011 9

  10. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Our Emulation Approach Test Control App config generates generates config uses uses starts starts Peer A Peer B n Your App Your App y o i r t a calls calls Core Core a r b l u i Peerinfo Peerinfo m L E calls calls Transport Transport OS GNUnet N. Evans, C. Grothoff (TUM) August 8, 2011 10

  11. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Single Peer Startup Sequence start write Configuration File config generate est Control Application Hostkey File Emulation Library hostkey start time Running Peer peer running launches subsystems r e q u e s t s e v e n t s Running P2P App stop stop T peer clean up shutdown GNUnet N. Evans, C. Grothoff (TUM) August 8, 2011 11

  12. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Peer Group “Peer group” is the handle to running peers Layering — peer group reuses single peer startup code Peer group features Configuration mangling Resource allocation, throttling Connects peers in desired topology Capture running topology/statistics Start/stop/reconfigure peers Induce churn Provide handles to specific peers GNUnet N. Evans, C. Grothoff (TUM) August 8, 2011 12

  13. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Peer Group Startup, Code Example 1 s t r u c t GNUNET TESTING PeerGroup ∗ 2 GNUNET TESTING peergroup start ( const s t r u c t GNUNET CONFIGURATION Handle ∗ cfg , 3 unsigned i n t t o t a l , 4 s t r u c t GNUNET TIME Relative timeout , 5 GNUNET TESTING NotifyConnection connect cb , 6 GNUNET TESTING NotifyCompletion peergroup cb , 7 void ∗ p e e r g r o u p c l s , 8 const s t r u c t GNUNET TESTING Host ∗ h o s t s ) ; 1 GNUNET CONFIGURATION load ( t e s t i n g c f g , ”˜/ t e s t . conf ” ) ; 2 s t r u c t MyClosure ∗ data ; / ∗ your data here ∗ / 3 s t r u c t GNUNET TESTING Host ∗ h o s t s = GNUNET TESTING HOSTS load( ”˜/ h o s t s . conf ” ) ; 4 pg = GNUNET TESTING peergroup start ( t e s t i n g c f g , 20000 , TIMEOUT, &connect cb , 5 &peergroup cb , data , h o s t s ) ; 6 / ∗ peergroup cb must e v e n t u a l l y c a l l : ∗ / 7 GNUNET TESTING daemons stop ( pg , TIMEOUT, &shutdown cb , data ) ; GNUnet N. Evans, C. Grothoff (TUM) August 8, 2011 13

  14. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Network Topologies Simple (a) 2d-grid (b) Small-World topology creation/im- port/export (c) Erdos-Renyi (d) InterNAT GNUnet N. Evans, C. Grothoff (TUM) August 8, 2011 14

  15. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Topology Generation and Evolution (a) Initial (b) 2 Minutes (c) 5 Minutes (d) 10 minutes (e) 15 minutes (f) 30 minutes GNUnet Figure N. Evans, C. Grothoff (TUM) August 8, 2011 15

  16. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Limitations of Emulation Timing accuracy Network latency Speed Throughput Shared IP/hostnames Underlying OS interference Peer diversity CPU scheduling GNUnet Disk access Memory usage Limitations N. Evans, C. Grothoff (TUM) August 8, 2011 16

  17. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Overcoming Limitations Single OS per peer ⇒ Testing framework can be used on lower level emulators focused on timing accurate results PlanetLab, Emulab, DETER, etc. Sacrifice scalability Shared IP/hostnames — Virtual addresses, VMs Peer diversity — Configure per-peer bandwidth, VMs GNUnet — Benefit and limitation Limitations N. Evans, C. Grothoff (TUM) August 8, 2011 17

  18. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Important Lessons Learned Cryptography Start-up time Periodic tasks 01010101 Sockets Memory Lessons Learned N. Evans, C. Grothoff (TUM) August 8, 2011 18

  19. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Peer and Emulation Performance Memory consumption Service Non-shared Heap Shared supervisor 228 KB 32 KB 2,364 KB transport 359 KB 99 KB 2,888 KB core 300 KB 84 KB 2,428 KB dht 536 KB 240 KB 3,684 KB total 1,424 KB 456 KB 11,364 KB Lessons Learned N. Evans, C. Grothoff (TUM) August 8, 2011 19

  20. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Peer and Emulation Performance Architecture Hosts Cores Memory Peers Connections Time to (Total) (Total) per second start peer Cortex-A8 1 1 512 MB 100 ∼ 1 ∼ 206 ms Xeon W3505 1 2 12 GB 2,025 ∼ 60 ∼ 12 ms Xeon W3520 1 8 12 GB 2,025 ∼ 188 ∼ 5 ms Opteron 8222 1 16 64 GB 10,000 ∼ 327 ∼ 27 ms Opteron 850 31 124 217 GB 80,000 ∼ 559 ∼ 1 ms Lessons Learned N. Evans, C. Grothoff (TUM) August 8, 2011 20

  21. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Example: Comparison of DHT Performance Performance comparison of different DHT implementations 60,000 peers Specific peers were changed into malicious sybil nodes Success rate of requests measured DHT Example N. Evans, C. Grothoff (TUM) August 8, 2011 21

  22. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Example: NSE Implementation Network Size Estimation algorithm 2 days to implement 2 weeks from idea to paper Single host: 4,000 peers NSE Example N. Evans, C. Grothoff (TUM) August 8, 2011 22

  23. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Conclusion Framework available at https://gnunet.org We encourage people to use our framework 80,000 peers on cluster: what happens on supercomputer? at least consider: emulation vs. simulation even at large scale Conclusion N. Evans, C. Grothoff (TUM) August 8, 2011 23

  24. Beyond Simulation: Large-Scale Distributed Emulation of P2P Protocols Questions? The End. N. Evans, C. Grothoff (TUM) August 8, 2011 24

Recommend

More recommend