broker bro s new communication library
play

Broker Bros New Communication Library Robin Sommer ICSI / LBNL / - PowerPoint PPT Presentation

Aug 20, 2022 •188 likes •880 views

Broker Bros New Communication Library Robin Sommer ICSI / LBNL / Broala robin@icsi.berkeley.edu robin@broala.com http://www.icir.org/robin Bro Communication 2 Broker Bro Communication Exchanging Events Separates event generation from

  1. Broker Bro’s New Communication Library Robin Sommer ICSI / LBNL / Broala robin@icsi.berkeley.edu robin@broala.com http://www.icir.org/robin

  2. Bro Communication 2 Broker

  3. Bro Communication Exchanging Events Separates event generation from handling. 2 Broker

  4. Bro Communication Exchanging Events Separates event generation from handling. Logging Remotely Aggregates logs on remote side. 2 Broker

  5. Bro Communication Exchanging Events Separates event generation from handling. Logging Remotely Aggregates logs on remote side. Distributing State Provides shared view across instances. 2 Broker

  6. Bro Communication Exchanging Events Separates event generation from handling. Logging Remotely Aggregates logs on remote side. Distributing State Provides shared view across instances. 2 Broker

  7. Use Case: Bro Cluster Bro Manager Logs & Events Bro Bro Bro Worker 1 Worker 2 Worker 3 Load-balanced Traffic Stream 3 Broker

  8. Use Case: Bro Cluster Bro Bro Manager Proxy Logs & Events &synchronized State Bro Bro Bro Worker 1 Worker 2 Worker 3 Load-balanced Traffic Stream 3 Broker

  9. Use Case: External Integration Bro Events Application Application Application A B C 4 Broker

  10. Traditional Implementation Events Logs State Updates Bro A Bro B 5 Broker

  11. Traditional Implementation Events Logs State Updates Child Child Bro A Bro B Process Process 5 Broker

  12. Traditional Implementation Events Logs State Updates Child Child Bro A Bro B Process Process Events libbroccoli C Python Ruby Perl 5 Broker

  13. Traditional Implementation Events Logs State Updates It’s showing its age … Race conditions with &synchronized. Child Child Bro A Bro B No good persistence story. Process Process Not much control over data flow. Events Complex & inefficient protocol. Implementation deficiencies. libbroccoli Two independent implementations. C Python Ruby Perl 5 Broker

  14. Introducing the Broker Library Events Logs State Updates Bro A Bro B 6 Broker

  15. Introducing the Broker Library Events Logs State Updates Bro A Broker Broker Bro B 6 Broker

  16. Introducing the Broker Library Events Logs State Updates Bro A Broker Broker Bro B Broker C C++ Application Python 6 Broker

  17. Introducing the Broker Library Events Logs State Updates Broker Bro A Broker Broker Bro B Unified Library New protocol Broker Publish/subscribe C Limiting type system’s flexibility. C++ Application Explicit state operations. Python 6 Broker

  18. Exchanging Events with Broker my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver 7 Broker

  19. Exchanging Events with Broker Topic: “bro/event” my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver 7 Broker

  20. Exchanging Events with Broker subscribe_to_events(“bro/event”) Topic: “bro/event” my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver 7 Broker

  21. Exchanging Events with Broker subscribe_to_events(“bro/event”) Topic: “bro/event” my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver my_event(“Bro”, 42) 7 Broker

  22. Exchanging Events with Broker subscribe_to_events(“bro/event”) Topic: “bro/event” my_event(“Bro”, 42) my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver my_event(“Bro”, 42) 7 Broker

  23. Exchanging Events with Broker Demo 8 Broker

  24. Forwarding Logs with Broker Log::write(…) Bro Broker Broker Bro Sender Receiver 9 Broker

  25. Forwarding Logs with Broker Topic: “bro/log” Log::write(…) Bro Broker Broker Bro Sender Receiver 9 Broker

  26. Forwarding Logs with Broker subscribe_to_logs(“bro/log”) Topic: “bro/log” Log::write(…) Bro Broker Broker Bro Sender Receiver 9 Broker

  27. Forwarding Logs with Broker subscribe_to_logs(“bro/log”) Topic: “bro/log” Log::write(…) Bro Broker Broker Bro Sender Receiver Log::write(…) 9 Broker

  28. Forwarding Logs with Broker subscribe_to_logs(“bro/log”) Topic: “bro/log” Log::write(…) Log::write(…) Bro Broker Broker Bro Sender Receiver Log::write(…) 9 Broker

  29. Data Stores with Broker 10 Broker

  30. Data Stores with Broker Broker maintains global, persistent key/value stores. 10 Broker

  31. Data Stores with Broker Broker maintains global, persistent key/value stores. Broker Bro Bro Broker Client Master 10 Broker

  32. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) Broker Bro Bro Broker Key Value Client Master Authoritative Version 10 Broker

  33. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) insert(s, 192.150.187.12, 21) Broker Bro Bro Broker Key Value Client Master 192.150.187.12 21 Authoritative Version 10 Broker

  34. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) Broker Bro Bro Broker Key Value Client Master 192.150.187.12 21 131.159.14.1 5 Authoritative Version 10 Broker

  35. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) Broker Bro Bro Broker Key Value Client Master 192.150.187.12 21 131.159.14.1 5 Authoritative Version 10 Broker

  36. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) clone() Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 131.159.14.1 5 Cached Version Authoritative Version 10 Broker

  37. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) clone() Broker Bro Bro Broker { content } Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Cached Version Authoritative Version 10 Broker

  38. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 10 Broker Cached Version

  39. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) n = lookup(c, 192.150.187.12) insert(s, 131.159.14.1, 5) Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 10 Broker Cached Version

  40. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) n = lookup(c, 192.150.187.12) insert(s, 131.159.14.1, 5) insert(c, 192.150.187.43, 1947) Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 10 Broker Cached Version

  41. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) n = lookup(c, 192.150.187.12) insert(s, 131.159.14.1, 5) insert(c, 192.150.187.43, 1947) insert(…) Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker 192.150.187.43 1947 Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 10 Broker Cached Version

  42. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) n = lookup(c, 192.150.187.12) insert(s, 131.159.14.1, 5) insert(c, 192.150.187.43, 1947) insert(…) Broker Bro Bro Broker insert(…) Key Value Key Value insert(…) Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker 192.150.187.43 1947 192.150.187.43 1947 Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 192.150.187.43 1947 10 Broker Cached Version

  43. Data Stores with Broker Demo 11 Broker

  44. Data Store Features 12 Broker

  45. Data Store Features Data Types Supports Bro data types for keys & values. 12 Broker

Recommend

Who is Broker Direct? Broker Direct is a blend of Underwriting Agency, Wholesale Broker and Third

Who is Broker Direct? Broker Direct is a blend of Underwriting Agency, Wholesale Broker and Third

Who is Broker Direct? Broker Direct is a blend of Underwriting Agency, Wholesale Broker and Third Party Administrator focussed on the UK Personal and Commercial Lines market. The brand was established in 1997 with a unique proposition: a general

234 views • 8 slides
Broker Seminar Oct 17 Agenda Broker Solutions Manufacturer & Wholesaler of Investment

Broker Seminar Oct 17 Agenda Broker Solutions Manufacturer & Wholesaler of Investment

Broker Seminar Oct 17 Agenda Broker Solutions Manufacturer & Wholesaler of Investment Product 100% Broker focused Broker FX A new FX and International Payments Platform MiFID II & the New Consumer Protection

1.07k views • 80 slides
Training, Development and Communication at the University Library Introduction by Ilaria Moroni

Training, Development and Communication at the University Library Introduction by Ilaria Moroni

Training, Development and Communication at the University Library Introduction by Ilaria Moroni September, 2012 TDC - University Library Summary TDC Office: History, Staff and Activities Communication and Marketing Organizational

721 views • 27 slides
DRAFT DRAFT A service of Maryland Health Benefit Exchange Broker Information DRAFT DRAFT 2

DRAFT DRAFT A service of Maryland Health Benefit Exchange Broker Information DRAFT DRAFT 2

Broker & TPA JAD May 23, 2013 DRAFT DRAFT A service of Maryland Health Benefit Exchange Broker Information DRAFT DRAFT 2 Broker Topics Transaction files Broker XSD Group file 834 Individual initial enrollment with broker

730 views • 25 slides
Effective searching with the UvA Library What does the UvA Library offer you? 1. Books and

Effective searching with the UvA Library What does the UvA Library offer you? 1. Books and

Library - Stefano Giani august 2016 Effective searching with the UvA Library What does the UvA Library offer you? 1. Books and journals 2. Databases for subject-specific search: Communication and mass media complete , Web of science ,

845 views • 49 slides
SK Telecom 1 U U U U U U U- U - - communication - - - - - communication

SK Telecom 1 U U U U U U U- U - - communication - - - - - communication

U U U U U U U- U - - communication - - - - - communication communication communication communication communication communication communication Korea Mobile Market Trend for Convergence & Ubiquitous Communication 2004.

400 views • 16 slides
SMS Communication Library Lee, Chia-Peng 2015/12/09 Outline What can this lib do?

SMS Communication Library Lee, Chia-Peng 2015/12/09 Outline What can this lib do?

PCS Term Project: SMS Communication Library Lee, Chia-Peng 2015/12/09 Outline What can this lib do? Architecture Prerequisite Data structure How to use this lib? Example : FindMe! Error reporting Reference

460 views • 31 slides
XMLBlaster Rokey MOM (Message oriented Middleware) Mediator for messages (Broker) Asynchronous

XMLBlaster Rokey MOM (Message oriented Middleware) Mediator for messages (Broker) Asynchronous

XMLBlaster Rokey MOM (Message oriented Middleware) Mediator for messages (Broker) Asynchronous communication architecture Accessing to remote data through an internet or intranet Building applications with multiple servers and clients

116 views • 7 slides
UCX-PYTHON: A FLEXIBLE COMMUNICATION LIBRARY FOR PYTHON APPLICATIONS March 21, 2018 OUTLINE

UCX-PYTHON: A FLEXIBLE COMMUNICATION LIBRARY FOR PYTHON APPLICATIONS March 21, 2018 OUTLINE

UCX-PYTHON: A FLEXIBLE COMMUNICATION LIBRARY FOR PYTHON APPLICATIONS March 21, 2018 OUTLINE Motivation and goals Implementation choices Features/API Performance Next steps 2 WHY PYTHON-BASED GPU COMMUNICATION? Python use growing

315 views • 29 slides
K.SKOURAS CUSTOMS BROKER (AEOF CERTIFIED AGENT) Brief Company Presentation K.Skouras Customs

K.SKOURAS CUSTOMS BROKER (AEOF CERTIFIED AGENT) Brief Company Presentation K.Skouras Customs

K.SKOURAS CUSTOMS BROKER (AEOF CERTIFIED AGENT) Brief Company Presentation K.Skouras Customs Broker K.SKOURAS Customs Broker Kyriakos Skouras Customs Broker, is one of the leading agencies in the customs brokers industry in Greece, operating

534 views • 12 slides
ASN Broker Public Company Limited Opportunity Day 26 November 2018 ASN Broker Public Company

ASN Broker Public Company Limited Opportunity Day 26 November 2018 ASN Broker Public Company

ASN Broker Public Company Limited ASN Broker Public Company Limited Opportunity Day 26 November 2018 ASN Broker Public Company Limited Agenda Company Overview I Q3 2018 Financial Performance II Q3 2018 Key Developments III 2 ASN Broker Public

542 views • 16 slides
Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon

Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon

Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon '16 Communication in Bro Exploiting Independent State For Network Intrusion Detection Tap Internal Internet Firewall Tap Network Robin Sommer

802 views • 26 slides
eBooks In Your Library: Discoverability, Communication and The Future of eBooks. An interactive

eBooks In Your Library: Discoverability, Communication and The Future of eBooks. An interactive

eBooks In Your Library: Discoverability, Communication and The Future of eBooks. An interactive event . Simply drop your business card into the bowl and listen out at the end of this session it could be you! Interactive = have your say

465 views • 42 slides
Broker/Navigator Changes 4.1 Washington Healthplanfinder System Re l e as e April 11, 2017 65

Broker/Navigator Changes 4.1 Washington Healthplanfinder System Re l e as e April 11, 2017 65

Washington Health Benefit Exchange Broker/Navigator Changes 4.1 Washington Healthplanfinder System Re l e as e April 11, 2017 65 Broker/Navigator Changes Broker/Navigator Partnership Enhancements Navigators can Terminate Partnership with

512 views • 12 slides
Software Engineering and Architecture Broker IPC over HTTP History In the 1990, there was a

Software Engineering and Architecture Broker IPC over HTTP History In the 1990, there was a

Software Engineering and Architecture Broker IPC over HTTP History In the 1990, there was a lot of hype about building distributed systems in the OO paradigm / Broker based CORBA Version 1 - 1991 Microsoft DCOM About the same

554 views • 22 slides
Library support for your teaching LAW LIBRARY All commencing students are enrolled in online

Library support for your teaching LAW LIBRARY All commencing students are enrolled in online

Library support for your teaching LAW LIBRARY All commencing students are enrolled in online units: ACE / CARS / ISE Academic Conduct Essentials (ACE) In addition, undergrads are enrolled in: Communication and Research Skills (CARS)

405 views • 16 slides
Mortgage Broker Mortgage Broker 1. Brokers teach you how to buy. Good brokers will guide you

Mortgage Broker Mortgage Broker 1. Brokers teach you how to buy. Good brokers will guide you

8 Powerful Reasons 8 Powerful Reasons You Need A You Need A Mortgage Broker Mortgage Broker 1. Brokers teach you how to buy. Good brokers will guide you through the home buying process, from application, liaising with your solicitor,

674 views • 10 slides
The profitability analysis of the multi-band spectrum broker Marcin PARZY Poznan University of

The profitability analysis of the multi-band spectrum broker Marcin PARZY Poznan University of

The profitability analysis of the multi-band spectrum broker Marcin PARZY Poznan University of Technology Spectrum crunch 2 New licensing scheme DVB-T PMSE DVB-T PMSE BROKER BROKER Trading and auctioning

417 views • 13 slides
Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon

Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon

Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon '16 Communication in Bro Tap Internal Internet Firewall Tap Network Frontend Nodes Manager + Proxy + ... ... Proxy Backend Nodes

755 views • 40 slides
Efficient Communication Library for Large-Scale Deep Learning Mar 26, 2018 Minsik Cho

Efficient Communication Library for Large-Scale Deep Learning Mar 26, 2018 Minsik Cho

IBM Research AI Efficient Communication Library for Large-Scale Deep Learning Mar 26, 2018 Minsik Cho (minsikcho@us.ibm.com) Deep Learning changing Our Life Automotive/transportation Medicine and Biology Security/public safety Consumer Web,

342 views • 19 slides
Digital Communication Syst Digital Communication Systems ems ECS 452 ECS 452 Asst. Prof. Dr.

Digital Communication Syst Digital Communication Systems ems ECS 452 ECS 452 Asst. Prof. Dr.

Digital Communication Syst Digital Communication Systems ems ECS 452 ECS 452 Asst. Prof. Dr. Prapun Suksompong prapun@siit.tu.ac.th Fading Channels Office Hours: Rangsit Library: Tuesday 16:20-17:20 BKD3601-7: Thursday 16:00-17:00 1

470 views • 22 slides
ASN Broker Public Company Limited Opportunity Day 11 November 2016 ASN Broker Public Company

ASN Broker Public Company Limited Opportunity Day 11 November 2016 ASN Broker Public Company

ASN Broker Public Company Limited ASN Broker Public Company Limited Opportunity Day 11 November 2016 ASN Broker Public Company Limited Agenda I II III IV 2 ASN Broker Public Company Limited Company Overview 3 Company structure ASN

644 views • 21 slides
SEC Fiduciary Rule Initiative HISTORICAL DISTINCTION BETWEEN ADVISERS AND BROKER-DEALERS In

SEC Fiduciary Rule Initiative HISTORICAL DISTINCTION BETWEEN ADVISERS AND BROKER-DEALERS In

SEC Fiduciary Rule Initiative HISTORICAL DISTINCTION BETWEEN ADVISERS AND BROKER-DEALERS In the aftermath of the Great Depression, the U.S. securities industry was reorganized and regulated based on a fundamental premise that broker-dealers

447 views • 11 slides
Latest Case Law Developments Navigating the Evolving Scope and Breadth of a Broker's Duties and

Latest Case Law Developments Navigating the Evolving Scope and Breadth of a Broker's Duties and

Presenting a live 90-minute webinar with interactive Q&A Insurance Broker Liability to Policyholders for Denied Claims: Latest Case Law Developments Navigating the Evolving Scope and Breadth of a Broker's Duties and Obligations WEDNESDAY,

745 views • 51 slides

More recommend