MIDDLEWARE SYSTEMS RESEARCH GROUP Publish/Subscribe Hans-Arno Jacobsen Bell University Laboratory Chair in Software Engineering Middleware Systems Research Group University of Toronto 1
Amazon to Chapters to you ... . MIDDLEWARE SYSTEMS RESEARCH GROUP Monday, October 10th in Cyberspace Thursday, November 15th, in Toronto Your book “...” is available at .... $10 off Publish/Subscribe Lecture 2
Business Process Execution & MIDDLEWARE SYSTEMS RESEARCH GROUP Web Service Composition Business Process Business Process Receive Receive Assign Switch Client Flow Scope Scope Pick Invoke Reply Scope Wait Reply Agent WS Broker … Broker … Broker Broker Broker Agent Database WS Client Publish/Subscribe Lecture 3
Other “Similar” Applications MIDDLEWARE SYSTEMS RESEARCH GROUP � Selective information dissemination � Location-based services � Personalization � Alerting services � Application integration � Job scheduling � Monitoring, surveillance, and control � Network and distributed system management � Workforce management � (Scientific) workload management � Business activity monitoring � Business process management, monitoring, and execution Publish/Subscribe Lecture 4
What Relates All These Applications? MIDDLEWARE SYSTEMS RESEARCH GROUP � Asynchronous state transitions captured as events that � drive and underlay � all applications and � infrastructures implementing these applications � Require middleware support for event processing � Publish/Subscribe is ideally suited to fulfill these requirements Publish/Subscribe Lecture 5
MIDDLEWARE SYSTEMS RESEARCH GROUP Publish/Subscribe Publish/Subscribe Lecture 6
Publish/Subscribe MIDDLEWARE SYSTEMS RESEARCH GROUP TSX Stock markets NASDAQ NYSE Publisher Publisher AMGN= 58 IBM= 84 Publications ORCL= 12 JNJ= 58 INTC= 19 HON= 24 MSFT= 27 Broker(s) Subscriptions: Notification Notification IBM > 85 Subscriptions ORCL < 10 JNJ > 60 Subscriber Subscriber Publish/Subscribe Lecture 7
That’s Like Data Base Querying � !! MIDDLEWARE SYSTEMS RESEARCH GROUP query publication About future About past Subscriptions Data (a lot of) (a lot of) Matching subscriptions Sets of tuples Query and subscription is very similar. Set of tuples and publication is very similar. However, the two problem statements are inverse. Publish/Subscribe Lecture 8
Key Benefit of Publish/Subscribe MIDDLEWARE SYSTEMS RESEARCH GROUP � Decoupling of publishers and subscribers � Publishers do not need to know subscribers � Publishers and subscribers do not need to be up simultaneously � Amenable for physical distribution Publish/Subscribe Lecture 9
Benefits of Publish/Subscribe MIDDLEWARE SYSTEMS RESEARCH GROUP � independence of participants � lends itself well to distributed system development � de-coupled development & processing � (dynamic) system evolution � interaction with large number of entities facilitated � naturally supports non-continuous operations � potential for scalability & fault-tolerance � open for (legacy) system integration on either end Of course it is not a one size fits all paradigm, but a good solution for certain kinds of problems. Publish/Subscribe Lecture 10
Publish/Subscribe Matching Problem MIDDLEWARE SYSTEMS RESEARCH GROUP � Given a set of subscriptions, S , and a publication, e , return all s in S matched by e . � e is referred to as event or publication � Splitting hairs � Event is a state transition of interest in the environment � Publication is the information about e submitted to the publish/subscribe system � Simple problem statement, widely applicable, and lots of open questions Publish/Subscribe Lecture 11
Problem Instantiations I MIDDLEWARE SYSTEMS RESEARCH GROUP � Text / search strings (information filtering) � Semi-structured data / queries � attribute-value pairs / attribute-operator-value- predicates � XML, HTML � Tree-structured data / path expressions � XML ./ XPath expressions � Graph-structured data / graph queries � RDF / RDF queries (e.g., SPARQL) � Regular languages / regular expressions � Tables / SQL queries Publish/Subscribe Lecture 12
Problem Instantiations II MIDDLEWARE SYSTEMS RESEARCH GROUP � Different matching semantics � Crisp � Approximate, � Similar � n-of-m (n of m predicates match) � Probability of match Publish/Subscribe Lecture 13
Problem Instantiations III MIDDLEWARE SYSTEMS RESEARCH GROUP � Centralized and distributed instantiation � Networking architecture � Internet (as overlay network) � Peer-to-peer style interface (DHT, table-lookup) � With mobile publishers, subscribers, brokers � Ad hoc network Publish/Subscribe Lecture 14
Publish/Subscribe Models MIDDLEWARE SYSTEMS RESEARCH GROUP � Channel-based model � Subscribe & publish to a channel � Topic-based model � … topics and topic hierarchy � Type-based model � … typed objects � Content-based model � … to content of messages � Subject-spaces (State-based model) � Maintain state in publications and subscriptions Publish/Subscribe Lecture 15
Channel-based MIDDLEWARE SYSTEMS RESEARCH GROUP Publisher Publisher Broadcast Channel Subscriber Subscriber Subscriber Publish/Subscribe Lecture 16
Topic-based publish/subscribe MIDDLEWARE SYSTEMS RESEARCH GROUP news publication US Canada politics sports politics sports soccer soccer Publish/Subscribe Lecture 17
The Content-based Model MIDDLEWARE SYSTEMS RESEARCH GROUP � Language and Data model � Conjunctive Boolean functions over predicates � Predicates are attribute-operator-value triples � [class,=,trigger] � Subscriptions are conjunctions of predicates � [class,=,trigger],[appl,=,payroll],[gid,=,g001] � Publications are sets of attribute-value pairs � [class,trigger],[appl,printer],[gid,g007] � Matching semantic � A subscription matches if all its predicates are matched Publish/Subscribe Lecture 18
Content-based routing MIDDLEWARE SYSTEMS RESEARCH GROUP � Distributed publish/subscribe � Network of publish/subscribe brokers � Subscriptions & publications are injected into network at closest edge broker � Routing protocol distributes subscriptions throughout network � Network routes relevant publications to interested subscribers � Routing is based on content ; it is not based on addresses, which are not available � Subscriptions may change dynamically Publish/Subscribe Lecture 19
Content-based Routing MIDDLEWARE SYSTEMS RESEARCH GROUP 1. Advertise Publisher 3. Publish Subscriber 2. Subscribe A: [class, =, stock], [name, =, HP], [price, >, 50] S: [class, =, stock], [name, =, *], [price, >, 50] P: [class, stock], [name, *], [price, 50] Event-Based Content Routing Flexible Decoupled Declarative Responsive
Applications MIDDLEWARE SYSTEMS RESEARCH GROUP Supply chain and logistics Workflows, business processes and job scheduling J o b A d o In flight n e A Trigger Delivered E B C Fault Order F D Event-Based Callback Razor SKU Invoke Loan Light T e m Transform p e r a t u r e Service oriented architecture RFID and sensor networks Publish/Subscribe Lecture 21
Publish/Subscribe in Industry MIDDLEWARE SYSTEMS RESEARCH GROUP � Emerging technologies � Standards � RSS aggregators � CORBA Event Service PubSub.com, FeedTree � � CORBA Notification � Real-time data Service dissemination � OMG Data Dissemination TIBCO, RTI Inc., � Service Mantara Software � Java Messaging Service � Application integration � WS Eventing Softwired � � Hardware-based brokers � WS Notification Sarvega (Intel), Solace � � INFO-D (Grid Forum) Systems, DataPower (IBM) Publish/Subscribe Lecture 22
Publish/Subscribe in Academia MIDDLEWARE SYSTEMS RESEARCH GROUP � Research projects � Elvin (Australia) � Gryphon (IBM) � Hermes (Cambridge) � SIENA (Boulder) � REBECA (Darmstadt) � ToPSS (UofT) � PADRES (UofT) Publish/Subscribe Lecture 23
The Toronto Publish/Subscribe MIDDLEWARE SYSTEMS RESEARCH GROUP System Family (ToPSS) ToPSS A-ToPSS CS-ToPSS (matching) (approximate) (composite subs) � Matching algorithms S-ToPSS L-ToPSS Rb-ToPSS � Language expressiveness, (semantic) (location-based) (rule-based) scalability, speed X-ToPSS persistent-ToPSS (XML matching) (subject spaces) � Routing protocols M-ToPSS P2P-ToPSS LB-ToPSS (mobile) (peer-to-peer) (load balancing) � Network architectures, scalability Federated-ToPSS Ad hoc-ToPSS (federation of ToPSS brokers) (ad hoc networking) � Higher level abstractions Historic-ToPSS FT-ToPSS � Workflow execution (historic data) (fault tolerance) � Monitoring JS-ToPSS BPEL-ToPSS (job scheduling) (BPEL execution) Publish/Subscribe Lecture 24
Overall Project Vision MIDDLEWARE SYSTEMS RESEARCH GROUP A Real-Time Event-driven Enterprise Publish/Subscribe Lecture 25
Matching and Content-based Routing MIDDLEWARE SYSTEMS RESEARCH GROUP output queues Routing Tables input queue … A n A n A P S S n S n A P S Publish/Subscribe Lecture 26
Recommend
More recommend
