created: 1243947419|%e %B %Y, %H:%M
Xump is meant to become an embeddable messaging engine that solves two recurrent problems (we see in OpenAMQ and Zyre), namely routing and storage. It should do for messaging products what game engines do for video games. Standard tactic for software engineering: make 1 and 2 by hand, then solve N with a general solution.
The design of Xump is a mix of ideas collected over the last few years. The engine manages a set of FIFO queues. Messages flow through one or more queues in a directed network of any complexity. Everything is a queue so the whole network is asynchronous. The calling application (that is, the product into which we embed the Xump engine) constructs this network dynamically at runtime, though the network may also be persistent. The lifecycle of all resources is visible to, and managed by, the calling application.
Queues pull messages from other queues via selectors, which specify static matching and filtering operations. Selectors are like stored search criteria that act on the head of the queue, eating up or otherwise operating on messages. A selector reads from one queue, and writes to another queue or set of queues. Matching is a O(logn) set operation, in which Xump compares a message address to the set of selectors that read on a queue. Filtering is a O(n2) operation in which Xump compares one message to one selector. In theory we match first, and filter second.
Selector match and filter operations are: Move, Copy, and Delete. These work as you would expect. Here is how two applications divide the flow of messages on a queue (the Wolfpack pattern for workload distribution):
Queue1
-> Selector MOVE
-> Queue2
-> Selector MOVE
-> Queue3
Here is how two applications each get the full flow of messages (the Parrot pattern for information distribution):
Queue1
-> Selector COPY
-> Queue2
-> Selector COPY
-> Queue3
A mix of selector operations is valid. Here is how two applications divide the work on a queue, while a third gets a copy of every message:
Queue
-> Selector MOVE
-> Recipient
-> Selector MOVE
-> Recipient
-> Selector COPY
-> Recipient
The implementation of selectors is extensible, via a virtual class interface, and the intention is that message processing applications that use Xump would provide their own implementations for the xump_select class.
To read messages off a queue, applications (that is, messaging servers which use Xump as their embedded engine) open and close "cursors" on queues. Xump cursors are analogous to the SQL cursor concept. The main difference is that when a cursor is open, messages are sent asynchronously to the application.
Applications use Xump by:
- Constructing a directed network of queues and selectors;
- Setting up cursors.
- Publishing messages to queues.
- Delivering messages received from cursors.
The persistence layer is extensible, via a virtual class interface, with the intention being that applications which embed Xump would provide their own storage layer implementations. Persistence happens at the queue level, so it is possible to e.g. provide persistent subscriptions for some clients, and transient ones for others.
Delivery semantics (e.g. message acknowledgement) affect selectors insofar as selectors may want acknowledgment for their operations, and effectively block until they get it. This would cause the queue to build up, typical for Wolfpack. Without this, typical for Parrot, queues will process messages at full speed and mainly remain empty (unless sheer processing costs caused backlogs).
Rate this post:
