Tag Archives: JMS

Guaranteed Topic Delivery using ActiveMQ Virtual Destinations

Background

The original JMS spec first arrived in 2001 with JSR 914. At the time several enterprise messaging systems were already widely available however each had their own unique features and mechanisms which required software that wanted to talk on a given messaging bus to be tightly-coupled to the specific messaging system implementation in use.

Given that “tightly-coupled” and “enterprise system” are phrases you generally want to avoid pairing, there was a push to abstract away the implementation-specific system aspects from the messaging client code. To address this, the JMS was developed to define a standardized, consistent programming interface that would work across different “JMS provider” implementations. The JMS spec, by its abstracted nature, could only define the lowest-common-denominator of messaging constructs so that they could apply to (or be adapted for) the widest number of messaging systems. The JMS therefore defined two core messaging delivery mechanisms (also referred to as “destinations”): queues and topics.

  • Queues are used for point-to-point messaging (1-to-1), where a message sent to a given queue will only be delivered to a single consumer
  • Topics are used for pub/sub messaging (1-to-many), where a message sent to a topic will be delivered to all subscribers currently listening to that topic.

Beyond the producer/consumer topology differences between the two, there is another key aspect to consider: time dependency.

Time Dependency

A message sent to a queue will remain on that queue until a consumer removes it. In other words, a queue consumer can be brought online and all messages that were sent to the queue prior to the consumer coming online will be delivered to it.

However, as hinted above, subscribers need to have an active subscription at the time a message is published in order to receive it. Any messages published when the subscriber is not listening will not be delivered to said subscriber. There is a slight exception to this rule, durable subscriptions, which allow subscribers to receive all messages between the time they subscribe until they unsubscribe, even if the subscriber was offline when a message was sent.

There are practical limitations to durable subscriptions which reduces the number of use-cases where they would be sufficient. One of the biggest constraints is that durable subscriptions don’t cover messages that were sent prior to the subscriber creating the durable subscription. For example if the sending system starts broadcasting messages at 7:00am and the recieving system registers a durable listener at 7:30am, all messages produced between 7:00a-7:30a will not be delivered to the subscriber. Although there are some situations where this type of operation is sufficient, the more common requirement is that the recieving system get ALL messages from the sending system, whether the recieving system was alive or not. What is needed is a pub/sub model but with the delivery guarentees that queues provide. This is where ActiveMQ virtual destinations can help.

Virtual Destinations

ActiveMQ Virtual Destinations are a way of mapping logical destinations to one or more actual destinations. There are a couple types of virtual destinations available but the one we will focus on is referred to as a Composite Topic. This particular virtual destination will behave as a topic destination for publishers, but we can map that topic to a series of queues (and/or topics) in order to achieve the delivery characteristics that we are shooting for.

Side note: If the termonology hasn’t been confusing enough thus far, I did not use the phrase “virtual topic” above which would have seem to have been a better word choice because “virtual topic” denotes a specific kind of ActiveMQ virtual destination! Virtual topics are similar to composite topics except that virtual topics are simplified (dare I say opinionated) virtual destinations that can be dynamically created by messaging clients based on a naming convention (ie ActiveMQ topic “VirtualTopic.FOO” will create a dynamic, virtual topic).

Composite Topics

Composite Topics are set up in the ActiveMQ broker configuration and allow for flexibility in designing messaging architecture.

Both these aspects are key: we want the flexability to map a single topic to multiple queues, and we need to define this setup in the broker’s initialization file so that the broker knows to create the physical queues on startup so it can begin dropping messages sent to the topic to them.

This is what it looks like inside the conf/activemq.xml:

<broker ...>
...
    <!-- Optional: Pre-define topic/queues -->
    <destinations>
       <topic physicalName="myLogicalTopic" />
       <queue physicalName="queueA" />
       <queue physicalName="queueB" />
    </destinations>

    <destinationInterceptors>
        <virtualDestinationInterceptor>
            <virtualDestinations>
                <compositeTopic name="myLogicalTopic">
                    <forwardTo>
                        <queue physicalName="queueA"/>
                        <queue physicalName="queueB"/>
                        <queue physicalName="queueC"/> <!-- This queue was not defined above to 
                                                            demonstrate that ActiveMQ will create 
                                                            a target destination on the first message -->
                    </forwardTo>
                </compositeTopic>
            </virtualDestinations>
        </virtualDestinationInterceptor>
    </destinationInterceptors>
</broker>

Producer/Consumer Setup

With the broker squared away, the only thing left is to add the appropriate code to the producer(s) and consumer(s). Fortunately since the broker is doing the heavy lifting of forwarding messages sent to the logical topic behind the scenes, we can just set up producer(s) to simply publish to the topic and each consumer to connect to their assigned queue. From the messaging client perspective there is nothing special or exotic that needs to be configured.

Trade-offs

Astute readers may have already guessed that there is some trade-offs involved going this route. The most obvious one is that in the way we’ve configured it, each receiving system would require its own queue, effectively losing the ability for subscribers to dynamically register. In practice this isn’t much of an issue since a) normally the number of receiving systems is known and limited and b) we could have easily added a physical topic to the mapping so that messages would also be forwarded to a standard topic. In addition, placing messages in queues rather than say durable subscriptions makes the operation much more visible and easy to work with and even promotes receiving systems’ ability to implement load-balancing.

Conclusion

Using virtual destinations the producer can enjoy the simplicity of a pub/sub model with the assurance that messages will be waiting for consumers to pull them from the broker when they are up and ready. Though any robust messaging architecture will require more design, planning, and configuration than we’ve touched on, virtual destinations are a tool that one should be aware of that may greatly simplify some otherwise very difficult use-cases.