Distributing Razor Events Using Pub/Sub (2/N)
You know how everyone talks about enterprise busses, but you've never met anyone who actually has one in production? Its because they're hella complicated. I mean, don't get me wrong, the documentation for WSo2 Enterprise Integrator is very good. But even knowing exactly what I want to accomplish it still feels like I've been handed a box of legos without any assembly instructions.
The core of pub-sub messaging in WSo2 is the Message Broker, which provides all the necessary configuration and message-schlepping functionality. However, the Message Broker only speaks JMS, which means that in most cases you need to wrap it in translation layers in order for it to talk to other systems. These translation layers are provided by the Enterprise Service Bus (ESB), which implements the Messaging Gateway pattern using something called a proxy service.
I'll stop here to add that figuring out everything in the previous paragraph took a non-trivial amount of time.
So in this post we're going to do the following:
- Configure the Message Broker for pub-sub
- Configure the ESB to talk to the Message Broker
- Set up a couple of mock proxy services to act as message consumers.
- Set up a proxy service to provide HTTP -> JMS message translation.
Configuring the Message Broker
Previously, we installed all the WSo2 components on our Razor server. Log into this server and fire up the Message Broker:
[root@razor ~]# wso2ei-6.5.0-broker
...
[2019-08-09 10:37:27,334] [EI-Broker] INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - WSO2 Carbon started in 40 sec
[2019-08-09 10:37:27,591] [EI-Broker] INFO {org.wso2.carbon.ui.internal.CarbonUIServiceComponent} - Mgt Console URL : https://192.168.15.254:9446/carbon/
Note that any log messages generated by the broker will show up in this window. In order to make the admin interface accessible from the host you'll need to set up a port forwarding rule:VBoxManage natnetwork modify --netname natnet1 --port-forward-4 'broker:tcp:[127.0.0.1]:9446:[192.168.15.254]:9446'
So, navigate to https://localhost:9443/ and log in using the default credentials (admin/admin), which will take you to the main interface for the broker. Pub-sub systems are ordered around the concept of topics, a collection of publication channels organized hierarchically in some sort of semantically meaningful fashion. Our initial task is to set up a topic hierarchy that makes sense for the Razor use case.
Razor emits various kinds of events, so it makes sense for the topic hierarchy to mirror this scheme:
razor + node-registered + node-bound-to-policy + node-unbound-from-policy + node-deleted + node-booted + node-facts-changed + node-install-finishedIn each case use the "Add" link under "Topics" to create the appropriate topic; you'll need to swap out '_' for '-' in topic names since the broker doesn't seem to like '-'. For the sake of experimentation make both subscription and publication available to everyone; in the real world you'd probably want to set up separate roles for the producers and consumers of Razor events.
Configuringing the ESB to talk to the Message Broker
Next up on the agenda is to tell the ESB to use the Message Broker as its source/sink for JMS-based communications. This process is covered in gory detail at https://docs.wso2.com/display/EI650/Configure+with+the+Broker+Profile, and amount to:
- Edit /usr/lib64/wso2/wso2ei/6.5.0/conf/axis2/axis2.xml:
- Search for 'EI Broker Profile', and uncomment the transportReceiver block immediately following.
- Ensure that the transportSender block with name jms is uncommented.
- Edit /usr/lib64/wso2/wso2ei/6.5.0/conf/jndi.properties:
- Ensure that the TopicConnectionFactory URL, which provides the address, protocol, and credentials for the Message Broker connection, is correct. The default value of amqp://admin:admin@clientID/carbon?brokerlist='tcp://localhost:5675' should work unless you've messed with the broker's credentials or interface bindings.
- Create a topic entry for the razor topic: topic.razor = razor.
Configuring mock message consumers
We don't yet know which systems are going to eventually end up consuming Razor messages, so to get started we're going to set up a couple of fake consumers which log that they've received a message and then drop it on the ground. Start the ESB:
[root@razor ~]# wso2ei-6.5.0-integrator ... [2019-08-09 11:41:58,272] [EI-Core] INFO - StartupFinalizerServiceComponent WSO2 Carbon started in 39 sec [2019-08-09 11:41:58,644] [EI-Core] INFO - CarbonUIServiceComponent Mgt Console URL : https://192.168.15.254:9443/carbon/As with the broker, keep an eye on this window for log messages originating from the ESB. Add an appropriate port forwarding rule:
VBoxManage natnetwork modify --netname natnet1 --port-forward-4 'esb:tcp:[127.0.0.1]:9443:[192.168.15.254]:9443'and then navigate to https://localhost:9443/, which will take you to the ESB interface. Credentials are the same as for the broker, admin/admin.
What we're going to do at this point is add a couple of proxy services via Services -> Add -> Proxy Service. We'll do this by creating a "Custom Proxy", switching to "source view", and then pasting in a modified version of the XML available at https://docs.wso2.com/display/EI650/Publish-Subscribe+with+JMS#Publish-SubscribewithJMS-Configuringthesubscribers. Here's what I arrived at after a little bit of tinkering:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="RazorEventSubscriber1"
startOnLoad="true"
statistics="disable"
trace="disable"
transports="jms">
<target>
<inSequence>
<property name="OUT_ONLY" value="true"/>
<log level="custom">
<property name="Subscriber1" value="I am Subscriber1"/>
</log>
<drop/>
</inSequence>
<outSequence>
<send/>
</outSequence>
</target>
<parameter name="transport.jms.DestinationType">topic</parameter>
<parameter name="transport.jms.Destination">razor</parameter>
<parameter name="transport.jms.ConnectionFactory">myTopicConnectionFactory</parameter>
<description/>
</proxy>
So what does this all mean? Here's my stab at how the above is being interpreted, based on https://docs.wso2.com/display/EI650/Using+the+ESB+as+a+JMS+Consumer#UsingtheESBasaJMSConsumer-One-waymessaging:
- Create a proxy service named RazorEventSubscriber1 which starts automatically and listens on the JMS transport.
- The specifics of how to connect to JMS are provided by the myTopicConnectionFactory paramter of the JMS transportReceiver block in axis2.xml (which we uncommented in the previous section).
- Listen on the channel razor, which is a pub-sub topic channel.
- When processing inbound messages:
- Do not expect a reply, as denoted by OUT_ONLY being set to true
- Log the key value pair Subscriber1/I am Subscriberl on receipt of a message.
- Drop the message.
- Outbound processing (return of a reply to the sender of the inbound message) just sends on the message unmodified. I believe this block should never be triggered because OUT_ONLY is set to true
You should see some activity as soon as you paste the XML above into the source view of ESB. The broker console should have messages indicating that the ESB has created a subscription to the razor topic:
[2019-08-07 14:34:06,457] [EI-Broker] INFO {org.wso2.andes.kernel.AndesChannel} - Channel created (ID: 127.0.0.1:63378)
[2019-08-07 14:34:06,596] [EI-Broker] INFO {org.wso2.andes.kernel.AndesContextInformationManager} - Queue Created: AMQP_Topic_razor_NODE:localhost/127.0.0.1
[2019-08-07 14:34:06,598] [EI-Broker] INFO {org.wso2.andes.kernel.AndesContextInformationManager} - Binding Created: [Binding]E=amq.topic/Q=AMQP_Topic_razor_NODE:localhost/127.0.0.1/RK=razor/D=false/EX=true
[2019-08-07 14:34:06,659] [EI-Broker] INFO {org.wso2.andes.kernel.subscription.AndesSubscriptionManager} - Add Local subscription AMQP subscriptionId=d506a4d7-6d1f-4c51-b294-f19004b04ff8,storageQueue=AMQP_Topic_razor_NODE:localhost/127.0.0.1,protocolType=AMQP,isActive=true,connection= [ connectedIP=/127.0.0.1:63378/1,connectedNode=NODE:localhost/127.0.0.1,protocolChannelID=b6cbb77f-2d95-4e7a-bfba-b8bd9ceb21f1 ]
Additionally, if you navigate to the topic subscription list on the broker (https://localhost:9446/carbon/subscriptions/topic_subscriptions_list.jsp?region=region1&item=topic_subscriptions) you should see a non-durable subscription corresponding to the proxy which was just configured.Provided that all worked as expected you should rinse and repeat the proxy creation process, changing the string Subscriber1 to Subscriber2 in the associated XML to differentiate the two proxy services.
Ok, let's make sure this part of the plumbing is working. Navigate to https://localhost:9446/carbon/topics/topic_manage.jsp and, in the "Publish" section set the topic to razor and the text message to "foo", then click "Publish". The ESB console should come back with
[2019-08-09 12:52:40,904] [EI-Core] INFO - LogMediator Subscriber2 = I am Subscriber2 [2019-08-09 12:52:40,904] [EI-Core] INFO - LogMediator Subscriber1 = I am Subscriber1We have successfully published a message (using the broker management interface) and verified that it was received by both of the consumers.
Create an HTTP -> JMS proxy
We've finished the backend (sort of, for the moment), now its time to work on the frontend. Again, we're going to create a proxy, but this one is going to take HTTP and translate it to JMS. Here's what I ended up with after modifying the inbound proxy from https://docs.wso2.com/display/EI650/Publish-Subscribe+with+JMS#Publish-SubscribewithJMS-Configuringthepublisher:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="RazorEventBusProxy"
startOnLoad="true"
statistics="disable"
trace="disable"
transports="http">
<target>
<inSequence>
<property name="OUT_ONLY" value="true"/>
<property name="FORCE_SC_ACCEPTED" scope="axis2" value="true"/>
</inSequence>
<outSequence>
<send/>
</outSequence>
<endpoint>
<address uri="jms:/razor?transport.jms.ConnectionFactoryJNDIName=TopicConnectionFactory&java.naming.factory.initial=org.wso2.andes.jndi.PropertiesFileInitialContextFactory&java.naming.provider.url=conf/jndi.properties"/>
</endpoint>
</target>
<description/>
</proxy>
Here's what's going on:- Create a proxy service named RazorEventBusProxy that starts automatically and listens on HTTP.
- When processing an inbound message:
- Do not expect a reply.
- Automatically return a 202 to the HTTP client, as denoted by FORCE_SC_ACCEPTED being set to true. This is apparently the appropriate pattern for a fire-and-forget API.
- The endpoint for these operations is the razor topic of the JMS connection defined by the TopicConnectionFactory entry in the conf/jndi.properties file. Messages should be forwarded to this endpoint with an implicit translation from HTTP to JMS.
[2019-08-09 13:42:04,618] [EI-Core] ERROR - Axis2Sender Unexpected error during sending message out java.lang.NullPointerException at javax.naming.NameImpl.so I used the longer, uglier version above.(NameImpl.java:283) at javax.naming.CompositeName. (CompositeName.java:231)
Ok, let's test out the front end. The address of the service, which can be obtained by navigating to https://localhost:9443/carbon/service-mgt/service_info.jsp?serviceName=RazorEventBusProxy, is http://razor.localdomain:8280/services/RazorEventBusProxy. So, set up an appropriate port forwarding rule:
VBoxManage natnetwork modify --netname natnet1 --port-forward-4 'services:tcp:[127.0.0.1]:8280:[192.168.15.254]:8280'And send a message:
curl -H 'Content-Type: application/xml' --data '' http://localhost:8280/services/RazorEventBusProxy The log messages from the consumers should show up as usual:
[2019-08-09 13:45:14,889] [EI-Core] INFO - LogMediator Subscriber1 = I am Subscriber1 [2019-08-09 13:45:14,890] [EI-Core] INFO - LogMediator Subscriber2 = I am Subscriber2The astute reader will notice that I changed the message payload. Seems like ESB really wants valid XML; setting the body to foo makes ESB sad:
[2019-08-09 13:59:05,759] [EI-Core] ERROR - RelayUtils Error while building Passthrough stream
org.apache.axiom.om.OMException: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character 'f' (code 102) in prolog; expected '<'
at [row,col {unknown-source}]: [1,1]
There's probably some way to change that behavior, but I've not been able to determine how just yet.
That concludes our initial foray into the world of WSo2. It's not for the faint of heart, but it shows promise in being able to solve some interesting problems. When we pick up next time we'll look at getting Razor to talk to the inbound proxy.
...
Or maybe not... when I picked this back up I started getting the following error when sending messages to the proxy service:
[2019-08-15 10:06:49,970] [EI-Broker] WARN {org.wso2.andes.server.AMQChannel} - MESSAGE DISCARDED: No routes for message - Message[(HC:346717419 ID:0 Ref:0)]: 0; ref count: 0
There has been some sort of breakdown between the proxy and the message broker; the broker is receiving messages but then is failing to route them to the subscribed listeners. I have not, despite much poking, been able to determine why this behavior is occurring, which puts an end to this exercise.
If I feel so inclined I may try this again with a different ESB system; it'll be interesting to see whether they're all as complicated to configure as WSo2.
posted by GG at 7:39 PM
0 Comments:
Post a Comment
<< Home