The “EAI MSMQ Transport” Business Service

Inspire by the “reloaded” Siebel Hub Business Service Library, I felt motivated to write about the “EAI MSMQ Transport” business service. We use MSMQ extensively in my current project and I’m impressed with the simplicity and power of the technology. The JMS equivalent (“EAI JMS Transport”) has it’s own very nice article from @lex and works in much the same way as the MSMQ flavour. What follows is a short summary of some of the key features of the MSMQ technology, used alongside the various EAI functions provided by our favourite enterprise software.

The premise is relatively simple: Microsoft Message Queuing (hereafter refered to as MSMQ) allows systems and processes to communicate in a “fail safe” manner. That is, if the connection between systems were to break or disappear, message communication resumes when the connection is re-established. This is in contrast to socket or point to point communication mechanisms which require an “always on” connection.

Creating an MSMQ queue

Setting up MSMQ is simple, provided “Message Queueing” has been enabled on the Windows host. This is typically enabled via Server Manager on Windows Server hosts and through Windows Features on Windows Client hosts, such as Windows 7.

Creating a queue is equally as straight forward and can be done using PowerShell or via the UI. Via the UI, the process is easy:

  1. Right click ‘My Computer’ and select ‘Manage’
  2. Under ‘Services and Applications’ select ‘Message Queueing’
  3. Expand ‘Private Queues’ to see a list of queues enabled on that machine
  4. Right click ‘Private Queues’ and select ‘New > Private Queue’
  5. Give the queue a name and check the ‘transactional’ checkbox
    • A transactional queue allows blocks of messages to be treated as a single, atomic unit. If MSMQ fails to process that entire block, all messages that have been processed are rolled back
  6. Click ‘OK’ to create the queue

CreateQueue

MSMQ introduces the concept of Private, Public, Outgoing and System queues. We use Private queues in order to strictly control which service and administrative users have access to the queues. You can read more about the other queue types on MSDN.

Sending to an MSMQ

Sending XML to an MSMQ is simple, using the “EAI MSMQ Transport” business service. Two methods should peak your interest:

  • Send – this method will send a message to a specified queue. The method is “fire and forget” and Siebel will not be informed of the fate of the message
  • SendReceive – this method will send a message to a specified queue and await a response message. This allows for end to end processing, where the recipient sends either a confirmation of receipt or some sort of response or feedback from a process initiated by the outound message.

Method arguments that you’ll typically use are:

  • <Value> this should be initialised with the XML or text that you want to post to the queue
  • MsmqPhysicalQueueNamethe name of the queue to post to. This should be an MSMQ path such as “private$\mytestqueue”
  • MsmqQueueMachineName – the host name of the machine that contains the target queue. This can be localhost if you’re running everything locally

Using this Business Service method in a Workflow Process is usually combined with calls to the following EAI Business Services:

  • EAI Siebel Adapter.Query – this method creates a Integration Object instance from data in your Siebel Database.
  • EAI TransformationEngine.Execute – this BS method uses a Siebel Data Map to transform an IO instance into an external IO instance with a different schema
  • EAI XML Converter. IntObjHierToXMLDoc – this method tranforms the IO instance into an XML string, ready for transmission to MSMQ

An MSMQ outbound Workflow Process may look something like this:

MSMQSendWorkflow

Running the Workflow, with an appropriate “Object Id” of course, will result in an XML message containing your EAI query results on the queue. You can see it by going back to your MSMQ machine, expand your queue name and click on “Queue Messages”:

NewMessage

You could also invest a few pennies in the MSMQ Explore tool, available exclusively from Ollerenshaw IT Ltd!

Receiving from an MSMQ

Receiving a message is a little more involved, especially if you want Siebel to monito a queue and automatically process messages from it. Three additional elements are required to configure Siebel to systematically process messages from an MSMQ queue. There elements are:

  1. An “MSMQSubsys” profile configuration – this profile details the name and host of the MSMQ queue. The following parameters should be set:
    • MsmqPhysicalQueueName – the name of the queue to post to. This should be an MSMQ path such as “private$\mytestqueue”
    • MsmqQueueMachineName – the host name of the machine that contains the target queue. This can be localhost if you’re running everything locally
  2. A “EAITransportDataHandlingSubsys” profile configureation – this profile configuration specifies what Siebel should do with a message when it is received. Typical scenarios would invoke either a Workflow Process or a Business Service. The following parameters should be set:
    • DispatchWorkflowProcess – <Name of the Workflow process to invoke>
    • DispatchService – Workflow Process Manager
    • DispatchMethod – RunProcess
  3. A  “Enterprise Application Integration Receiver” Component Definition – this component effectively references the two profiles to “poll” the MSMQ and pass received message contents onto the handling Workflow or Business Service. Set the following parameters as a minimum:
    • ReceiverConnectionSubsystem – the name of the MSMQ Queue profile
    • ReceiverDataHandling Subsyst – the name of the Handler profile
    • ReceiverServiceName – EAI MSMQ Transport
    • ReceiverMethodName – ReceiveDispatch
    • Default Tasks – set this to 1 to ensure the component comes up and is listening from when the Siebel Server starts up

A Workflow Process, who’s name you’ll reference in the handler profile above, will look remarkabely similar to the Workflow you created for sending messages, using different methods of the same Business Services:

  • EAI XML Converter. XMLDocToIntObjHier – this method tranforms the XML message back to an Integration Object instance
  • EAI TransformationEngine.Execute – this BS method uses a Siebel Data Map to transform the external IO instance into an internal IO instance with a different schema
  • EAI Siebel Adapter.Upsert – this method applies your IO instance to the business layer, updating and creating data

A receiving Workflow Process may look something like this:

MSMQReceiveWorkflow

Using listeners and handlers is a mechanism to consistenly poll queues for messages, acting upon them as soon as they are received. It is possible to “manually” receive a message from the queue using one of the three “Receive” methods:

  1. Receive – simply receives / pops the next message from the queue and places it into a property
  2. ReceiveDispatch – receives the next message from the queue and dispatches it to a handling service
  3. ReceiveDispatchSend – receives the next message from the queue, dispatches it to a handling service then sends a response message

MSMQ, and other queuing technologies such as JMS, are a simple and elegant mechanism for providing failsafe, transactional communication between systems. The Siebel EAI business service family provide everything you need, and more, to extract, manipulate and manage structured message data using industry standards. Combined, you have a very easy to use mechanism for Siebel to communicate with the outside world.

 

MSMQ and Windows Powershell

I’m deviating a bit from Siebel these days, but it’s nice to play around with different technologies. We use MSMQ as the message transport system in our Siebel / BizTalk environment so there is some connection.

We had an issue recently where we had a load of messages dumped as XML files to a file system and we needed to get them onto the queue for BizTalk to process them. In the interest of getting things done quickly, I knocked up a little Powershell script that did the job nicely. Powershell is just one of these little tools that lets you do what you need to do quickly and efficiently, leveraging the power of the .NET platform without having to get into the nitty gritty of implementation specifics.

Anyway, here’s the script in all it’s glory in case you come across a need to do the same. I wonder if Oracle are planning Siebel .NET assemblies in the future? Now that would be nice!

As ever, code is provided out of the kindness of my heart – please use at your own risk!

MSMQ Explorer 1.3

On a more positive note following my earlier posts, I’ve been tweaking my MSMQ Explorer tool some more. It’s a nice change to work in Visual Studio after a hard day in Siebel Tools! 😉

The latest version has a number of new features:

  • Redesigned UI
  • Scintilla.NET controls with XML code highlighting and folding
  • Bug Fixes

It can be found in the Siebel Store for download today.

MSMQ Explorer

More dev updates to my MSMQ tool, including a rename to MSMQ Explorer. New features of this build are:

  • Source XML control supports drag and drop and double click to load XML from a file
  • New ‘Send’ and ‘Receive’ tabs
  • New ‘Explorer View’ in ‘Receive’ tab allows you to:
    • View all private and public queues active on the current host
    • See message counts across all queues, including their journal queues
    • ‘Preview’ any message from the queue, without ‘popping’ it from the queue
  • ‘Label’ a message on the ‘Send’ tab and see the label, message ID, time sent and originating host in the ‘Receive’ tab

[soliloquy id=”6775″]

MSMQ Explorer is available for download from The Siebel Store.