Azure Service Bus Messaging with Dynamics AX 2012 (part 3)


Our final post in the Azure Service Bus series for AX 2012.  As mentioned in our previous posts, this is generally applicable to all versions of AX 2012 and not just R3 ( as used in these demonstrations ).  Our focus for this entry will be on using the SysOperation Framework to empower a multi-threaded approach to Service Bus.

AX has several options for multi-threading your business processes.  Each has positives and negatives in terms of throughput and performance. 

One option is called Task-based processing, which will generate a separate thread for each unit of work you have to process.  This has an advantage of scaling 100% to the work load, but may become a bottleneck on your batch server as it can use 100% of the threads. 

Another is called the Helper approach, which uses a master thread and a staging table to keep track of the work, and pessimistic locking of that table to lock records as they are processed in separate threads.  This does not scale well, but may actually finish quicker in some scenarios.

In terms of Service Bus, I wanted to avoid using a staging table, but wanted something that could be ramped up if needed.  Using bundling logic with a fixed amount of threads, I am able to achieve high throughput without impacting other uses of the batch servers.

The data contract produces the following dialog:

The above is an example.

  • Number of Helpers – This determines the number of threads to process for a given company. 
  • Queue Name – This is required to access the Topic.
  • Security Key – This is from Microsoft Azure. 

This job runs in the context of a single AX company.  It will query the Azure Topic with its own DataAreaID.  If a subscription to the topic does not yet exist for this company, one is created.

The service operation which uses this contract is below. 
 

You’ll see we initialize our connection to the service bus proxy and then query to get the message count on the Topic.  From there, we determine our bundles by performing integer division of the total by the number of helpers.

If the topic is empty, no additional threads are generated.  Conversely, if the topic has messages to process, we split up the work by two threads in the above example.  The job can always be put on Withhold, and then more helpers added if we are finding the job isn’t finishing up on time.

You could easily modify this example to a task approach, generating a single thread for every single message in the topic or queue.  It’s up to you, if you have dedicated resources for your job then go for it.  Otherwise, you may find this bundling approach to be more appropriate. 

For stress testing, load the queue with 10,000 messages and make a judgment call on how many threads you need based on your hardware.

Enjoy!