Microsoft ESB Guidance - Message flow : Part I

The ESB Guidance are set of tools that extends BizTalk from regular EAI 'Hub-and-Spoke' Broker, into the 'SOA' world. One of the first thing you need to know about the guidance is that there's (almost) no magic or new stand alone feature in there - the guidance takes you beyond 'EAI' by understanding BizTalk 2006 architecture and leading BizTalk features to the wanted results.

The main components are:

• Itinerary On/Off ramps - Extends BizTalk Messaging and BizTalk Adapter framework.
• Core services (Agents) - Extends BizTalk Orchestration.
• Exception management - Extends BizTalk Error report (For Orchestrations) and ACK/NACK notification (For pipelines)
• Web services - Extends BizTalk transformation API and ExplorerOM API

In the following, slides from the session I gave together with Daniel Ben Zikry at 'SOA open day' ,Microsoft - Israel.

BizTalk Server 2006 was designed to fully encapsulate the End-Point native technology and the broker Routing engine. this was done by separating the Messaging engine from the Adapter layer and the EPM. in this strategy, the communication with the End-Point and the translation of the incoming bytes into something BizTalk can handle with - is under the responsibility of the adapter and the Message engine is route the message to the destination without worry about protocols ,specifications or programing languages.

Unfortunately all the above is not so simple, BizTalk Routing Engine is working in pub/sub pattern and the topics are Promoted Properties. the broker has no idea what to do with the incoming stream unless it came with instructions as metadata. the instructions are the Topic that all subscribers are listening to. in BizTalk routing engine, the Topic is a key/value entry in the message context that signed as "propPromoted" - a promoted property.

The component in the layer that promote those properties is the Adapter - and each adapter promote it's own properties. so you can't really be sure the message that submitted through SOAP adapter is routed in the very same way as a message submitted by MQSeries adapter - unless the routing is defined by a common promoted property. this architecture leads to tight coupling between service provider and client endpoint technology. In 'SOA' term, the service is an atomic unit of functionality that has no limit to the client protocol, this problem traditionally handled by a Logical Port that contains physical port. each physical port has one adapter to work against one endpoint technology. the topic for all subscribers was the logical port. the disadvantage with this solution is that the only way to active and consume the service is through this logical port, which means, not all other messages submitted into the MessageBox from any other logical ports. so, the real solution is to use ESB Routing context property and use them as topics for all subscriptions.

This is exactly what happened within the ESB Guidance. All routing inside the ESB are done by common ESB Routing properties. The layer between the Adapter framework and the Message Engine is the ESB Pipelines. this means that the message is submitted into the adapter and after this the message is moved to the next layer and only then - absolutely isolated from the adapter and the endpoint - the routing properties are initialized.

Dynamic Send Port - Host instance limitation

In a previous post, I wrote about the importance of choosing the right Host instances for a peacefully working 'real-world' BizTalk solution. Although this is right for all BizTalk artifacts, unfortunately there's one exception: Dynamic Send Ports.

The analogy behind Dynamic ports is a wish to provide BizTalk the flexibility of 'SOA' - a way to create an atomic service that has no intension with the End-Point protocol/technology. the Dynamic ports are the way BizTalk server is combining it's Orchestration engine with the Adapter framework layer. in runtime - the orchestration will inform the adapter engine with the information about the destination Protocol and URI and the adapter engine will ship the message to that location using the right Adapter instance.

This magic become possible thanks to the Microsoft.XLANG.BaseTypes.Address class who derived from PortPropertyBase class under Microsoft.XLANG.BaseTypes.dll.

When passing those's parameters to the port, the Adapter engine is checking the value of Address to be associated with the adapter prefixes table. for example the following will cause the FILE adapter to handle the sending process:

 

The ability to "late-bind" the Adapter instance to associate with the Send process actually forces us to disallow any Host Instance configuration at design-time - the adapter can be In-Process Host like the FILE or WCF Adapter and equally can be Isolated Host like SOAP Adapter.

The BizTalk team choose to handle the Host-Instance resolving is to use the default Host Instance configured for the Adapter. because of this most unacceptable design, use can find your process running under BizTalkApplicationHost and BizTalkApplicationIsolatedHost instances. so eventually, this means we can't use Dynamic Ports in most of our BizTalk Solutions!.

After all this, I've been surprised to find that in the newly published ESB Guidance (v1.0), the Agent who responsible of routing (Delivery agent) was implemented using a Dynamic Send Port.

Just imagine what will happen if ALL of the specific Adapter of the entire ESB are running on the same Host Instance. So, I really can't understand this design, more then that - I encourage you NOT to use the default Delivery Agent in a real ESB environment.

Let's just hope that the team will fix this behavior and there will be a way of choosing the correct Host Instance within Dynamic Send port - then we'll can enjoy the benefits of 'SOA' with BizTalk solutions.

The ESB Guidance – hello ‘SOA’ world

Introduction to BizTalk solutions architectures.

Almost all IT professionals are familiar with the benefits Service Oriented Architecture (‘SOA’) approach is offering, therefore the term ‘ESB’ is not a buzz word any more.

As a BizTalk developer the architecture you often use is known as ‘Hub-And-Spoke’. This means we’re using BizTalk as a middleware application (known as a ‘Broker’) between of all other applications. When app A needs to interact with app B, it will submit the request to the broker application. The broker then will make the preparations needed to redirect the request to app B (one example is transformation) and then route the message into app B and vice versa.

Enterprise Service Bus ('ESB') extend of the traditional EAI and MOM broker with some capabilities like:

• Web Service standards
• Integration with rang of protocols and technologies.
• metadata registry

As a rule of thumb, ESB is the gate into the ‘SOA’ world.

It means you don’t develop ‘Application’ anymore; from now on it is about ‘Services’.

If you’re new with ‘SOA’ you can learn about it here.
Also, you can find an overview of BizTalk Server based 'ESB' here.

Now, because of BizTalk ‘Hub-And-Spoke’ nature, developing ESB solution based on BizTalk Server require massive development of infrastructures, The ESB guidance is Microsoft way of achieving it in the fastest way by integrating and relaying on common Microsoft technologies.