Friday, May 30, 2008

Microsoft ESB Guidance - Message flow : Part I

The ESB Guidance are set of tools that extends BizTalk from regular EAI 'Hub-and-Spoke' Broker, into the 'SOA' world. One of the first thing you need to know about the guidance is that there's (almost) no magic or new stand alone feature in there - the guidance takes you beyond 'EAI' by understanding BizTalk 2006 architecture and leading BizTalk features to the wanted results.

The main components are:

  • Itinerary On/Off ramps - Extends BizTalk Messaging and BizTalk Adapter framework.
  • Core services (Agents) - Extends BizTalk Orchestration.
  • Exception management - Extends BizTalk Error report (For Orchestrations) and ACK/NACK notification (For pipelines)
  • Web services - Extends BizTalk transformation API and ExplorerOM API

In the following, slides from the session I gave together with Daniel Ben Zikry at 'SOA open day' ,Microsoft - Israel.

BizTalk Server 2006 was designed to fully encapsulate the End-Point native technology and the broker Routing engine. this was done by separating the Messaging engine from the Adapter layer and the EPM. in this strategy, the communication with the End-Point and the translation of the incoming bytes into something BizTalk can handle with - is under the responsibility of the adapter and the Message engine is route the message to the destination without worry about protocols ,specifications or programing languages.

Unfortunately all the above is not so simple, BizTalk Routing Engine is working in pub/sub pattern and the topics are Promoted Properties. the broker has no idea what to do with the incoming stream unless it came with instructions as metadata. the instructions are the Topic that all subscribers are listening to. in BizTalk routing engine, the Topic is a key/value entry in the message context that signed as "propPromoted" - a promoted property.

The component in the layer that promote those properties is the Adapter - and each adapter promote it's own properties. so you can't really be sure the message that submitted through SOAP adapter is routed in the very same way as a message submitted by MQSeries adapter - unless the routing is defined by a common promoted property. this architecture leads to tight coupling between service provider and client endpoint technology. In 'SOA' term, the service is an atomic unit of functionality that has no limit to the client protocol, this problem traditionally handled by a Logical Port that contains physical port. each physical port has one adapter to work against one endpoint technology. the topic for all subscribers was the logical port. the disadvantage with this solution is that the only way to active and consume the service is through this logical port, which means, not all other messages submitted into the MessageBox from any other logical ports. so, the real solution is to use ESB Routing context property and use them as topics for all subscriptions.

This is exactly what happened within the ESB Guidance. All routing inside the ESB are done by common ESB Routing properties. The layer between the Adapter framework and the Message Engine is the ESB Pipelines. this means that the message is submitted into the adapter and after this the message is moved to the next layer and only then - absolutely isolated from the adapter and the endpoint - the routing properties are initialized.

Sunday, May 18, 2008

xsl:for-each statement with BizTalk mapper

Honestly, I'm not a big fan of the XSL mapper editor shipped with BizTalk extensions for VS2005. likewise with the xsl and C# functions visual representation called functoids. Personally, I find writing the XSL code to transform between one XML to another as easy approach and more easy tuning then fighting with the editor until it generates the appropriate code.

One common scenario is generating xsl:for-each loop in the XSL stylesheet. I stopped counting the times people asked me how to to obtain this with the mapper. The task is easy when you write the xsl code by yourself but much more complicated when you're trying to generate it with the mapper.

  • unbounded element to another unbounded element.

The basic scenario is transforming from one repeating element to another. consider the following:

Source Schema:

 

Destination Schema:

XSL Code:

BizTalk Mapper:

As you can see, to create the above XSL, all you have to do is to drug the source elements to the destination. this is because both source and destination elements are unbounded, so the mapper 'understand' and generating xsl:for-each statement.

  • Unbounded element with maxOccurs="1" child and unbounded child:

consider the following:

Source Schema:

Destination Schema:

Input XML:

<ns0:Root xmlns:ns0="http://Pinhas_Kolsky.Demos.Mapping.Looping.Schema1">

  <Record1>

    <Record>

      <Record2>

        <TargetField1>1</TargetField1>

        <TargetField1>2</TargetField1>

      </Record2>

      <Record2>

        <TargetField1>3</TargetField1>

        <TargetField1>4</TargetField1>

      </Record2>

    </Record>

  </Record1>

  <Record1>

    <Record>

      <Record2>

        <TargetField1>5</TargetField1>

        <TargetField1>6</TargetField1>

      </Record2>

    </Record>

  </Record1>

</ns0:Root>

in this case, you cannot drug the elements like in the basic scenario:

BizTalk Mapper:

 

BizTalk mapper can 'guess' how to build the for-each statement only in the basic scenario, in the case above, the mapper will generate the first child of the unbounded destination element more then once:

XSL Code:

Output XML (Not valid):

<ns0:Root xmlns:ns0="http://Pinhas_Kolsky.Demos.Mapping.Looping.Schema2">

  <LoopRecord1>

    <Record>

      <LoopRecord2>

        <DestField1>1</DestField1>

        <DestField2>2</DestField2>

      </LoopRecord2>

      <LoopRecord2>

        <DestField1>3</DestField1>

        <DestField2>4</DestField2>

      </LoopRecord2>

    </Record>

    <Record>

      <LoopRecord2>

        <DestField1>5</DestField1>

        <DestField2>6</DestField2>

      </LoopRecord2>

    </Record>

  </LoopRecord1>

</ns0:Root>

As you can see, the element 'Record' was created twice.

In this case you must use the looping functiod:

BizTalk Mapper:

XSL Code:

Output XML (valid):

<ns0:Root xmlns:ns0="http://Pinhas_Kolsky.Demos.Mapping.Looping.Schema2">

  <LoopRecord1>

    <Record>

      <LoopRecord2>

        <DestField1>1</DestField1>

        <DestField2>2</DestField2>

      </LoopRecord2>

      <LoopRecord2>

        <DestField1>3</DestField1>

        <DestField2>4</DestField2>

      </LoopRecord2>

    </Record>

  </LoopRecord1>

  <LoopRecord1>

    <Record>

      <LoopRecord2>

        <DestField1>5</DestField1>

        <DestField2>6</DestField2>

      </LoopRecord2>

    </Record>

  </LoopRecord1>

</ns0:Root>

It's impotent to notice that when the mapper is not automatically creates the XSL by drugging the elements, you can't use both - looping functiod for the parent and drugging unbounded to unbounded. this will generate only the first xsl:for-each. you can see this in the following:

BizTalk Mapper:

  

XSL Code:

Output XML (Not valid):

<ns0:Root xmlns:ns0="http://Pinhas_Kolsky.Demos.Mapping.Looping.Schema2">

  <LoopRecord1>

    <Record>

      <LoopRecord2>

        <DestField1>1</DestField1>

        <DestField2>2</DestField2>

      </LoopRecord2>

    </Record>

  </LoopRecord1>

  <LoopRecord1>

    <Record>

      <LoopRecord2>

        <DestField1>5</DestField1>

        <DestField2>6</DestField2>

      </LoopRecord2>

    </Record>

  </LoopRecord1>

</ns0:Root>

Conclusions: generating xsl:for-each statement with BizTalk mapper:

  • In a simple scenario you can generate xsl:for-each by drugging the source unbounded element to the destination unbounded element.
  • When you need to add more logic to the code (like creating maxOccurs="1" element inside the looping) you must use looping functoid.
  • In a case when a looping functoid is necessary, you must use one looping functoid for each unbounded element.  

Thursday, May 1, 2008

BizTalk project namespace and schema-type namespace duplication issue

When adding web reference to a C# project in Visual Studio, VS generates a class named Reference.cs. this class is the C# code of the XSD types of the web service. BizTalk installation includes extensions to visual studio that additionally creates classes for BizTalk artifacts that will be used by the BizTalk messaging engine.

When you create XSD schema - VS will generate cs file with the schema as a CLR object. the naming convention for this file is <schemaName>.xsd.cs 

When you add a Web reference to the project - VS will create three classes:

  1. Reference.map.cs
  2. Reference.odx.cs
  3. Reference.xsd.cs

In a future post I'll write about those classes in great details.

The namespace and the CLR class name of the auto-generated classes is depends on some factors:

For web reference:

  • The class namespace is the project default namespace (by default it's Visual Studio Project filename ) + dot sign (.) +  the name of the web reference.
  • The class name is a factor of the WSDL artifact.

For XSD schema:

  • The class namespace is the schema-type namespace
  • The class name is the schema-type name.

This naming convention is auto-generated and you can't control or change it.

Another thing to know is that for the Compiler perspective, the default namespace for all artifacts in the project is default namespace (by default it's the VS project filename).

All the above leads us to this:

What if the default namespace (the project filename) is equals to the XSD schema-type namespace?

In a regular situation it's all right but in case you adding web reference to the project, VS will generate the namespace for the Reference.odx.cs classes in the form of <ProjName>.<WebRefName> so if the name of the project file is Test.Schemas it will look like: Test.Schemas.<nameOfWebRef>

In the sample below Test.Schemas is a namespace.

When VS generates the XSD schema class it will generate it in the form of <schemaNS>.<schemaName>

in the sample below Test.Schemas is the class namespace and Teat is the class name; explicit: Test.Schemas.Test

When the compiler tries to compile the project it will translate the word 'Test' in namespace of the web reference file Reference.odx.cs as the class that represents the XSD schema-type, because the schema has the same namespace as the project default namespace – by default the VS project name.

 

This process will cause the following error:

error CS0426: The type name '<Name>' does not exist in the type.

The error means that the XSD generated class 'Test' does not contains member with the name 'Schemas'. this is completely true because 'Test.Schemas' is only a namespace – copied from the project namespace.

To fix the error, all we have to do is to change the schema name or the schema namespace. additionally, we can change the project default-namespace and then import the web reference.

Be aware that to change the web reference namespace you need to remove them and adding them again.

The insights of all of this: if your project default namespace is composed of two words separated by a dot (.) sign:

  • Unless you have a web reference in your BizTalk project – you can name the schema namespace just like the VS project file including 'dot' sign, you can also name the schema-type name as the first word in the Project filename.
  • If you need to include a web reference in the project change the schema-type namespace to be different from the project namespace. additionally; you can name the schema-type name different from the first word in the project namespace.

 
Copyright © 2007 | Diseñado por Blog and Web