Tuesday, July 16, 2013

Debatching(Splitting) XML Message in Orchestration using DefaultPipeline - BizTalk 2010

In this post we will walk through the process of debatching an xml message in Orchestration using pipeline in Biztalk.

I have used the Default XML Receive pipeline to achieve it, but it can also be done by creating a custom pipeline which uses XML disassembler (where you can set the Envelope and Document Schema).

Pipeline is not available in Orchestration like other shapes, thus to use it we need to add reference to following assemblies(which will allow us to use methods in those assemblies) :

  • Microsoft.XLANGs.Pipeline.dll
  • Microsoft.BizTalk.Pipeline.dll

You can browse to the location BizTalk Installation Directory to find above dll's.

Most of the part of this post is borrowed from my earlier post which talks about debatching xml at receive port:
http://tech-findings.blogspot.in/2013/07/debatchingsplitting-xml-message-biztalk.html

Scenario:


We receive many item information but its wrapped (Enveloped) , so to process each item we need to unwrap it (remove the envelope and split individual Item message).

Below is what we receive (Input) :
<ns0:Items xmlns:ns0="http://TestingSchemas.ItemEnvelope">
  <ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_0</ID> 
  <Name>Name_0</Name> 
  <Quantity>100</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
  <ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_0</ID> 
  <Name>Name_1</Name> 
  <Quantity>200</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
  <ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_2</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_3</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_4</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_5</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_6</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_7</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_8</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
<ns1:Product xmlns:ns1="http://TestingSchemas.Item">
  <ID>ID_9</ID> 
  <Name>Name_2</Name> 
  <Quantity>300</Quantity> 
  <UnitPrice>100</UnitPrice> 
  </ns1:Product>
  </ns0:Items>

But we want (Output) :


      <ns0:Product xmlns:ns0="http://TestingSchemas.Item">
           <ID>ID_0</ID>
          <Name>Name_0</Name>
          <Quantity>100</Quantity>
          <UnitPrice>100</UnitPrice>
      </ns0:Product>
             .
             .
             .
             .
             .

     <ns0:Product xmlns:ns0="http://TestingSchemas.Item">
        <ID>ID_9</ID>
        <Name>Name_9</Name>
        <Quantity>100</Quantity>
        <UnitPrice>100</UnitPrice>
     </ns0:Product>


All right, let's see how we do it:

1. Create schema - document schema 

2. Create the wrapper - Envelope Schema
  • Name the root node, I have named it Items.




  • Click on "Schema" and go to Properties Window
  • Set the property "Envelope" as Yes, as this schema will be used as envelope and this is the property which helps disassembler to recognize it.
  • So far good, envelope schema is ready but what about the document which will be wrapped.
  • In the Property window select "Imports" and click on the ellipsis 
  • You will get Imports wizard pop-ed  out  .
  • Click on add and select the document schema which we created in step 1.(Here it is Item schema)


  • Click on the "Items" and go to property window and select the property Body XPath and set it to /*[local-name()='Items' and namespace-uri()='http://TestingSchemas.ItemEnvelope']. Doing so allows an Items node to contain any number of Item Document.
  • Cool... Now we are ready with the resources, next is to validate schema, build it, sign it  and deploy.


3. Now lets create Orchestration which will receive the enveloped item and debatch/split it using the pipeline.


  •         Receive shape is configured to receive untyped message i.e. of type System.Xml.XmlDocument (which will receive enveloped message -ItemEnvelope Type) .
  •         Then we have a "DebatchScope"  which is of type "Atomic" and Orchestration  is of type Long running. Why Atomic scope? Because we will be calling/executing Pipeline within it and the pipeline is of non- serializable type.
  •         Next we have Expression shape named as "Execute Pipeline". It is here where we make call to pipeline  and execute it which results in splitted messages.
  •         It contains following line:  GetPipelineOutput=Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(Microsoft.BizTalk.DefaultPipelines.XMLReceive),ItemsIn);
  •         GetPipelineOutput is a DebatchScope variable of type: Microsoft.XLANGS.Pipeline.ReceivePipelineOutputMessages
  •       So we are executing the method ExecuteReceivePipeline()  method of XLANGPipelineManager class  which belongs to Microsoft.XLANGs.Pipeline and its output assigned to  GetPipelineOutput.

  •      Then we have a loop shape "UntilLastMessage" and its same as  while loop, below is the condition(till spillited messages are available) : GetPipelineOutput.MoveNext()
  •         Next is Construct shape with Message Assignment within it, which has following code:
 ItemOut = null;
          GetPipelineOutput.GetCurrent(ItemOut);

It is here where the splitted single  message is assigned to ItemOut , where ItemOut is a Message variable which is of type Item.xsd
  •        At last we have Send shape which accepts message of type ItemOut and sends it .
  •        Now what, build the project , sign it and deploy .
4.Now create a Receive and Send port :

  •        Receive Pipeline is PassThruReceive  as we don't want its message type to be detected until the message reaches to ExecutePipeline shape in Orchestration.

  •        Now after both the ports are ready, it's time to bind it to the logical ports of our orchestration:

5. Start the application and test it, I will drop a envelope message at receive location which we saw at start  . So I should be getting 10 individual xml message at the destination location:




        Will keep on posting as an when I find something to share!!!!!!!!!!!!

13 comments:

  1. Hi Mahesh, when i try your example, i get an error... "Use of unconstructed message ItemsOut" when i have declared in the construct message shape the ItemsOut... what can be the issue.. please comment

    ReplyDelete
    Replies
    1. Hi,

      Did you create message "ItemsOut" ?
      And check whether you have done following in MessageAssignment shape:
      ItemsOut = null;
      GetPipelineOutput.GetCurrent(ItemsOut);

      Also check if all the shapes are in Atomic scope.

      Delete
  2. Yes, i created it... I got it resolved now... the issue was, accidentally i had the send shape out of the loop. it working... thanks for your response

    ReplyDelete
    Replies
    1. Hi,

      Even after doing all the above mentioned, am getting error as"Use of unconstructed message ItemsOut" can you please let me know what could be the issue

      Delete
  3. hi
    I am new in biztalk. I follow you step and try to import schema but after I import in envelope schema I can not see imported schema like it is show on you picture. Is there something what have to done to see imported schema?

    dan

    ReplyDelete
  4. Wonderful article.
    It woks fine for me.
    may be a missing detailed explanation (Under items we need to create another record of type any to hold the debached elements) but it should be known to every one

    ReplyDelete
  5. Hi Mahesh,
    Thanks for the such a nice post.But I have one doubt : when I set the receive shape with envelope msg instead of untype msg then I got a error in admin console :
    The published message could not be routed because no subscribers were found. This error occurs if the subscribing orchestration or send port has not been enlisted, or if some of the message properties necessary for subscription evaluation have not been promoted.
    I am confused why it is happening ?

    ReplyDelete
  6. Hi Mahesh,
    Thanks for the such a nice post.But I have one doubt : when I set the receive shape with envelope msg instead of untype msg then I got a error in admin console :
    The published message could not be routed because no subscribers were found. This error occurs if the subscribing orchestration or send port has not been enlisted, or if some of the message properties necessary for subscription evaluation have not been promoted.
    I am confused why it is happening ?

    ReplyDelete
  7. Hi Prakash,

    Please check the pipeline in receive port. It should be of type XMLReceive.

    Thanks

    ReplyDelete
  8. My Message is not being de-batched

    ReplyDelete
  9. hello Dear,while sending the message to message box (direct binding in send port),i am getting infinite message means infinite loop occuring .Do u know the reason y?

    ReplyDelete
  10. Hi Piyush,

    This is very common scenario -- it happens when you are subscribing to same message which you are sending.
    Say, you are sending msgXYZ to MessageBox and your orchestration first receive shape is also looking msgXYZ. So you send a message and it instantiate another instance,thus this process goes on.

    ReplyDelete