Wednesday, September 24, 2014

BizTalk 2013: Inserting RawXML (Whole Incoming XML Message) in SQL database

In some scenarios, it is required to store the message which is received from sender along with transforming it to the destination format.  One such scenario is to store the RawXml (Whole Incoming XML Message) in database as it is.

To demonstrate this, used the Scenario : Order Message is received and we are to store the OrderId and the whole Order in database. For that I have created a TestDb, a table Order with two fields Id and RawXML.

Input Schema:

For the purpose of demo, I have kept only three fields in the input schema, and ID field is marked as distinguished as it will be used while constructing Outbound Message.

Destination Schema:

I have a table created in TestDb database in to which the Order Id of the incoming message and the Order as it is received  i.e. whole incoming Message is inserted.

To do so we need to create a destination schema for BizTalk but based on already defined table thus we would use Add Generated Items -> Consume Adapter Service option for it.

Binding : sqlBinding as the database we are dealing is SQL Server
IntialCatalog : TestDb as this is database which is to be connected and it holds the table we are dealing with
Server : “.” As database resides on same machine, if it was on other machine then it would have been that machine’s address.
Select contract type: Client (Outbound Operations) as message is going out of BizTalk
Operations: Insert as data will be inserted in the table

Click ok and you will see that three schemas and one binding file is added to the solution.

Out of the three schemas we will be using TableOperation.dbo.Order.xsd, so question might arise what about the rest two schemas? Well those are also used but implicitly by the schema TableOperation.dbo.Order.xsd.

Orchestrating the requirement:

Order Message(as per input schema) is received and passed on to Construct message, where the destination message is created and then send to InsertOrder_SQL port to do the actual insertion .

Constructing the Outbound Message:

The message construction is done inside the Message Assignment shape, with the help of two XmlDocument variable XMLDocIn and XMLDocOut.

Why XmlDocument variable? The class "XmlDocument" is a super class for all XML messages, and hence it can hold any type of XML message and subsequently any type of orchestration message.

And with XmlDocument , we can use a property called “OuterXml “ which points to the current node and its children.

So the first step is to assign the incoming message to XMLDocIn.
XMLDocIn = InMsg;

Second step is to create the outbound message. Apart from Map, Message Assignment, using .Net, the fourth way to construct message is with the help of LoadXML method – which provides a way to load the instance of schema of the message which is to be created (with or without actual values). For this demo we need to create the message which expects ID and the Whole Order as per the schema generated from SQL Table.

As we have made the ID field as distinguished it is available and we have the incoming message in XmlDocIn, with the help of OuterXml property we can get the whole Xml message.  Thus using the LoadXml message the values are loaded in the XmlDocOut variable and later assigned to OutMsg, typed message which is expected.

XMLDocOut.LoadXml("<ns0:Insert xmlns:ns0=''>"+
    "<ns1:Order xmlns:ns1=''>"+
      "<ns1:RawXML>"+"<![CDATA[" + XMLDocIn.OuterXml + "]]>"+"</ns1:RawXML>"+

OutMsg = XMLDocOut;

Question might arise why <![CDATA [ ]]> is used?  It is because while inserting into Database the parser will detect the opening tag (<) and the closing tag (>)  as a markup, so to avoid this <![CDATA[]> is used, which tells parser to treat the data enclosed within it as characters.

Sign the project, build and deploy.

Next is to configure and Test, check the part2 of this post : Configuring the Application and Testing

Download Sample - BizTalk 2013: Inserting RawXML (Whole Incoming XML Message) in SQL database Sample

Related Post:

BizTalk 2013: Inserting RawXML (Whole Incoming XML Message) in SQL database - Part 2

Configuring the Application and Testing

After Developing and  deploying the application start with creating ReceivePort with a File based receive location.

Send port which will send data to SQL, in this case you need to use WCF-SQL adapter or WCF-Custom adapter can be used.
1. PassThru pipeline is used as no processing is to be done .

2. SQL binding is to be selected as WCF-Custom adapter is used, if WCF-SQL was used then it would have already been there.

After both Receive port and Send Port are created, bind it to the Orchestration. Right click the orchestration and configure.


After done with configuration, time to test the application. Start the application and drop the sample Order File at Receive Location and you should be able to see in SQL as shown below.

Download Sample - BizTalk 2013: Inserting RawXML (Whole Incoming XML Message) in SQL database Sample

Related Post:

Monday, September 1, 2014

Begineer to BizTalk Expert : Interview with Rohit Sharma

Welcome to third Interview of the series, today's BizTalk Expert is Rohit Sharma.

Rohit Sharma is technical lead developer and application architect and has good experience in architecting, designing, developing, and supporting sophisticated applications using many different Microsoft technologies and products. Rohit Sharma is very active in BizTalk community as blogger, Wiki author/editor and forums. He has been awarded the Microsoft Most Valuable Professional (MVP) award for two consecutive years (2012 and 2013) based on his contributions to the world-wide BizTalk Server community.

Mahesh: Who are you and what you do? 
Rohit: I am Rohit Sharma born and brought up in Punjab and now settled in area near to Chandigarh. I am technology enthusiast and work with Microsoft Integration technologies. 

Mahesh: When did you start working on BizTalk? 
Rohit: The organisation where I started my career was having Microsoft as one of its clients. My journey with BizTalk Server started in 2007 when I got the chance to work with Connected System Division of Microsoft (the BizTalk Server product team). My team was responsible for handling the BizTalk Server Accelerators (HL7, RosettaNet and SWIFT). 

Mahesh: How did you mastered BizTalk (Learning path)? 
Rohit: I got chance to work during release of BizTalk Server 2006 R2 and BizTalk Server 2009 and during these releases I got the chance to learn about the different aspects of BizTalk Server.

Mahesh: Which are the major projects you handled so far? 
Rohit: There has been few being part of CSD gave me lot of exposure to Microsoft Technologies another one I would like to highlight was for a leading airline company where my team was responsible to set-up two new data centres. As a part of this engagement we got chance to did almost everything from configuration, disaster recovery, development, optimization and experimentation.

Mahesh: How do you see BizTalk compare to other integration platform? 
Rohit: Based on my experience with other platform I would say solution management is bit easier in BizTalk but it lack a web based interface but this gap has been filled by some third party products. 

Mahesh: What as per you is must to know to become an Integration (BizTalk) Expert? 
Rohit: There is no substitute to experience in integration space. By doing more and more integrations you come to know about the challenges of integrating with specific products. 

Mahesh: What are your thoughts on forums, blogs and articles etc.? 
Rohit: These are good platform to look for help and to learn from experience of others and to share your own experience. 

Mahesh: Your suggestion to a newcomers? What should be approach to get sound knowledge in Biztalk? 
Rohit: If you get chance to be part of a team where lot of good BizTalk experts are available then grab it, learning through them would be much easier. 

Mahesh: What are your thoughts around BizTalk certification? 
Rohit: As I have already mentioned there is no substitute to experience so for me certification is not that important. 

Mahesh: What is the future of BizTalk? 
Rohit: BizTalk Server is very mature product and I feel that on-premises version of BizTalk will be there as many organizations are still using legacy systems and are not comfortable in moving the key strategic systems to the cloud. 

Mahesh: Any thoughts on cloud? 
Rohit: Cloud is a game changer and is the future. It is the solution to lot of challenges being faced by organizations these days. Looking forward to see cloud based integration offering to be more mature. The on-premises offering are already providing the way to integrate with cloud. 

Thanks a lot Rohit, am sure many of us will be benefited from your insights !!!!!!

Feel free to ask questions to Rohit, in the comments.

Wednesday, August 13, 2014

ESB Portal: Unhandled Exception

After installing and configuring  ESB 2.2, I tried opening ESB portal and got below error.

As suggested in the error, checked the event log. In event log it wasn't error but warning, means nothing wrong was done instead something required was not done .. :)

To find out exact reason, checked the Exception Message and there it was , HttpModule from Service Model assembly was not available in GAC.

Why it happened: This error can occur when there are multiple versions of the .NET Framework on the computer that is running IIS, and IIS was installed after .NET Framework 4.0 or before the Service Model in Windows Communication Foundation was registered.

What to do :
1. Right-click Command Prompt, and then choose Run as administrator.
2. At the command prompt, type the following command to change to the Microsoft.NET\Framework64\v4.0.30319 folder, and then press Enter.
3. At the command prompt, type the following command, and then press Enter.
       aspnet_regiis.exe -iru
4. At the command prompt, type the following command, and then press Enter.

Wednesday, August 6, 2014

Could Not load file or assembly "Microsoft.BizTalk.Scheduler" or one of its dependencies

While working on the Scheduler POC for the article BizTalk Server : Scheduling Orchestration using Trigger Message, received below error while deploying the application using the visual studio.

Why it happened:  It happened because Microsoft.BizTalk.Scheduler  assembly which is used by Scheduled Task Adapter was missing in the GAC .

What to do :
·         Open BizTalk administration console
·         Right click on application "BizTalk Application 1"  (The shared application in which assemblies related to scheduled Task Adapter is installed)
·         Select Add | BizTalk Assemblies
·         Click the Add button
·         Browse and select C:\Program Files (x86)\Microsoft BizTalk Server 2013\Microsoft.BizTalk.Scheduler.DLL
·         Click Open
·         Check on Overwrite all
·         Check on:
·         “Add to the global assembly cache on add resource (gacutil)”
·         “Add to the global assembly cache on MSI import (gacutil)”
·         “Add to the global assembly cache on MSI file install (gacutil)”

             Click OK

The Filename, directory name, or volume label syntax is incorrect

In an application, it was required to set the file name of outgoing message with the current date.
And for that in the Orchestration, I set the FILE.ReceivedFileName context property with the current date.

And on send port used %SourceFileName% macro, thus expected was file with today’s date in the Out folder.

But while testing, no files were dropped in Out folder. This meant something was done wrong and to find it checked the Event Log, and I see there were no errors instead Warning. 

Why it happened:  As can be seen in above figure, the details of the warning states that there was something incorrect in Filename, directory name. The issue was semicolon(“:”). Semicolon is not accepted as a part of FileName.

What to do: To get rid of it and also as only date was needed, did following:

And now the FileName was set as Date only (no time, which had semicolon).

What are other characters which are not allowed?
To find this I tried with all characters and found some more which are not allowed/accepted. It’s easy or less error prone when setting file name to know what are not allowed, but it won’t be easy if value is set somewhere else.

Friday, August 1, 2014

Begineer to BizTalk Expert : Interview with Leonid Ganeline

Welcome to second Interview of the series, today's BizTalk Expert is Leonid Ganeline.
With 10+ years BizTalk Server experience Leonid is working as a BizTalk Developer, Architect, and System Integrator. He got awards: The Microsoft Most Valuable Professional [MVP] Awards 2007, 2008, 2009, 2010, 2011, and 2012 in BizTalk Server; The Microsoft MVP Award 2013 in Microsoft Integration.

Leonid is a Moderator of the BizTalk Server General forum on the Microsoft MSDN site, and author of the Microsoft TechNet articles.

Let's start with Interview....

Mahesh: Who are you and what you do? 
Leonid: I was born on Lake Laberge near Whitehorse, then my family moved to Madagascar, then Uganda, then Oman, then Paris, etc. etc. Now I have settled down into Vancouver. I was pilot, accountant, hunter, base jumper, cook, and now I am a programmer. 

Mahesh: How do you see BizTalk compare to other integration platform? 
Leonid: BizTalk is useful when you need a separate integration platform, if you can afford a separate integration team (see a Conway's low). In this regard it competes with IBM Websphere MQ/ESB and Tibco platform. 
BizTalk useful if you have a zoo of different systems. Legacy systems, monolithic systems, custom database oriented systems. Weird and old protocols and data formats as EDI, FTP, SOAP.

Mahesh: What is the future of BizTalk? 
Leonid: We see an explosion of new technologies in area of distributed systems. New ESB implementations in different language platforms as Mule ESB, Spring Integration for Java, as NServiceBus, Rebus for .NET, Zato for Python. New message brokers and queues: RabbitMQ, ActiveMQ, Azure ServiceBus. Completely different approaches as ZeroMQ, Netty. New powerful serialization technologies as JSON, MessagePack, Protocol Buffers. New communication protocols as torrent, mesh. 
So new systems are distributed, they don't need a separate integration layer because they can communicate each other out-of-box. Right? Not really. You still need to design and implement intersystem communications. But now it is a matter of generic development, commodity programmer skills. 
What about the BizTalk Server and similar systems? They would happily work for years as all legacy systems do. If it ain't broke, don't fix it. Microsoft still supports BTS on new OS, SQL and Visual Studio versions, so don't worry. 

Mahesh: What as per you is must to know to become an Integration(BizTalk) Expert? What should be approach to get sound knowledge in Biztalk? 
Leonid: Just work with it. Do your mistakes, do your stupid errors. More errors means more experience. 

Mahesh: What are your thoughts on forums,blogs and articles etc.? 
Leonid: MSDN BizTalk forum is a great place to communicate, to make sure you are not the last BizTalk developer in the world. 

Mahesh: What are your thoughts around BizTalk certification? 
Leonid: It tooks you a minute to get all certification answers in internet. But the main problem is the BizTalk is a large system and it is not possible to cover all main BizTalk aspects in one exam. Moreover, the question coverage is too far away from the real life, IMHO. For example, the BizTalk Business Rule Engine has a good coverage in the exam, but in I would hardly recommend to use BRE in production. 
So the short answer is the BTS certification is useless, sorry. 

Mahesh: Your suggestion to a newcomers? 
Leonid: Keep your hands out of BizTalk Server. Invest your time in new stuff. 

Thanks a lot Leonid, am sure many of us will be benefited from your insights and your journey is inspiring !!!!!!

Friday, July 4, 2014

Interview questions on HL7

This post intends to cover the BizTalk HL7 related questions, which a BizTalk developer can face during an interview.

1.What happens when HL7 accelerator is installed?

2. What does BTAHL72VXCommon project contains?

3. What does BTAHL7231common project contains?

4. Is it mandatory to configure party for HL7?

5.Is it required to specify schema per pepiline?

6.What consideration is to be taken while dealing with MSH entries?

7.When does DASM generates acknowledgemets?

8.Why is it required to have party configured?

9.HL7 accelerator runs in which mode?

10.Can we override values in the MSH segment of the outgoing message?

11.Can we alter the namespace in the MSH schema?

12. What is difference between BTAHL72XReceive and BTAHL7XMLReceive?

13. What are the acknowledgement types in HL7?

For answers please have look at following article:
BizTalk Developer Interview Questions and Answers - HL7

Wednesday, July 2, 2014

Begineer to BizTalk Expert : Interview with Steef-Jan Wiggers

Starting a blog series where I would be interviewing the BizTalk experts all around the world, the focus will be on getting the insights on how to become expert and what experts did to become expert. Today's BizTalk guru is Steef-Jan Wiggers. Before starting the interview I would like to congratulate Steef on being rewarded MVP award for 5th time in a row(

I met (virtually) Steef through TechNet Wiki last year and it was him, the first person who guided me proactively and helped me improvising the BizTalk articles. And I know, like me, he has guided many new contributors, playing the role of guardian very well. Before that, I was already found of his articles, which demonstrated the amount of knowledge, expertise he has in BizTalk. He is one who has been consistent for many years with the eagerness and desire to serve the community by sharing his knowledge and experience via TechNet Wiki , blogs and forums. Great to have such person in the community.

Lets start the interview....

Mahesh: Who are you and what you do?
Steef: My name is Steef-Jan, married, three children and living in Ede a village near the National Park “The Hoge Veluwe”. As an all-round integration specialist I work on different projects with various customers building integration solutions with Microsoft Technology like .NET (WCF), WF, SQL, and BizTalk Server. My role involves architecting and designing solutions, perform audits, guide installation and configurations of BizTalk environments, development and do consultancy.

Mahesh: When did you start working on BizTalk?
Steef: I started working with BizTalk Server in 2005 as junior BizTalk professional. By then I had six years of .NET development, SQL Server and OS experience. I was a certified MSCD.NET, MCDBA and MCSA 2003. It was not too difficult for me to enroll into BizTalk Server product and technology behind it. I started with installations and configurations of BizTalk 2004. Not an easy task back then. Subsequently I developed my first BizTalk solutions.

Mahesh: How did you master BizTalk (Learning path)?
Steef: The learning path can be different from person to person. I myself already had a vast experience with Visual Studio, SQL Server, XML, and XSD’s and so on. Therefore, I could focus on learning the product itself and gain experience by doing projects. I believe the best way to master BizTalk is by experience. The way you can approach this is by reading one of my TechNet Wiki articles named: BizTalk Server 2013: Beginners Guide. I did the write up so others can use that approach it that suits them.

Mahesh: Which are the major projects you handled so far?
Steef: During the last couple of years I have done various large BizTalk projects for global and local operating customers in verticals like retail, telecom, energy, and insurance. In those projects the challenging aspects are deployment and operations not the development.

Mahesh: How do you see BizTalk compare to other integration platform?                              
Steef: BizTalk is the best choice as an integration platform in a heterogeneous environment. The choice for BizTalk is platform depended. Therefore, in a predominantly Microsoft environment BizTalk is the logical choice and challenging to compare with other products. I feel Gartner does a better job of comparing integration platforms than we as BizTalk professionals.

Mahesh: What as per you is must to know to become an Integration (BizTalk) Expert?
Steef: I have a passion for technology and BizTalk in particular. I started working with the products in 2005, and after a year I decided to share my experience and knowledge through a blog. I started blogging on, where I created a blog called “SOA Thoughts, EAI Challenges”. SOA was a hot topic at that moment and Microsoft organized conferences like SOA & BPM, where BizTalk was featured as Microsoft's offering for SOA and BPM. BizTalk in my view can be a right fit as an enabler for SOA with Microsoft Technology. I visited the last three conferences held at Microsoft Campus in 2006, 2007, and 2009. With every visit I shared my experience through blog posts and with my colleagues. Soon after the first conference in 2006 I started writing articles for Dutch magazines and did presentations, sharing my gained insights to fellow BizTalk professionals.
After writing articles and blogging, I started helping people out in BizTalk forums. In June 2010 I was nominated for MVP and July 1st I was awarded. Filled with joy, excitement, and appreciation, I became even more active on forums, writing more articles, and presenting nationally and internationally for BizTalk User Groups. Being an MVP gave me a boost and more opportunities to share my knowledge and experience with the community. For instance, beginning of this year, PackT publishing gave me the opportunity to review a BizTalk book, and even the opportunity to write one. So I started writing a BizTalk Cookbook and it is due to be released the beginning next year.
The great thing about Microsoft MSDN Forums, Blogs, Code Gallery, and the TechNet Wiki is that you can share your knowledge and experience. Recently, I even started being active on Code Gallery and TechNet Wiki sharing code samples and writing Wiki articles. The general message that's coming from myself is to share your knowledge with passion, dedication, and persistence. At the end of the day you will be appreciated for your efforts, and it can lead to becoming a well-respected community member like an MVP.

Mahesh: What are your thoughts on forums, blogs and articles etc.?
Steef: Forums, blogs, articles, books, sites, and so on are all resources you can access to learn more on BizTalk Server, or help you with building a solution, or can aid in helping to solve an issue you are facing. The number of resources grew substantially on BizTalk the last couple of years. When I started with BizTalk Server there not as much of the resources as there are today. These resources play in my view a vital role in the success of BizTalk Server today.

Mahesh: Your suggestion to a newcomers? What should be the approach to have a sound knowledge in BizTalk?
Steef: I would suggest to read my TechNet Wiki articles named: BizTalk Server 2013: Beginners Guide.

Mahesh: What are your thoughts around BizTalk certification?
Steef: Certification is a means of showing your knowledge of the product towards customers. However, certification alone will not tell how skillful you are. A better indicator for that is experience, the number of projects and endorsements from peers.

Mahesh: What is the future of BizTalk?
Steef: The product, BizTalk Server is still standing strong and Microsoft will be investing in now with the upcoming release and beyond that. Besides BizTalk Microsoft is putting tremendous effort in its cloud counterpart Microsoft Azure BizTalk Service. My prediction is that in some point of time there will be a convergence between the on premise BizTalk Server and Microsoft Azure BizTalk Server. This means there will be a kind of symmetry between both of them, where you build your solution and deploy it either on premise or in Azure.

Thanks a lot Steef, am sure many of us will be benefited from your insights and your journey is inspiring !!!!!!

Wednesday, June 11, 2014

This schema has the same type name as its namespace

After installing BizTalk 2013, just to check I started to create a project with name as Test. Then added a schema named Test with Root tag named as Test and two elements. Signed the project and tried to build it and following error appeared.

To find out the problem,  I went through the properties of the schema and found that Namespace and Type name is same. Well I don’t  the exact reason but it seems having same namespace and type name causes ambiguity in resolution. 

Why it happened:  Namespace (Project Name) and Type Name (Schema name) were same
What to do : Rename the schema and Type Name Or delete the schema and create new schema with name different than that of Project Name.