Wednesday, September 2, 2015

Begineer to BizTalk Expert : Interview with Daniel Toomey

Welcome to Fourteenth interview of the series, today's expert is Daniel Toomey. 

Dan is an Enterprise Integration Consultant, Solution Architect, Application Developer, and Systems Analyst with over thirteen years experience working on large enterprise systems in the private sector, Queensland Government and Australian Federal Government. Microsoft certified in BizTalk Server, WCF and Azure, Dan specializes in large integration projects, Business Process Integration and Connected Systems built upon the Microsoft solution stack. While his primary technical focus is on Microsoft BizTalk Server and Microsoft Azure BizTalk Services, he is also highly experienced with using Azure Service Bus, WCF, Web Services, and C#.NET.

Prompted by Microsoft, Dan founded the Brisbane BizTalk User Group in 2005 and has presented at many of its sessions as well as for the Brisbane Azure User Group and Barcamp QLD in Brisbane and on the Gold Coast. He has also posted a number of “how-to” webcasts on the Brisbane BizTalk User Group YouTube channel, and numerous presentations on SlideShare.
Dan is also Co-Organiser of the Brisbane Azure User Group, a Microsoft Certified Trainer, a member of the Microsoft Azure Advisors group, and a published Pluralsight author.

His blog can be found at

Let's begin the interview....

Mahesh: Who are you and what you do?
I’m a Senior Consultant for Mexia, Australia’s best enterprise integration consultancy. We focus on delivering top solutions for both cloud and on-prem based integration using the Microsoft stack. During my six years with Mexia I have worked mainly with BizTalk Server, but more recently I’ve also designed and built hybrid solutions using Microsoft Azure BizTalk Services, Service Bus, Logic Apps and Hybrid Connections.  I started the Brisbane BizTalk User Group in 2005, and also lead the Brisbane Azure User Group. In addition, I am a blogger ( and a Pluralsight author.
Before I started my career in IT in 2001, I was a professional musician. Born in New York, I studied trombone and piano, graduating The Juilliard School with a Master of Music degree. Whilst serving as Principal Trombonist with the Singapore Symphony Orchestra, I met my wife Sherryn who is from Australia. After getting married and enduring four years of service in the US Air Force Band of Liberty, we moved to Australia where I earned a second Master’s Degree in IT from Bond University.

Mahesh: When did you start working on BizTalk?
My first BizTalk project was for the Queensland Dept. of Justice in 2004, where we built a cross-agency electronic interface that allowed the Queensland Police Dept. to send electronic Bench Charge Sheets directly into the Courts management system. Shortly after that project, I was recruited for BizTalk contracting roles within other government agencies for a few years before becoming a founding member of Mexia in 2009.
Mahesh: How did you mastered BizTalk (Learning path, amount of time)? Dan: I’m still learning!! BizTalk is an enormous platform, I don’t know if anyone can claim to know everything about it. But I’d have to say aside from a 5-day training course, several textbooks, reading blogs & forums, and lots of experience, the single biggest facilitator to my skills development was mentoring by MVPs like Bill Chesnut (“BizTalk Bill”), who I’m delighted to say is a fellow teammate at Mexia.

Mahesh: Which are the major projects you handled so far?
Too many to name here. But have a look at Mexia’s website ( to see our impressive list of clients.

Mahesh: How do you see BizTalk compare to other integration platform?
To be honest, I’ve never worked with other integration platforms such as Tibco, MuleSoft, IBM, Oracle, etc.   But every time I talk to people who have, I hear about so much grief. My impression is that there is no other platform that delivers so much capability out of the box for the same price.

Mahesh: What as per you is must to know to become an Integration(BizTalk) Expert?
The first thing to realise is that Integration != BizTalk. Integration covers a wide spectrum; it is a means to solve a business problem, and there are many tools out there to help. BizTalk Server is one of the biggest tools, but it is not necessarily the right tool for every problem. Learn the concepts of integration first (e.g. message exchange patterns), then analyse the requirements before deciding what tool to use.

Mahesh: What are your thoughts on forums,blogs and articles etc.?
These are invaluable not only for newbies but as a continual reference for veterans. There is an outstanding community of MVPs and other seasoned professionals who are openly sharing their hard-learned knowledge with the community. I believe this is an advantage that Microsoft has over other integration providers – the strong community presence. For example, check out the Integration Monday events ( – there are so many outstanding professionals around the globe with something to say that this initiative can support weekly events, often over-subscribed!  I certainly would not be where I am today without having consumed the resources freely provided by giants in this industry like Saravana Kumar, Scott Colestock, Charles Young, Sandro Pereira, Michael Stephenson, and many others.

Mahesh: Your suggestion to a newcomers? What should be approach to get sound knowledge in Biztalk?
Learn the basic integration concepts first (e.g. “Enterprise Integration Patterns” by Hohpe), then the tool will make more sense to you. I see a lot of C# developers that start using BizTalk without making the necessary adjustment in approach. There are lots of excellent books and Pluralsight courses out there, but nothing beats experience. Try to become part of a team of more experienced people than yourself and learn from them; even better find a good mentor. In my case, Bill Chesnut got me started with BizTalk and is still the first person I go to with questions I can’t find the answer to myself.

Mahesh: There are many tools from community which support BizTalk in some or the other way(like BTDF, Bizunit etc), what do you say about it? Which ones you would recommend?Why?
We use the BizTalk Deployment Framework (BTDF) quite heavily now in our projects. Integration deployment is a complicated and often painful endeavour, even with BTDF – but it does make it less painful. BizUnit is another great tool for testing, especially when combined with SpecFlow which allows the test cases to be expressed in business language (Gherkin). I’ve found the Winterdom Pipeline Component framework to be very handy as well.  And what I’ve seen of NoS looks very impressive – although I’m embarrassed to say I haven’t actually tried using it myself yet. Finally, I must mention BizTalk360 as an excellent tool for managing BizTalk Server platforms; we have several clients for whom we’ve installed this product.

Mahesh: What are your thoughts around BizTalk certification?
I think that certification is one of those boxes you need to tick if you want to validate yourself to some employers. But these days there are so many cheat exams out there that I really wonder if there is any credibility left in this as a measurement of capability(?)  I think you can tell much more about a prospective candidate’s skills by asking a few relevant questions in an interview.

Mahesh: What is the future of BizTalk?
I’ll assume you mean specifically BizTalk Server here(?) Many people have the impression that it is eventually going away and will be replaced by cloud technologies. But the reality is there are always on-prem systems that need to be integrated, and nothing developed for the cloud matches the power of the server product…yet. I think BizTalk Server will still be around for a long time, however each version “next” is going to introduce more & more capability for cloud integration. Hybrid integration is where the real power is, and it is clear that Microsoft is pushing to make that more and more accessible.

Mahesh: Any thoughts on cloud?
It is essential that integration developers today learn the new technologies such as API Management, Azure App Service, Service Bus, Event Hubs, etc. because these are continually becoming more important via their ability to facilitate rapid, cost-effective implementation of solutions that scale. Integration devs who insist on staying focussed purely on the BizTalk Server product will eventually find themselves eclipsed by those who can leverage these cloud platforms to deliver better solutions.

Mahesh: What motivates you to do the community work?
As mentioned earlier, I would never have grown to my current level of expertise without the benefit of knowledge shared by so many industry giants through their blogs, forums, and community presentations. After being a “sponge” for the better part of my career in consuming all of these resources, it gives me immense pleasure and fulfilment to finally be able to give some of this back to the community via my own contributions. Each time I publish a new blog post, deliver a new Pluralsight course, or present something to the user group, I relish the idea that there’s someone out there just like me six years ago, who’s life is going to be just a little bit easier that day because of the titbit of knowledge I’ve shared.

Mahesh: Do you think the current Azure offferings are stable/matured enough for an organisation to move from On-Prem to Cloud?
I think they are getting there. It’s all so new, so there are many hurdles to overcome. But the great advantage of the cloud is the rapid pace at which things improve – something that server products can’t compete with. My advice to any organisation seeking to leverage Azure is to ensure you enlist the assistance of strong Microsoft Partners, because they will have the necessary access to the product team which is essential to gaining proper support.
As for us integration professionals, it’s our duty to master the new tech so we can guide our clients appropriately in wielding it. Microsoft has opened its feedback channels more widely than ever before, so we are in a great position to influence the direction that the product team takes in solving real world problems. It’s an exciting time to be in the integration space!

Thanks a lot Daniel for sharing your experiences, this will surely benefit many !!!

Feel Free to ask questions to Daniel in the comments!!!!!!!!

Related Post:

Tuesday, August 4, 2015

The cause for this error may be that the message part data has not been initialized

In an aplication am working upon now, I had to create a message from a response received and one of the field to be created  had to be populated from a variable.
It's simple,
1.Mark the filed as Distinguished field
2.Drop a construct shape with a Transform Shape and followed by Message Assignment shape
3. Create a map which populates all the fields of the message to be created from the response received  except the one which needs to be assigned value from variable
4. In message assignment shape, msgToBeCreated.DistinguishedField = variable;
But, I got following error

Why It Happened

As can be seen in below image, the distinguished field has minOccurs = "0" i.e it is optional field. In the map it is not mapped so it does not become available in the resultant message, and when I try to assign value to this field (which is not present) I get error, makes sense :) .


What to Do

The error is quite straight forward, the distinguished field node is not present in the Message created after the transform shape.
Inner exception: A failure occurred while evaluating the distinguished field xxxx against the message part data. The message part data does not contain at least one of the nodes specified by the XPath expression (listed below) that corresponds to the distinguished field. The cause for this error may be that the message part data has not been initialized or that the message part data does not conform to the message
So to make the node available, in the map I added a String Concantenate functoid with empty value so that it gets intialized and later assigned the actual value in Message assignment shape.

Saturday, July 18, 2015

When was BizTalk Application Last Modified

Few days ago, there was a situation where we had to confirm that the BizTalk application running in production env is the latest, i.e. against the latest checked in on TFS.

At first it was quite natural to check the admin console to find out the last modified date and time of the assembly, so checked the Resources section of the application of interest.


That was not helpful, as can be seen in above image because only five information are displayed Name, Type ,Source Location , Destination Location and Application. There was no information about last updated time.

So how to find when was assembly last modified? 

Whenever you deploy BizTalk application/assemblies, entries are made in two places BizTalkMgtDb(BizTalk Management Database) and GAC(Global assembly cache).

But GAC too, does not have any info about when assembly was added - as can be seen in below image.

BizTalk management database contains all artifacts that are part of a BizTalk solution - Schemas, Maps, Orchestrations,Ports. Assemblies and applications also go into this database. Thus here you get all the information, below are steps to find last updated date of an an application.

1. Login to SQL Management Studio
2. Databases--> BizTalkMgmtDb--> Tables

3. Select dbo.bts_application table --> right click --> Select Top 1000 rows (you can even query against this table -- select * from dbo.bts_application)

An application can comprise of one or more assemblies, so to check about assemblies query dbo.bts_assembly

Actually the Admin console fetches information from this table and displays it, but only five and not all, that's how it is being designed (In my opinion Microsoft should consider including one more column in Admin console for Updated date and time).

If you wish to go for more granular details like when the specific schema/map/orchestration was last updated then you can check following tables

1. dbo.bt_DocumentSpec      

2. dbo.bt_MapSpec 

3. dbo.bts_orchestration      

Note: The modifieddate does change after every restart of that particular Orchestration.


Wednesday, July 8, 2015

There is an error in the XML document : InvalidOperationException

In current application I have requirement to choose the flow based on a field's value. As it can't be distinguished I used Xpath. It's simple, to fetch value following is what needs to be done:

var = xpath(messageOnWhichXpathIstoBeapplied, "xpathQuery");
I did same, but got following error:
xlang/s engine event log entry: Uncaught exception (see the 'inner exception' below) has suspended an instance of service 'Palette.E1.CreateInvoiceE1(0b616f5a-048d-e8d5-b014-91cde67160f9)'.

The service instance will remain suspended until administratively resumed or terminated.

If resumed the instance will continue from its last persisted state and may re-throw the same unexpected exception.

InstanceId: ba30a5f2-c173-4eff-8d31-e1cb9dc8c3b3

Shape name: Get_Status

ShapeId: 7b7bd44c-bf3f-4cd8-ba61-232c818d8271

Exception thrown from: segment 1, progress 6

Inner exception: There is an error in the XML document.

Exception type: InvalidOperationException

Source: System.Xml

Target Site: System.Object Deserialize(System.Xml.XmlReader, System.String, System.Xml.Serialization.XmlDeserializationEvents)

The following is a stack trace that identifies the location where the exception occured

at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)

at Microsoft.XLANGs.Core.Part.XPathLoad(Part sourcePart, String xpath, Type dstType)

at Palette.E1.CreateInvoiceE1.segment1(StopConditions stopOn)

at Microsoft.XLANGs.Core.SegmentScheduler.RunASegment(Segment s, StopConditions stopCond, Exception& exp)

Additional error information:

<Status xmlns=''> was not expected.

Exception type: InvalidOperationException

Source: System.Xml

Target Site: System.Object Read_int()

The following is a stack trace that identifies the location where the exception occured

at System.Xml.Serialization.XmlSerializationPrimitiveReader.Read_int()

at System.Xml.Serialization.XmlSerializer.DeserializePrimitive(XmlReader xmlReader, XmlDeserializationEvents events)

at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)


Why it Happened

As can be seen in image below, I am trying to assign value of Status field to varStatus variable(of type Int32), looks correct but this is not. Xpath is used to select nodes or node sets from an Xml document, thus if I am using xmlVariable to get the value from below xpath query it would have been correct. Xpath queries do not implicitly do type conversion, thus the error.


What to Do

In order to fetch values other than XML, you need to explicitly specify the Xpath about the type conversion and for that we need to use xpath functions depending upon requirement.
For Number it is number(xpathquery), for String value it is string(xpathquery) , for Boolean value it is boolen(xpathquery) etc.
Thus to get rid of error, I used number xpath function to fetch value from status field, and assigned to varStatus variable of type Int32.

Wednesday, July 1, 2015

Begineer to BizTalk Expert : Interview with Mohamad Halabi

Welcome to thirteenth interview of the series, today's expert is Mohamad Halabi.

He is six time MVP award receiver and  work as a Solution Architect in the e-Government secto and his main responsibilities are:
-Designing an Enterprise Service bus using BizTalk Server, WCF, WF, ADFS, and WIF
-Designing and automating the enterprise Application lifecycle management through TFS
-Managing IT Governance through COBIT and ITIL

He is also a speaker in Microsoft events such as GDC, TechEd ME, and Open Door and member of Devlifestyle Microsoft User Group.

Let's begin the interview....

Mahesh: Who are you and what you do?
I am Mohamad Halabi, a Microsoft Integration MVP. I am from Lebanon. I work as a Chief Architect in the government sector. My work revolves around Enterprise Architecture - from Business Architecture to IT architecture. It might seem that my work is more at a higher level than implementation, but I'm a firm believer than an (practical) architect should guide projects from architectural design all the way to implementation and technology, and that's what I do.

Mahesh: When did you start working on BizTalk?
I started with BizTalk Server back in 2006. It was a self learning journey driven by big interest in integration. I knew back then that integration was the path I want to take. After I learned BizTalk, I was lucky that my company - who till then was mainly working on web technologies - got it's first BizTalk project. I immediately took the chance and asked to be handed the project. I got my wish and I started from there.

Mahesh: How did you mastered BizTalk (Learning path, amount of time)?
As I said, it started as a self learning journey. I purchased a book, read it, and started a project. Of course as you know, studying BizTalk and actually implementing it are two totally different things. I starting learning as I implemented - and still do till this very day. It's hard to specify a time, because as I said, I still learn every day - but that's the fun part!

Mahesh: Which are the major projects you handled so far?
I'd say the major projects (12 months +) are 4 mainly in the retail and government sectors. There are other smaller to medium projects ranging from a couple of process to re-engineering existing implementations.

Mahesh: How do you see BizTalk compare to other integration platform?
Well I have not used other integration platforms although I've seen the latest IBM Integration Bus (IIB) in action. Therefore I cannot do an objective comparison, but I have to say that BizTalk has always served me well in all scenarios. BizTalk - with the ESB toolkit - has covered every need whenever the scenario calls for the ESB architectural style.

Mahesh: What as per you is must to know to become an Integration(BizTalk) Expert?
I think this is a very important question, because I've met many people who think they would be integration experts just by learning the tool - which I believe is incorrect. As an integration architect you are required to know a lot of standards, protocols, and technologies. For example HTTP, XML, XPath, WS-*, REST, WCF, MEPs (just to name few) are just some of the things an integration architect must know. Learning BizTalk architecture is of course essential, but mastering the tool alone is not enough: after all BizTalk is intended to be a middleware 'sitting' in the middle of different platforms, protocols, and styles; if you do not know how these things work, then knowing the tool alone won't help much.

Mahesh: What are your thoughts on forums,blogs and articles etc.?
The wealth of existing online resources is a great asset for any aspiring BizTalk developer/administrator. I think that BizTalk has such a rich online community that it rarely happens that a problem has no solution posted somewhere.

Mahesh: Your suggestion to a newcomers? What should be approach to get sound knowledge in Biztalk?
Well why not start with a little bit of marketing for myself :) I'd tell them go check my Pluralsight BizTalk course. Of course there are a lot of great books about BizTalk - all are great learning resources. But I repeat what I said before: the learning path should cover a large scope of technologies, protocols, standards, addition to BizTalk as a tool.

Mahesh: There are many tools from community which support BizTalk in some or the other way(like BTDF, Bizunit etc), what do you say about it? Which ones you would recommend?Why?
I've used various tools along the way, such as the BizTalk Deployment Framework, BizTalk Terminator, and BizTalk MsgBox Viewer. In general I always keep an eye on the community for what it's producing and I advise every BizTalk dev/admin to do the same. Reusability is critical, and if someone has put time and effort to serve the community, others must take advantage.

Mahesh: What are your thoughts around BizTalk certification?
They're nice to have :) but as someone who interviews people regularly, frankly certifications are not an indication for knowledge anymore. For various reasons it became easy to gain certifications. In addition, in my opinion the team behind certifications at Microsoft must work to make the exam questions more like real-world scenarios, because right now they are not.

Mahesh: What is the future of BizTalk?
Well with Microsoft recently ending support for Windows Server AppFabric - which for a while looked like BizTalk's alternative (regardless whether that was actually their plan) - it seems BizTalk is here to stay for on-premise integration. That being said, the future path is cloud-based, but do not expect enterprises with millions of dollars worth of investment to just dump everything and move on. On-premise integrations will continue and with them - I believe - BizTalk will continue to be strong. Meanwhile, Azure BizTalk Services is there to help with hybrid (on-premise / cloud) applications which will allow organizations to gradually switch to the cloud. That being said, BizTalk Services is still in its early stages and surely Microsoft will invest more on it.

Mahesh: Any thoughts on cloud?
It's clear that the cloud is major shift. It's the perfect example of how technology provides opportunists to business. The advantages are there for everyone to see; however, as I said, the movement takes time and existing on-premise investments will continue to function. In addition many government agencies will not move easily - if they ever do.

Mahesh: What motivates you to do the community work?
Networking! In my career I probably learned from people more than I did from books. There is no better way to connect with people of different expertise and backgrounds than the community.

Mahesh: Being MVP, do you feel that responsibilities get added? What is your thought on MVP?
Well I don't see the MVP award as a responsibility, but rather as an appreciation that I must live up to. I frankly never worked to be an MVP, but when I was rewarded I cannot hide the fact that I was very proud to be part of this community. Besides all the technical benefits, the MVP community is the perfect place to connect with people of different backgrounds and expertise - which as I said before, is the main reason I do community work at the first place.
Thanks a lot Mohamad for sharing your experiences, this will surely benefit many !!!

Feel Free to ask questions to Mohamad in the comments!!!!!!!!

Related Post:

Monday, June 8, 2015

Primary Receive Location cannot be deleted

In an application, I had created two receive locations one FILE based and the other FTP based.
The FILE based location was to be removed (as it was not required now), so it was to be deleted and while deleting it, got below error:


Why it happened:

The FILE based receive location was created first and then FTP based receive location. So whenever you have a Receive port having more than one receive location then the receive location which was created first is automatically marked as the primary location. If there is only one then that is the primary location.
As can be seen above, FILE based receive location has been marked as Make this the primary location and checkbox is disabled for change.
FTP based receive location has not been marked as primary location and checkbox is enabled to do change if required.

What to do:

There are two ways to delete primary location
1. Delete the Receive port, create new with only FTP based receive location (so this will be the primary location)  ---> This is ok in this scenario where I have only two receive location but if there are more then not at all good solution.

2. Mark the other Receive location  as Make this the primary location, doing this will make the earlier receive location as not primary - which makes it available for deletion

So I went ahead with second way and mark the FTP based receive location as primary location
After doing this the FILE based receive location got it's Make this the primary location unchecked and become available for deletion. And I was able to delete it.


Monday, June 1, 2015

Begineer to BizTalk Expert : Interview with Richard Seroter

Welcome to twelfth interview of the series, a year ago started this series for getting the insights on how to become expert and what experts did to become expert, first interview was Begineer to BizTalk Expert : Interview with Steef-Jan Wiggers and today's expert is Richard Seroter

Richard is the Vice President of Product for cloud leader CenturyLink, a Microsoft MVP, InfoQ editor, blogger, author, trainer and frequent public speaker. He has spent the majority of his career working with organizations as they planned and implemented their enterprise software solutions. He recently earned a Masters Degree in Engineering from the University of Colorado, Boulder.

Richard is the author or contributor to three recent books: “Applied Architecture Patterns on the Microsoft Platform” (Packt Publishing, 2010) which discusses where to use which Microsoft platform technology; “SOA Patterns with BizTalk Server 2009″ (Packt Publishing, 2009) which takes a look at how to apply good SOA principles to a wide variety of BizTalk scenarios; “Microsoft BizTalk 2010: Line of Business Systems Integration” (Packt Publishing, 2011) in which he wrote chapters explaining integration strategies for Windows Azure, and Dynamics CRM 2011.

Richard maintains a semi-popular blog at that recounts his exploits, pitfalls, and musings with enterprise software. You can follow him on Twitter at, read his regular contributions to at, and find his Pluralsight training courses at

Let's begin the Interview ......

Mahesh: Who are you and what you do?
I am the Vice President of Product for global technology provider CenturyLink. I lead product for cloud and hosting solutions.

Mahesh: When did you start working on BizTalk? 
Richard:  The summer of 2000. I worked for a consulting startup named Avanade, and was asked to look at this "BizTalk thing" that was in beta form from Microsoft. I found myself on a handful of subsequent projects and became an accidental expert.

Mahesh: How did you mastered BizTalk (Learning path, amount of time)? 

Richard: Does anyone ever REALLY master BizTalk? After the 2004 release, I got pretty engaged and became fairly competent with the product. That was through many hours experimenting and reading the early product material and blogs.

Mahesh: Which are the major projects you handled so far? 

Richard:  I worked on a number of BizTalk projects over the years, but most significant ones were during my time as an integration architect for the world's largest biotechnology company. BizTalk was (and is) the service bus at the company and I worked on most of the early projects to link line of business systems together.

Mahesh: How do you see BizTalk compare to other integration platform? 

Richard:  Mature, robust, and nicely integrated with other Microsoft technology. With the recent push to cloud-enabled integration, Microsoft has split their attention a bit between things like Service Bus, Logic Apps, and other interesting cloud-first solutions. If they can marry the on-premises with cloud services, it'll be a killer combo.

Mahesh: What as per you is must to know to become an Integration(BizTalk) Expert?

Richard: To become an expert in anything, you must focus on learning and hands-on experience. There are no paper experts. Book learning alone is insufficient. Use the product, stretch the limits, break things, and reflect on the lessons learned.

Mahesh: What are your thoughts on forums,blogs and articles etc.? 

Richard: It's one of the best ways to get exposed to new ideas and angles. The BizTalk blogging community was amazing in the early years, and helped so many people get up to speed. The Technet Wiki has taken over as a prime source of information, and that's excellent as well. The most important thing is for the entire community to engage in sharing information. Don't rely on a few individuals to write about their experiences; EVERYONE should get involved.

Mahesh: Your suggestion to a newcomers? What should be approach to get sound knowledge in Biztalk?
Don't try and learn everything at once. There isn't a person in the world that can claim expertise with EVERY aspect of a product like BizTalk. Focus on hooking up basic messaging scenarios, understand core principles, and keep layering on additional things like orchestration, BAM, rules, cloud, and more.

Mahesh: What are your thoughts around BizTalk certification?
Certifications can be useful, but they are no substitute for practical experience!

Mahesh: What is the future of BizTalk?
We'll see. Integration has never been more important, and a flexible, robust integration bus is a must-have for any organization. I suspect that we'll continue to see primary attention spent on cloud-based services, but on-premises integration solutions will be relevant for years and years.

Mahesh: Any thoughts on cloud?
I think it's here to stay! Organizations have seen the power of getting on demand infrastructure and using cloud services to quickly assemble applications that add value. There's no going back to purely on-premises systems that are governed by a central IT department. But with that freedom, comes new complexities. Architects will remain busy linking together disparate systems that now span multiple geographies. 

Mahesh: What motivates you to do the community work?
I love to learn, and appreciate that we're in such an interesting period of technology advancement. Has there ever been a faster rate of change? It's exciting, and I enjoy sharing my experiences so that others can learn from my successes and mistakes.

Mahesh: Being MVP, do you feel that responsibilities get added? What is your thought on MVP?
 It's always a privilege to get renewed as an MVP, and it definitely carries additional expectations. Specifically, the expectation to produce value to the community! The friendships I've made through the MVP program have been life-altering, and it's always enjoyable to collaborate with Microsoft product teams to make products even better.

Thanks a lot Richard for sharing your experiences, this will surely benefit many !!!

Feel Free to ask questions to Richard in the comments!!!!!!!!

Related Post:

Friday, May 29, 2015

Could Not Allocate Space for Object... because the 'PRIMARY' filegroup is full

Today lots of messages were getting suspended in the production env and to find the exact reason for this sudden messages getting suspended, looked into the Event Log and found below two errors:


Why it happened:

As can be interpreted from error message, the allocated space for BizTalkMsgBoxDb was full.

What to do:

The first thing was to check if AutoGrowth is enabled or not on BizTalkMsgBoxDb, and it was.
Then had a look at the jobs and found that DTA Purge and Archive Job was disabled since a week (don't know how) and that was the reason for this issue. So to fix the issue, reconfigured the job and enabled it and also purged the Tracking database.

Friday, May 1, 2015

Begineer to BizTalk Expert : Interview with Nino Crudele

Welcome to eleventh interview of the series, today's expert is Nino Crudele.

He is full of passion for integration world and he likes to use all possible technologies to do that, Microsoft Azure, Event Hub, Stream Analytic, BizTalk Microservices, BizTalk Server, and all possible Integration technologies patterns.

Nino Crudele has been an BizTalk MVP for 7 years, besides that he is also a Virtual Technology Specialist for BizTalk Server, a role in which he works closely with Microsoft. He enjoys this role as it enables him to work with many different clients seeing BizTalk in different scenarios from a technical point of view, size, and criticality.

He developed the famous BizTalk Visual Studio NoS Addin.
All information here:

Nino has worked with most of the BizTalk accelerators, Rosettanet, HL7, SWIFT and has done some big projects on EDI and RFID. In the past he also had the opportunity to be part of a big integration project with a major Italian enterprise organization that integrates many technologies like SAP, AS400, TIBCO and others using BizTalk.

He is founder of the first Italian BizTalk communty,
He is founder of the first Italian Connected System communty,

MCTS BizTalk Server, MCT, MCAD and MCSD.

His blog at

Let's begin the interview.....

Mahesh: Who are you and what you do? 
Nino: I’m Nino Crudele, I have three amazing children Rebecca, Carolina and Riccardo, one wonderful wife, Grazia, and my life is totally focused on them.
I have been Application Integration MVP for 8 years, I’m working in Solidsoft Reply as Application Integration Specialist.
I like sports as Kayak and Skateboard, when I need to relax I just like to take a run with my skateboard, I also like to read about maths, science and play Chess.

Mahesh: When did you start working on BizTalk? 
Nino: I start many years ago using BizTalk Server 2002 version and I have a good memory about this time.

Mahesh: How did you mastered BizTalk (Learning path, amount of time)? 
Nino: I was really lucky about it because I worked for many years as Virtual Technology Specialist for Microsoft and I had the opportunity to do a lot of different things with BizTalk.
I used all the adapters and accelerators and I followed a lot of different critical missions, this was a good way to improve my knowledge.

Mahesh: Which are the major projects you handled so far? 
Nino: I worked in many different projects and for all the biggest companies in Italy, I have many different and interesting situations to speak about, some of them because using hundreds of CPUs or BizTalk boxes other because using a lot of different accelerators and adapters.

Mahesh: How do you see BizTalk compare to other integration platform? 
Nino: Good question and not so simple to answer, I think that BizTalk is the most complete and solid platform that Microsoft has developed, I really love BizTalk Server and my opinion is not because I’m Microsoft MVP but because it is real.
I used other integration platforms but BizTalk Server is the only one which able to cover all the most important aspects, for example reliable, correlation and transaction (SAGA), in very easy way.

Mahesh: What as per you is must to know to become an Integration(BizTalk) Expert? 
Nino: A BizTalk expert is a person who able to know all about the product and how to solve problem using it, he also needs to have a very good vision about integration scenarios and different technologies.
BizTalk runs on SQL Server, Windows resources, network, he has a lot of different features inside which use IIS, Disk I/O, and BizTalk has hundreds of adapters and thousands of setting to know.

An Integration expert needs to know a universe of things and BizTalk is one part of these.

Mahesh: What are your thoughts on forums,blogs and articles etc.? 
Nino: I think that all of these things are important to share knowledge, I like to write article and share, this is the best option we have to merge our passion.

Mahesh: Your suggestion to a newcomers? What should be approach to get sound knowledge in BizTalk? 
Nino: My best advices are:Keep calm, sit and start understanding the engine, BizTalk is not a complicate thing and there is a lot of good documentation and books in internet.

Start following the BizTalk engine flow, concept of port -> receive location -> adapter -> pipeline -> orchestration
How it works, Publish / Subscribing and Message context.

Mahesh: What is a BizTalk Host and Host Instance?
Nino: Don’t start developing because drag and drop and don’t force yourself using only BizTalk things, you are a developer, and you have also to use code.
Understand the BizTalk domain and what it means developing and using an own external library and how to manage it in the best way, in term of monitoring and errors.

Mahesh: What are your thoughts around BizTalk certification? 
Nino: Is a good option to certify your capabilities.

Mahesh: What is the future of BizTalk? 
Nino: BizTalk is a great product and I think it will be present for a long time.

Mahesh: Any thoughts on cloud? 
Nino: Microsoft released a lot of new interesting things in Azure , now we have a lot of more options and a lot of different technologies to use , I think That now the challenge is not about a Particular product but is the understanding how to use all of them in the best and convenient way ,

I like to say that the power of Azure is directly proportional with the summation of things that we are able to combine together.
I think about Azure as a little universe full of things and the collision between them is able to create energetic solutions.

The world of integration is huge and we have many different options to solve problems, I’m working in this new engine, JiTGate (Just in Time Gate), it is able to use all of these things together to integrate technologies in very easy and fast way, I’m in a good point with the development and I hope to release a version soon.

Mahesh: What motivates you to do the community work? 
Nino: I think the knowledge That Has to be free and not drove by the main actors, I'm not just speaking only about technology but about all, we have to be free to share and help people to understand and to live a better.

Mahesh: Being MVP, do you feel that responsibilities get added? What is your thought on MVP?
Nino: Yes I’m feeling the responsibilities to be a MVP, it is a big thing, the MVP family is full of great persons and I’m honoured to be part of this.

Thanks a lot Nino for great insights, this will surely benefit many !!!

Feel Free to ask questions to Nino in the comments!!!!!!!!

Related Post:

Monday, April 6, 2015

Fetching Data From Multiple Tables Through Dynamically Invoking Stored Procedure

There is a comment/question posted on following post:Polling data from SQL using WCF-SQL Adapter in BizTalk 2010 - TypedPolling(From Multiple table)  where Vinoth posted the scenario:

"Please help me, I have to poll the data from 15 Tables, Table by Table, should not join in to single select statement. For example, if I have 15 Tables to poll, I have to poll Table1 and store as XML and i have to poll Table 2, then 3, 4, 5...
Multiple Table polling by looping, how to do it dynamically poll the data Table by Table...
Please provide me the solution? its very urgent... "

There are situations where we have to configure WCF port dynamically,

1. The information for the configuration is available at runtime through some content in message or through some other way.

2. The configuration information is already known but it can change or has to be changed depending on the business needs.

To demonstrate how to configure WCF Port dynamically and how  to Invoke stored procedures dynamically, have used following scenario:


 I have to poll the data from Tables, one by one and save it on file system. The challenge is the number of tables can increase or can be reduced .For this demonstration , I have used 3 tables (Logic remains same for 15 tables as well).

Ok, as per the requirement posted the configuration of the port which fetches data has to be done dynamically and in a loop, so nothing has to be static inside orchestration related to the storedprocedure thus used BRE as it provides a way to do changes without having to change anything in orchestration after initial CallRules configuration. 

I have made some assumptions:

1. There is chances of changes in number of tables to be fetched - thus have used BRE which has provision of changing number of table without touching the Orchestration.
2. The response received is not to be mapped, instead it is to be saved as it is.
3. Invoking stored procedure can be anywhere in the designed process, thus have used dummy Order to instantiate the orchestration (which has nothing to do with logic)

Building the solution:

1. Database

a. Creating three tables

Figure1. Demo Database with three tables

Figure2. Order table design

Figure3. Order1 table design

Figure4. Order2 table design

b.Creating three stored procedures with simple query

Figure5. uspGetOrder1 stored procedure
Figure6. uspGetOrder2 stored procedure
Figure7. uspGetOrder stored procedure
Figure8. Three tables and respective stored procedures

2. BRE

a. Creating Policy GetConfigDetails

Figure9. GetConfigDetails Policy

b.Creating four rules under GetConfig

The first rule is used to set the number of tables to be fetched.

Figure10. GetConfigDetails Policy - Rule1

The rest three rules are used to set the properties i.e. Action,Address,Bindingtype and TransportType which are required to configure the port dynamically.

Figure11. GetConfigDetails Policy - Rule2

Figure12. GetConfigDetails Policy - Rule3

Figure13. GetConfigDetails Policy - Rule4


3. BizTalk Solution

For solution the name given is DynamicInvocationOfStoredProcedures which has schemas(intiating message, Configuration message , storedprocedure message), Orchestration, strong name key(used to sign the project) and binding files which are generated by wizard while creating schemas from stored procedures. Steps are as below.

Figure14. Solution structure

a. Create schema for message to instantiate the orchestration

b. Create schema for message to hold configuration details - which is updated using BRE

All the elements are promoted as distinguished field so as to get/set values from those in orchestration

c. Generate schemas from the stored procedures which are to be executed dynamically

We need to do this so that we can refer to the binding files generated to get configuration details.

1. Right click the project and  select Add-->Add Generated Items
2. Then select "Consume Adapter Service"
3. Now the Consume Adapter Service wizard wants us to provide it the required information, so firstly select "sqlbinding" against the label Select a binding.
4. Click the Configure button, on security tab select the Client credential type
5. On URI Properties tab:
      i.Type the name of the server where your database is located ( I have used "." as the database resides on same machine)
     ii. Provide the name of the database you want to use.
     iii. I have left InstanceName as blank because am using the default instance of SQL (If need to use some other instance then provide the name of that instance)

6. After done with above configuration,click on Connect then select Contract type as Client(Outbound operations)
7. Select the category as Procedures then select procedure based on which schema is to be generated, click ADD
8. Give Filename Prefix (it's optional - but it helps) and click ok
Figure. Generating schema from uspGetOrder storedprocedure

9. Repeat above steps for other stored procedures

d. Create Orchestration

Figure.PollMultipleOneByOneDynamically Orchestration

The orchestration as shown above has following variables and types used in it

Ports and Port Type
1. prtRcvStart of type prttypeRcvStart - to receive intial message thus binded to Order schema as intial message is based on it

2. prtExecuteSP of type prttypeExecuteSP - it is dynamic port to send message to invoke storedprocedure and to receive response from storedprocedure,binded to System.XMLDocument.
Why System.XmlDocument? - As per the requirement multiple storedprocedures are to be invoked but those are not static and are to be set dynamically so this port can't be binded to single schema. And as System.XmlDocument is the base schema for all schema it can be used in the scenario where generic approach is required.

3.prtSndInvokeSPResp of type prttypeSndInvokeSPResp - to send the response received from stored procedure to a folder

Figure.PollMultipleOneByOneDynamically Orchestration variables and types

Message and  Variables
1. msgstart - based on Order schema, used to instantiate the orchestration
2. msgConfig - based on Config schema, used to get config details from BRE
3. msgInvokeSP - based on System.XmlDocument, used to send message expected by StoredProcedure
4. msgInvokeSPResp - based on System.XmlDocument, used to get response from stored procedure
5. varLoop - Int variable to hold the value (number of times loop is to be run)
6. varAction - String variable to hold Action value set on msgConfig by GetConfigdetails policy
7. varAddress - String variable to hold Address value set on msgConfig by GetConfigdetails policy
8. varBinding - String variable to hold Binding value set on msgConfig by GetConfigdetails policy
9. varTransportType- String variable to hold TransportType value set on msgConfig by GetConfigdetails policy
7. xmlDoc - Variable of type System.XmlDocument

Steps in Orchestration

1. Receive start message
2. Construct the config message in message assignment shape, this message will be passed, here to note is that GetTableNumber element has values as Yes  

 3. Configure  the CallRules shape, select the GetConfigdetails policy and message expected by it  - msgConfig

4. Assign  the Loop variable to the value received in NumberOfTables field of msgConfig
5. Set the Loop condition

6. Inside loop , call the policy
7. Assign  the varAction,varAddress, varBinding,VarTransportType to the value received in msgConfig

8. Construct the message to invoke the stored procedure and set it's WCF.Action property - this is where we are dynamically setting the action - which we get from Policy we execute in step 6

9. Send the message to prtExecuteSP and collect the response
10. Send the collected response to the folder
11. So  now reset the loop variable by reducing it by one and also now next table is to be fetched so set TableNumber field of msgConfig respectively.

Allright, so why CallRules is used twice in above Orchestration?

Considering that number of tables to be fetched can be changed, the first callrule is used to tackle this scenario. So when Orchestration passes config message to rule engine having GetTableNumber as Yes, then  the NumberOfTables is set  and returned. And in loop, Orchestration is interested in knowing which StoredProcedure is to be executed(the WCF.Action property) and here second callrule is used to get the Action against the TableNumber.

If you see at Rules above, it will be more clear :)

e. Build, Deploy and Configure

After deployment the dynamic port is automatically added to the Application in the Admin console, whose configuration we did in orchestration dynamically.

So we just need to configure one Recieve Port, one static send port and bind it to Logical ports of Orchestration

Figure. After  Binding Orchestration

To test I dropped the instance of Order message (message which instantiates the orchestration) and three xml files were generated in the folder which is binded to the port which sends the response received from stored procedures.

Note : System.XmlDocument is used as base for the message  to communicate with stored procedures so it's untyped . There are two limitation that the response received can't be used in any map and if there is subscriber in your group for untyped message then copy of it will be passed to it as well.

Download Sample: