Monday, May 22, 2017

JSON Decoder : Unexpected character encountered while parsing value

After fixing an error with JSON decoder -Property not specified in JSON decoder, got one more error from JSON decoder-



 



From the error it is clear that the data in response from the api is not valid JSON, thus the decoder throws error.


Why it happened


I went through the tracked service instance and checked what exactly is received, and as can be seen in below snap - it was HTML and no JSON message. 




And as the returned message starts with <  , the decoder was not able to parse it as valid JSON message is like:

{
   "company": "Maruti",
   "name": "Swift",
   "price": 800000
}

Note: The RestApi has not implemented the required functionality, only instance of webapi is provisioned in Azure - thus the HTML is returned.

What to do 


Here, we can't do anything except get the functionality implemented in Rest Api and let it return valid message. This error can also be encountered in case you are using JSON decoder and you receive XML.

Sunday, May 21, 2017

Error xxxx not specified in JSON decoder properties

Am working on a POC where I have to invoke a Rest Api, collect the response and do futher processing on it. So to accept JSON data coming in response, I created a custom pipeline having JSON decoder at decode stage. After deploying the project and configuring it, dropped the test instance at receive location and watching for output in OUT folder(configured to archive API response).

As no output was generated - means some error was encountered, visited event log and found below error:









Why this happened


It was my bad, missed to set couple of properties which are required by JSON decoder to convert JSON message in corresponding XML message i.e. Root Node Name and Namespace.







Above error is in situation where you do not specify Root Node Name and Namespace both or if only Namespace is set.
Out of curiosity wanted to check what JSON decoder says if I set Root Node Name only and leave Namespace as Blank - It says, I want it to build XML message for you ;)


What to do


When using JSON decoder it is mandatory to provide this two properties.It is because the Key is converted into element and Value as element value.

e.g.,

JSON Message:

{
   "company": "Maruti",
   "name": "Swift",
   "price": 800000
}

Corresponding XML:

   <company>Maruti</company>
   <name>Swift</name>
   <price>800000</price>



But a valid XML has a Root Node, thus that needs to be provided explicitly to JSON decoder to attach the converted XML structure under Root Node 

Friday, May 19, 2017

Installing BizTalk 2016 feature pack 1

Microsoft recently released feature pack 1 for BizTalk server 2016, this is first time such feature pack is presented and off course it is for enriching customers experience that too without breaking existing functionality.

Organizations can only use Feature Pack 1 with BizTalk Server 2016 if they have the Enterprise or Developer editions of the product, plus Software Assurance. No other editions are eligible. It's also possible to use Feature Pack 1 if BizTalk Server is run in Azure with an Enterprise Agreement, the MSDN article explained

So thought of installing it and exploring it, installation is quite straight forward - steps below


2. Run the installer 









This installation requires to restart the server to reflect the installed features.

Microsoft claims Feature Pack 1 is production ready and provides only non-breaking features (Off course we need to verify this before thinking of Production). Is there anything noticeable thing(after installation) supporting below features - there saw few things


Deployment:

1. Deploy with VSTS – Enable Continuous Integration to automatically deploy and update applications using Visual Studio Team Services (VSTS)
2.New management APIs – Manage your environment remotely using the new REST APIs with full Swagger support


Analytics:

1.Application Insights – Tap into the power of Azure for tracking valuable application performance and to better understand your workflow 

To support this there is an option provided in tracking section wherein we can Enable Analytics. What is Application insights - After creating Application Insight, capture the instrumentation Key, as this is needed when enabling Analytics on BizTalk group and it is passed as connection parameters









2.Leverage operational data – View operational data from anywhere and with any device using Power BI



Runtime:

1.Support for Always Encrypted – Use the WCF-SQL adapter to connect to SQL Server secure Always Encrypted columns



SQL Server 2016 introduced a feature that enables client applications to read/write encrypted data within a SQL table without actually providing the encryption keys to SQL Server and that's been included in WCF SQL adapter as a property, by default it is Disabled.



2.Advanced Scheduling – Set up advanced schedules using the new capabilities with Feature Pack 1 

If you would have used Scheduled task adapter, this was on developers wanted list for long time to have scheduling capability on granular level. Including Time zone is really thoughtful





Tuesday, May 16, 2017

The Login is from an untrusted domain and cannot be used with Windows authentication

During daily routine checkup, we found following warning and error in event viewer.

At first it might appear from the error that BizTalk Server machine and SQL server machine are in different domain or the user account used for configuring the BizTalk service is from a different domain than from SQl server's domain  -- None is the case 



The Login is from an untrusted domain and cannot be used with Windows authentication  warning


The Login is from an untrusted domain and cannot be used with Windows authentication error


Why it happened


It is known issue and this errors are result of restart of Domain controller(A domain controller is the centerpiece of the Windows Active Directory service. It authenticates users, stores user account information and enforces security policy for a Windows domain) or unavailability of same over the network,thus the BizTalk applications cannot authenticate users until the Domain Controller(s) is not up.And it is because SQL is not able to find the authenticated domain user (as domain controller is not available at moment) and thus throws error. 


What to do


There were no more errors as soon as Domain controller was up but there can be other reasons too, refer following link: https://blogs.msdn.microsoft.com/dataaccesstechnologies/2012/12/19/error-message-login-failed-the-login-is-from-an-untrusted-domain-and-cannot-be-used-with-windows-authentication/

Saturday, May 13, 2017

BizTalk host BizTalkServerApplication throttled because DatabaseSize exceeded the configured throttling limit

On one of the BizTalk servers were many of  BizTalk WCF service are published started giving  Timeout issue (it was working absolutely fine before that).

The first thought after seeing timeout error is to check the WCF timeout settings, but it couldn't be the reason as those were already configured while deployment, also it worked fine in load scenario too. So it was something else which was to look upon.

Upon visiting the event viewer, found a warning - BizTalkIsolatedHost throttled because DatabaseSize exceeded the configured throttling limit



Learning - Never ignore Warnings ;)



Why it happened

The most probable reason for increase in database size is due to misbehavior of the jobs which are intended to keep everything flowing smoothly in SQL(BizTalk). Thus it needs to be validated  that the jobs are configured properly and also if required clean/shrink the database. 

So it was MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb which was failing continuously with below error.
Message
Executed as user: xxx\xxxx. The SELECT permission was denied on the object 'sysjobs', database 'msdb', schema 'dbo'. [SQLSTATE 42000] (Error 229).  The step failed


MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb calls the MessageBox_Message_Cleanup_BizTalkMsgBoxDb which deletes the message which have fulfilled the subscription . Whenever message is published in Message box, subscribers get copy of the message and not the actual message as messages in BizTalk are immutable. Upon fulfilling the subscriptions messages are marked for deletion.

Thus messages were getting piled up.


What to do

Somehow there was access issue for the account, which was granted and the job MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb was enabled. By default MessageBox_Message_Cleanup_BizTalkMsgBoxDb  is disabled and should not be run manually.






Saturday, April 8, 2017

MissingMethodException - Method Not found

Below error was encountered after deploying the application and doing a smoke test on newly configured BizTalk server


method not found

The error says there was some exception while doing a transformation in orchestration- method not found: 'Void Microsoft.BizTalk.ScalableTransformation.BTSXslTransform.set_MapUseXSLTransform(Int32)'.

But this gets overruled by the fact that same application runs fine on other environment with same data used for testing, so there is no problem with the constructing of message.

It is reported on one of the forum thread that this issue is related to assemblies not deployed correctly. Following to this, redeployed the assemblies - didn't help. Another probable reason, is the deployed assembly latest - this doesn't apply here as no changes were made in the assemblies. 

Why it happened

We had try tried to apply CU 5 on this server(Note: Same CU5 is applied on other servers successfully and are running fine.), but post that the server started behaving weird and numerous error related to DB started to pop out - like few Stored procedures are missing and so on. 



So we decided to reconfigure the BizTalk server thus creating fresh BizTalk DB's (we didn't had Backup else restoring it would have been the choice).

The reconfiguration went successful with little surprise - last post about it

What to do


Although the configuration was successful, we are getting error - assembly redeploying not helping, so it was certain that the effect of CU5 still exists thus asking to go for reinstalling the BizTalk server, but before doing that thought of repairing - and it did solve the issue.

Still, the main problem remains unanswered why on few servers CU5 was successful and on this server it caused issue(same steps followed on all). Will try to dig into this issue and if found something will share it.



Friday, April 7, 2017

System.EnterpriseServices.TransactionProxyException

A little background - we were provisioned with new database server and a new database instance against which we have to configure the BizTalk server (which was configured against the old one).



It is no complex tasks, you run the configuration wizard and provide the details and you are done :) . But sometimes simple task blow your mind with some surprises, this is what happened to us.  The configuration wizard was not happy and it threw an error


Transaction Proxy Exception

MS DTC service is configured incorrectly
MS DTC Ports are not allowed in the Firewall rules

On BizTalk sever it was all good (as it was already configured with old database). So the doubt was on new database server, we checked on it too and there too all was good.

Now this was annoying and ate lot of our time. After lot of thought and trying various thing, we disabled the MSDTC settings and enabled again followed by restarting the server - and the trick worked.


Why it happened

The only deduction I can think of is - that while re-configuring the MSDTC settings were disabled and were not refreshed on GUI . 

If anyone knows the real reason, please share your views in comment. 


What to Do

First is to check the MSDTC settings and firewall setting , if that doesn't work then do as I did.

Another interesting post around this error, I read after resolving the above issue is by Kent  - msdtc adventures 


.

Sunday, April 2, 2017

Routing Failure Report

Recently we have started getting Routing Failure Report error(non resumable instances) as can be seen below:


routing failure report error message

routing failure report




But when cross verified by going through the tracked service instance, all looked good as all the instances of the interface for which we got error was completed successfully.

completed instances
No, it is not related to the bug reported with SP1 of BizTalk 2006 R2 , so why suddenly we started getting this error in spite of the instances getting completed (BizTalk 2013 R2) 


Why it happened


There were some errors in the event log at the same time when there was Routing Failure report generated in admin console. 


Orchestration xlang exception


The error says that there was exception raised in orchestration while processing the request, still the question persists - why Routing Failure Report for this?

There is one more thing pointed in error, service xxxx terminated without responding to the operation xxxx, thanks to my colleague Pravin Shinde for pointing out the reason. 

Let me explain the interface

As can be seen from the diagram, interface receives a request and using the information from it, it invokes the target webservice and collects the response. If there is any exception in the process, it is caught by exception block and in it the respective stake holders are notified about it.

But why Routing failure report? - It was because there was exception raised before the step 4 (Send response to caller) and that was over passed,as on exception control goes to exception block. Thus the operation of sending response was not completed although the Orchestration instance completed by sending email to stakeholder.


What to do

Then there might be question what to do with the error -- Terminate it. And yes, connect with stakeholders and let them know or understand yourself what data was incorrect, with a fix on it to avoid unnecessary error in admin console.

Sunday, March 5, 2017

Begineer to BizTalk Expert : Interview with Rene Brauwers

Welcome to the 26th interview of the series, today's expert is Rene Brauwers


Rene is Solution Architect with a strong focus on Microsoft Related Products and Technologies and a clear vision: “Enabling businesses to achieve, maintain and prolong a competitive lead in a mobile-first, cloud first world, by means of smart integration”.

Core Proficiencies: Integration Architecture, Solution Architecture, Enterprise Application Integration,Business Process Management,Service Oriented Architecture,Microsoft BizTalk Server 2004 – 2013R2,Microsoft Azure,Software AG WebMethods 6.x,Microsoft Dynamics CRM 2011 / 2013,.NET Development, C#, Database modeling/designing,Documenting,Testing,SQL and BizTalk Clustering,(IIS),Network Load Balancing,Troubleshooting & Problem Solving.

Personal characteristics: Upright,Professional,Social able,Good sense of humor,Innovator,Committed,Problem solver,Tenacious,Result driven


Personal strengths:Constant drive to learn,Planning / time-management,Not afraid to ask for help,Taking responsibility,Independent,Team player, Stress resistant,Thinking in solutions,Ability to explain technical IT issues/concepts to a non-technical audience, Documenting, Business Analysis skills.

Let's Begin Interview....

Mahesh:Who are you and what you do? When did you start working on BizTalk? 
Rene:I’m a Dutchie who emigrated to Australia 2 years ago, and work as a Solutions Architect at MOQdigital, based In Sydney. I started working with BizTalk more than a decade ago and as such have worked with BizTalk Server 2004, 2006, 2006r2, 2009, 2010, 2013,2013R2 an 2016.

Mahesh:How did you mastered BizTalk (Learning path, amount of time)? 
Rene: When I got first introduced to BizTalk, I already had experience in the integration space a few years using WebMethods as such I was already familiar with SOA principles, messaging etc. This helped me when moving to BizTalk. So, if I would have to put the amount of time against it, it would be countless hours. 
Having said this, it should not scare anyone as gaining experience is not something which happens overnight, it is something which you get better at the more you expose yourself to it.
The best advice I can give anyone who is starting on their integration journey, is jump in, get hands-on experience, follow best practices, engage with the integration community and do share your experiences; in short, work, -and play hard, enjoy the journey and remember we all had to start somewhere. Patience and perseverance is what gets you there at the end!

Mahesh:Which are the major projects you handled so far? 
Rene: There have been countless big projects, ranging from complete re-platforming; moving from one integration platform to another, to greenfield projects. All in all, the size of a project really doesn’t matter, the thing which matters is taking a customer on the integration journey and delivering the best possible outcome for the business, being it small or large projects.

Mahesh:What do you think is the most challenging part while working on Integration project?
Rene: One of the things I’ve noticed ever since moving from The Netherlands to Australia is that the challenges are everywhere more or less the same; and one of these challenges is around communication and then specifically communication with the different stakeholders as well as vendors of all these disparate systems we have to integrate with. Luckily for us integrators the majority of us are besides technically savvy good communicators as well; allowing us to act as the communication orchestration engine ;-) 

Mahesh:How do you see BizTalk compare to other integration platform? 
Rene: BizTalk is a one of a kind integration platform. It’s versatile, extensible, reliable and above all very mature if not one of the most mature integration platforms out there.  
Nowadays however I no longer only refer to BizTalk as Microsoft’s Integration platform, I refer to the Microsoft Integration Stack in which BizTalk plays an important role to enable (Hybrid) integration scenario’s and in which we can leverage Azure (iPaas) capabilities such as Logic Apps, API Management, Servicebus, Flow, Functions, Service Fabric, App Services and many, many more
So in my opinion; Comparing competitors’ offerings there is currently no competitor in the market who is able to match the integration capabilities which Microsoft offers both from a technology point of view as well as a commercial (capex, opex) point of view.

Mahesh: What as per you is must to know to become an Integration(BizTalk) Expert? 
Rene: It’s all about understanding the basics, so know your programming basics, understand Service Oriented Architecture read up on the Enterprise Integration Patterns, start thinking in messages and in a decoupled way ;-) 
But above all, you need passion, perseverance and enjoy what you do. If you have this, you are one of the lucky few who are able to make their work their hobby and their hobby their work.

Mahesh:What are your thoughts on forums, blogs and articles etc.? 
Rene: They are all great and important resources which enables the Microsoft Integration Community to share its knowledge allowing people just starting with integration as well as seasoned integration experts to interact with the community.  The Microsoft Integration community is all about sharing, as sharing is caring.

Mahesh:Your suggestion to a newcomers? What should be approach to get sound knowledge in BizTalk? 
Rene:There are plenty of online as well as offline resources. Ranging from books on BizTalk to online courses on sites such as Pluralsight. Alternatively; go and sign up for class-room based BizTalk training and so on.
Most importantly; Just dive in and start doing by playing with the awesomeness in the Microsoft Integration Stack and please do reach out to the community by visiting local user groups, view webcasts such as Integration Monday, integration Friday. 

Mahesh:There are many tools from community which support BizTalk in some or the other way(like BTDF, Bizunit etc), what do you say about it? Which ones you would recommend?Why? No Microsoft support available?
Rene:There are amazing tools available with support from the Community, and I honestly must say that they all fit certain needs. But like a lot of things, there are different ways to solve problems; so one has to look at the tools on a per case basis to determine which one fits for a given scenario. 

Mahesh:What are your thoughts around BizTalk certification? 
Rene:There is currently no up to date BizTalk certification program available (Except for the Partner Assessment)  and I do not expect that there will be an update pretty soon.
In general Certification is a way of proving you have certain level of understanding in a given technology; unfortunately, in the passed there have been a lot of brain-dump sites which were pointing to the answers as such I feel that passing an exam in some scenarios would not be an accurate representation of once capabilities
Fortunately the current Microsoft certification process has gotten much better and much harder and as such the traditional brain-dump sites are no, too little of use.  

Mahesh:What is the future of BizTalk? 
Rene:BizTalk 2016 has just been released; and other releases are already planned; as such there is no doubt that BizTalk fulfils an important role in Microsoft’s core Integration Strategy 

Mahesh:Any thoughts on cloud?
Rene:Cloud is here to stay, and one nowadays simply can no longer ignore the cloud. From an integration point of view this opens up a lot of exiting opportunities. Microsoft recognised long before the importance of the cloud and has gone all-in on it, thanks to this we as integrators now have the tools and services which allow us to succeed in the Hybrid Integration Space.

Mahesh:What motivates you to do the community work? 
Rene: One of my passions is sharing and the best way to share is by giving back to the community. The Microsoft integration community is very rewarding and by engaging the community actively I have made a lot of new friends over the years.

Mahesh:As per the Roadmap provided by Microsoft,LogicApps can be run on-premise in addition to Azure.Do you think Azure Stack Logic Apps on prem will supersede BizTalk Server? 
Rene:I don’t believe it will. 

Mahesh:From my perspective, Microsoft keeps coming up with Overlapping technologies like recent ones MABS and Logic Apps, in some situation it gets puzzling. What you say?
Rene: I don’t agree, I see it as paradigm shift and as such it is natural that in order to deliver a new platform or capabilities one goes through certain incarnations of products. So, from an early adopter point of view, it can be challenging at times, but hey I love learning and as such I embrace change and make it work. Just remember; Rome wasn’t built in one day.

Mahesh:Do you think BizTalk in cloud (IAAS) is accepted over BizTalk on Prem? Which one you prefer - what pros and cons you see?
Rene: I think hosting BizTalk in Azure (IaaS) is perfect for DEV and TEST scenario’s, especially whilst leveraging Azur DEV/TEST labs.  

Whatever I prefer and the pros and cons are not relevant as one has to look at which technology and platform to use on a case to case basis, at the end of the day is all about delivering the best solution for our customers.

Thanks a lot Rene for taking out time and sharing your insights,experiences, this will surely benefit many !!!

Feel Free to ask questions to Rene in the comments!!!!!!!! 



Related Post:

Friday, January 6, 2017

The hard Purge interval cannot be less than the soft purge interval

On one of the BizTalk env(non prod) there arouse a space crunch scenario which was call for purging the data from BizTalk tracking database(DTA DB).(dtasp_BackupAndPurgeTrackingDatabase is not configured on this env)





So do that, we decided to purge/delete only those instance which are completed and are older than 30 days and incomplete instances of only past day.

Note: No archiving was needed so instead of dtasp_BackupAndPurgeTrackingDatabase we used dtasp_PurgeTrackingDatabase

Thus the parameters set were:
@nHours tinyint(Any completed instance) :0
@nDays tinyint(Any completed instance) : 30
@nHardDays tinyint(Any including incomplete instance): 1
@dtLastBackup(Set this to GetUTCDate() to purge data): CurrentDateTime

For more details on parameters : https://msdn.microsoft.com/en-us/library/aa578470.aspx



And when started the job, it failed at second step and to check the reason went through history and found the error as below


The error says that 3rd parameter should be greater than 2nd parameter -- for me it is still puzzling and will try to get it understood by product team. But to proceed with purging went ahead and changed the parameters




And started the job and waiting for some time the job was successfully executed



Note: Even now the database size remain same -- here we have to select the DB and Shrink it. This actually forces to release the unused space from DB and as can be seen from below snap - around 20 GB data was purged.




Update: