Monday, November 10, 2014

BizTalk Server: Multiple XML files to Single FlatFile Using File Adapter

Introduction

It’s very common in integration project where we have to deal with debatching the batch of files and batching multiple files into single file. Depending upon requirements we can do xml debatching, EDI debatching, Flat file debatching and xml batching, flat file batching, EDI batching with the help of respective assembler and disassembler.

This post demonstrates how to batch multiple xml files into single flat file using flat file assembler and file adapter.

Scenario

The best example for this would be “Collection of errors for a particular day in a single file”. But to keep this simple and easy to understand following scenario is used for this post – Solution is to be designed to collect all the Employee messages(xml) coming in on that particular day and to be batched into single flat file and saved as “todaysdate.txt”

Say, 10 employee messages came in between 12:00 AM – 12:00 PM on 10th November 2014 then all the employee message should be batched into single flat file and saved as 10-11-2014.txt


How To Do 


Input Schema


Output Schema


Custom Flat File send pipeline

There is no default flat file pipeline provided out of the box, so we need to create custom flat file pipeline. As we need to create flat file at send side, we create flat file send pipeline with the help of Flat file Assembler

Creating Orchestration


Have used Orchestration for the sake of assigning Current date as filename to the outgoing message, but yes this solution can be implemented as pure messaging solution by creating a custom pipeline component for assigning the filename.

Construct shape

1. Transform Shape

Simple one to one mapping from xml to flat file.



2. Message assignment

In message assignment shape, System’s current date is assigned to a string variable SystemDate and later it is assigned to context property FILE.ReceivedFileName, so we are actually overwriting the values of FILE.ReceivedFileName. We did this so that we can use %SourceFileName% macro at send port, which reads this context property and uses it to name the message saved at destination location.

Possible error:  The Filename, directory name, or volume label syntax is incorrect


Build and deploy


After done with the development, sign the project and assign it a Name and deploy.

Configuring and testing the application 



Download Sample

BizTalk Server: Multiple XML files to Single FlatFile Using File Adapter - Configuring and testing

Configuring the application 


Receive Port


Receive Location


Send Port


Send pipeline: MXSFF (Custom flat file)

Destination URI




  • File name: %SourceFileName%.txt (puts the value of the FILE.ReceivedFileName)
  • Copy mode: Append (The file send handler opens a file if it exists and appends a message to the end of the file. If the file does not exist, the file send handler creates a new file)


So for example, when first message or batch of messages are received on date 10-11-2014 then new file will be created at destination location having 10-11-2014.txt and then after whatever messages are received until the end of date are appended to the same file. But a new file will be created for messages received on next day.



Configuring send pipeline (Flat file assembler)



Configuring Orchestration


Testing the application


To test the functionality of application, considered two test cases

Testcase 1: First files of the day


Input: Placed three sample files in XMLIN folder (input location)


Output: A file with today’s date was generated at FFOut (destination location)

Testcase 2: Rest files of the day


Input: Placed two sample files in XMLIN folder (input location)
Output: Records were appended to the existing file



Download Sample


Saturday, November 1, 2014

Begineer to BizTalk Expert : Interview with Tord Glad Nordahl

Welcome to fifth interview of the series, today's expert is Tord Glad Nordahl.

Tord Glad Nordahl is working as a BizTalk Administrator Expert for Bouvet ASA in Norway. He worked with BizTalk for the last 6 years however he’s been working with all versions of BizTalk. His main jobs is Tech Lead of Bouvets Integration Competence Center, health checking, traning, and as ad-hoc support during critical problems for clients. He’s set up BizTalk over a 100 times for different clients all over the world. Tord Glad Nordahl is also an active contributor to the community with his work on his blog www.biztalkadmin.com, on the MSDN TechNet forums and Wiki.

He was awarded Microsoft Integration MVP for his commitment for the BizTalk community for two consecutive years (2013 and 2014). He’s held presentation all over the world and is always up for a challenge when it comes to BizTalk.

Tord is also a member and co-founder of BizTalkCrew along with Nino Crudele, Saravana Kumar, Steef-Jan Wiggers and Sandro Pereira that are responsible for organizing BizTalk Innovation Day event all across Europe. Before starting the interview, I would congratulate Tord for the new journey which he is going to start today with Microsoft, by becoming part of it.

Mahesh: Who are you and what you do? 
Tord : My name is Tord Glad Nordahl, I'm 27 years and got three kids, Natala, Maxim and Maribell. I've worked as a BizTalk consultant for the lest three years, my nation focus has been operation, but I've done development too.

Mahesh: When did you start working on BizTalk? 
Tord: It all started 6 years ago, I was working for a rather big company in Norway collecting all web services and putting them on a single webfarm. I was in the same room as two BizTalk administrators. One day when both of them were out of office BizTalk stopped working and I was put on the task, i fixed it but at the same time my passion towards BizTalk grew drastically that day.

Mahesh: How did you mastered BizTalk (Learning path, amount of time)? 
Tord : I would say it was learning by doing and playing around with it. When i first started working with BizTalk the stuff for operation like blogs, wiki articles etc didn't exist. I remember learning alot but starting up my blog.

Mahesh: Which are the major projects you handled so far? 
Tord: Although I wish I could talk about all protects I've done, most of them are under NDA. However I can say this, the biggest one was an upgrade from BizTalk 2010 to BizTalk2013 for 16 environments.

Mahesh: How do you see BizTalk compare to other integration platform? 
Tord: BizTalk has its share of "competitor" products although none of these products it's similar to BizTalk, thanks to our message box BizTalk stands out. People tend to focus on the negative side of BizTalk, like there is little innovation, but remember that Microsoft BizTalk is the only product from Microsoft that has never had a security update.

Mahesh: What as per you is must to know to become an Integration(BizTalk) Expert? 
Tord: Becoming an integration expert is I impossible, there is simply too much to grasp it all, you could probably on the other hand be expert on certain areas, like HL7, Maps etc. Either what you choose you need to read, try, fail and do it all over again. 

Mahesh: What are your thoughts on forums,blogs and articles etc.? 
Tord: The BizTalk community has grown a lot the last years, the amount of articles, blogs and forum responses has grown. I love song the community blossom like this. Keep up the good work.

Mahesh: Your suggestion to a newcomers? What should be approach to get sound knowledge in Biztalk? 
Tord: Get yourself a sandbox and play around on it. Practice is the key and you'll get better and better for every day, Simone once said that a good developer it's a developer that knows how to use Google (or Bing). Ask anyone and don't be afraid to do so. You wont be looked at in a bad way.

Mahesh: What are your thoughts around BizTalk certification? 
Tord: Well, it's good to acknowledge your experience, but during recruitments we don't focus too much on the certification, the "solution" to all question is out there and there for focusing on good questions during the interview that is not found online that easy. :-)

Mahesh: What is the future of BizTalk? 
Tord: The future of BizTalk is bright. The marked is focusing on creating cost effective ways to run their business,integrating their systems is a good way to achieve these goals, BizTalk is still marked leader, with new releases every year even Microsoft shows off the fact that they are investing in the product.

Mahesh: Any thoughts on cloud? 
Tord: So the cloud is the way to go, and were all know that moving everything to the cloud in one step is not smart, however for many technologies, including integration, utilizing the cloud for processing can remove some of the dynamic lead you have on the on-prem solutions.i think the main focus for companies is to look at hybrid solutions.

Mahesh: What motivates you to do the community work?
Tord: Remember sharing is caring!

Mahesh: Would you like to share about your journey of becoming MVP and now joining Microsoft?
Tord: So I don't have any travel plans when it comes to my career, however I do have a personality that our me in positions to get awards like the MVP Award. But it was never a goal. I love to skate and the rewards you heartbeat is priceless, getting a thank you from someone you've never met it's worth more than any award.. Do community work because you want to, not because you want an award. It's not a race.

So loosing my MVP award this year is a little sad, although it's a part of my new journey when I join Microsoft (you cannot be an MVP and MSFT). I will however still work with BizTalk in MS as a technology solution professional.

Thanks a lot Tord, great insights, this will surely benefit many.


Feel Free to ask questions to Tord in the comments!!!!!!!!



Related Post:


Sunday, October 26, 2014

EDIFACT UNB and UNG Segment Fetcher Helper Class

Introduction

Many times there is requirement to pass or store the values from UNB(Interchange Control Header) and UNG (Functional group) segments. This helper class can be called from map and individual values can be fetched.

How To Do

To create helper class do following:
1. Open Visual Studio
2. Add new C# library project
3. Add below code

Code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace EDIFACTSegmentValueFetcher
{
    public class SegmentValueFetcher
    {
        public string fetchvalue(string segment, string elemsptr, string subelemsptr, int pos, int subpos)
        {
            string subsegment;
            char esptr = elemsptr[0];
            char sesptr = subelemsptr[0];

            var lst = segment.Split(esptr);
            subsegment = lst[pos];
            if (subsegment.Contains(subelemsptr))
            {
                var sublst = subsegment.Split(sesptr);
                return sublst[subpos - 1].Replace("'", "");
            }
            else
            {
                return lst[pos].Replace("'","");
            }
        }
    }
}

4. Sign the project
5. Add it to GAC


How It Works

EDIFACT  messages are wrapped with envelope, at header it has UNB segment (Interchange control header - Mandatory), UNG (Functional group - Conditional) and UNH (Transaction Set) and at the trailer it has UNZ, UNE and UNT. Enveloping segments work in pairs, UNB-UNZ represents an interchange, UNG-UNE is a functional group inside of the interchange and UNH-UNT is a transaction inside the group.

Many times there is required to pass or store the values from UNB (Interchange Control Header) and UNG (Functional group) segments. But this is not part of the data and not all properties from Envelope are promoted or written in the context of the message. Segments have data elements which are separated by element separator, also sub data elements which are separated by sub-element separator.

This helper class helps in fetching the individual segment and subsegment values. You need to provide the whole segment, element separator, subelement separator and the number of field you want and number of subfield.

Example(Header segments of INVOIC):

UNB+UNOA:1+BTS-SENDER:ZZZ+RECEIVE-PARTNER:ZZZ+141024:2231+201410242231'
UNG+INVOIC+BTS-SENDER+RECEIVE-PARTNER+141024:2231+201410242231+UN+D:96A'

Suppose from above sample,
1. If UNG7.1 value is to be fetched, in that case UNG segment, element separator, subelement separator, field value as "7" and subfield value as "1" will be provided as input and the output will be "D"
2. If UNG7.2 value is to be fetched, in that case UNG segment, element separator, subelement separator, field value as "7" and subfield value as "2" will be provided as input and the output will be "96A"


Download Sample


Related Posts


BizTalk Server : Fetching UNB and UNG Segment Values From Interchange Envelope and Mapping - Configuring and Testing

Step 6: Configure and test the application


1. Create receive port



2. Create receive location


2. Create send port



4. Set the destination URI



3. Configure the Orchestration


4. Create drop location
4. Drop the sample input file at input location
5. Check the output


Note: Create a party and agreement and use sample file as per it. OR Enable EDIFACT fallback settings and configure it and use values in sample file as per it. As EDI receive pipeline first checks Party for the configuration, if no matching party found then it looks up for Fallback settings.

Possible Error

In a scenario where the particular element is not present in the segment (input) and if we try to fetch that particular element then following error is encountered.
Say, if UNB8 is not there in input and if tried to fetch that data then error occurs.


Download Sample

Related Posts