Bouvet BizTalk Innovation Event

Together with Steef-Jan Wiggers, Sandro Pereira, Nino Crudele, Saravana Kumar and host Tord Glad Nordahl, I will do a presention at the Bouvet BizTalk Innovation Event which will be held on 26th and 27th September at Bouvet's office in Stavanger, Norway!

During my presentation I will tell about the added value of Message Flow Monitoring over Endpoint Monitoring. Further I'll show how Message Flow Monitoring is implemented in BizTalk360.

The program looks as follows:

DAY 1
09:00 Registration
09:30 Introduction
10:30 Pause
10:45 Steef-Jan Wiggers: Adapter Pack Integration Capabilities (BizTalk)
12:00 Lunch
13:00 Sandro Pereira: Introduction to the Azure Service Bus EAI/EDI features.
13:45 Pause
14:00 Tord Glad Nordahl: Throttling/& Thresholds
14:45 Pause
15:00 Saravana Kumar: Using BizTalk360
15:45 Lex Hegt: Message Flow Monitoring BizTalk 360
16:00 Pause
16:15 Nino Crudele: BizTalk assessment and architecture review
17:00 Tord Glad Nordahl: Wrap up – Road map BizTalk Next
17:30 Finish
20:00 Dinner

DAY 2
09:00 Startup
09:30 Labs
11:30 Lunch
12:30 Labs
15:30 Finish

I want to thank Tord and has company Bouvet for their hospitality and for giving me the opportunity to do a presentation.

Links:
- more information about the event
- prices
- registration

Off course this BizTalk event is added on http://www.biztalkevents.com/.

BizTalk Message Based Routing: why is my file not transfered to the location of the Send Port

Inspired by some tweets I had with Sam Vanhoutte (@SamVanhoutte), Mikael Sand (@MikaelSand) and Dean Robertson (@DeanRobertson), I decided to write down a number of our experiences of what can go wrong, or what you could forget, in case of a simple message based routing scenario.

So basically this article contains a checklist, which can be used as a reminder in case your message based routing scenario does not work, like you expected.

Scenario
Imagine you have a message based routing solution in its most simple form.
Your Receive Location listens to a local folder and your Send Port is subscribed to the Receive Port from the Receive Location. The URI of the Send Port is another local folder.
Although this is a very simple solution, still there is enough that can prevent the messages from flowing from A to B.

1. The Receive Location listens to the wrong folder or contains a typo
To be absolutely sure you have configured the correct folder, copy the path from the address bar in Windows Explorer and paste it at the URI field in the Receive Location.

2. The Receive Location is not enabled
If the Receive Location is not enabled, it won't poll the configured folder, so you must enable the Receive Location.

3. A Service Window or Start/Stop Date prevents the Receive Location to poll for incoming messages
On a Receive Location a Service Window can be configured. When a Service Window is set, the Receive Location will only process incoming messages during the configured timeframe. So be sure no Service window is active.
Furthermore you can also configure a Start/Stop Date. When a Start Date is configured (and enabled) which lies in the future, no processing will take place until that date. When a Stop Date is configured (and enabled) which lies in the past, no processing will take place from that date on.

4. BizTalk is not authorized to access the file location of the Receive Location
Make sure that the identity which is configured at the Host which runs the Receive Location, has enough permissions (read/write) on the folder of the Receive Location. Failing to do so, results in the Receive Location disabling itself (after some retries).


5. Files with names longer than 256 characters are encountered in the receive location
Files with names longer than 256 characters are ignored by BizTalk, so they will not picked up by the Receive Location. So make sure your file names are shortes than 256 characters.

6. Files have the System or Read-only attribute
If (one or more of) the files at the Receive Location have the System or Read-only attribute, they will be ignored by BizTalk, so make sure that the files do not have the System or Read-only attribute.


7. Empty (zero byte) files
If an empty (zero byte) file is picked up by the File Receive adapter, the file is deleted and a warning is written to the application log of the BizTalk server. The File Receive adapter deletes zero byte files by design.


8. The Host Instance which runs the Receive Location is not started
Host Instances take care of the actual processing of your Receive Locations, Send Ports and Orchestrations. So if the Host Instance from the Receive Location is not started, no files will be picked up, although your Receive Location is enabled, so start the designated Host Instance.

9. The filter on the Send Port contains the wrong Receive Port or contains a typo
Correct this by copying the name from the Receive Port and paste it in the filter of the Send Port.

10. The name of the Receive Port in the filter of the Send Port is surrounded with quotes
Quotes are not needed, so remove them.

11. The Send Port is not enlisted and started
If the Send Port is not enlisted, there is no active subscription for the Send Port. Further the Send Port must be started to send any files, coming from the Receive Port to the outgoing folder. So Enlist and Start the Send Port.

12. The Host Instance which runs the Send Port is not started
Host Instances take care of the actual processing of your Receive Locations, Send Ports and Orchestrations. So if the Host Instance from the Send Port is not started, the file will be picked up by the Receive Location, stay in the MessageBox, but won't be delivered to the outgoing folder, so start the designated Host Instance.

13. BizTalk is not authorized to access the file location from the Send Port
Make sure that the identity which is configured at the Host which runs the Send Port, has enough permissions (read/write) on the folder of the Send Port. Failing to do so, will result in suspended messages.


14. The File Send Adapter cannot write to the file specified because it has the Read-only or System attribute
If you want to append data to a file, but the file to which is written by the Send Port has the System or Read-only attribute, BizTalk won't be able to write to it, so make sure that the file on the Primary (or Backup) Location do not have the System or Read-only attribute.


15. The Send Port uses the Backup Transport (if configured)
In case anything went wrong while writing to the Primary Transport from the Send Port, BizTalk tries to write the message to the Backup Transport if it is configured. So if a message shows up on on the Backup Transport, apperently went wrong while delivering the message to the Primary Transport.


16. Mismatch of the file mask on the Receive Location
The file that you drop on the folder of the Receive Location does not match the file mask from the Receive Location. If you are dropping a .XML file while the Receive Location expects only *.TXT files, your .XML file won't be picked up by the Receive Location.

17. Tricked by Windows Explorer
You think you are dropping a .XML file, and the file mask expects .XML files, but still the file is not picked up. Perhaps you are tricked by Windows Explorer. Windows Explorer has an option to hide familiar extensions of files. So if you have a file called order.xml.txt, Windows Explorer hides the .txt extension, perhaps giving you the idea that you are dealing with a .XML file, while in fact you are dealing with a .TXT file. In Windows Explorer turn off the option to hide familiar extensions, making sure that you will always deal with the full file name.

Thanks to Mikael Sand for supplying a number of possible problems. Some of the problems are derived from this article.

New website: biztalkevents.com

Because there are so many BizTalk (and integration) related events, I thought it would be nice, to dedicate a website for them. So I created www.biztalkevents.com, added a calendar, an entry form and added a couple of upcoming BizTalk related events.

If you have an event, which other BizTalkers must know about, just fill out the entry form and I will take care that the event will be shown in the calendar.

International BizTalk Events 2012: May is Integration month!

In February 2012 Ordina (by means of Steef-Jan Wiggers) organised the Ordina BizTalk Innovation Event. We had the honour to receive 2 international guests, namely BizTalk MVP Saravana Kumar and Tord Glad Nordahl. Since that event multiple international BizTalk/Integration events are being organised. I'll mention a couple that I know off:

10 May: UK Connected Systems User Group
Location: London, United Kingdom
Speakers: Saravana Kumar, Tord Glad Nordahl
More information can be found here

22-24 May: Integration Masterclass
Location: Brisbane, Melbourne, Sydney, Australia
Speaker: Richard Seroter
More information can be found here

24 May: Overnet BizTalk Innovation Event
Event Location: Milan, Italy
Speakers: Steef-Jan Wiggers, Sandro Pereira, Saravana Kumar, Tord Glad Nordahl
More information can be found here

26-27 September: Bouvet BizTalk Innovation Event
Location: Stavanger, Norway
Speakers are not yet confirmed, but probably: Steef-Jan Wiggers, Saravana Kumar, Tord Glad Nordahl, Lex Hegt
More information: not yet available

So if you are into BizTalk (or integration) and in the neighbourhood of one of those events, join the party!
My pick? Milan and Stavanger! Hope to see you there!

My contributions to TechNet Gallery

As I like to research how BizTalk's interiors work, I often find myself writing queries against BizTalk's databases. Until now I've put literally a handfull of those queries at TechNet Gallery.

Here follows a description of those queries:

View Hosts and Host Instances Information from a SQL Query
This query shows information about the configured Hosts and the Host Instances within your BizTalk Group(s). The query becomes run against the BizTalkMgmtDb database.

It shows the following information:
• The name of the BizTalk Group
• The name of the Host
• Host Type (In-process or Isolated)
• 32-bits only (Yes or No)
• Tracking Host (Yes or No)
• The name of the Server where the Instance exists
• Name of the Host Instance
• Disabled from starting (Yes or No)

The result is sorted on:
• BizTalk Group
• Host Name
• Server Name

Show BizTalk Applications, Orchestrations, Ports and Pipelines with a SQL query
This query queries the BizTalk Management database. In one view it shows the following information:
• The BizTalk Applications from the BizTalk Group at hand
• The Assemblies which are deployed to the Applications
• The Orchestrations which are in the Applications
• The Status of the Orchestrations:

1. = Unenlisted 
2. = Stopped 
3. = Started

• The Ports as defined in the Orchestrations
• The Port Types
• The Port Type Operations
• The Receive Ports as they are bound to the Orchestrations
• The Receive Locations as they belong to the Receive Ports
• The Receive Pipelines as they are configured at the Receive Locations
• The Send Ports as they are bound to the Orchestrations
• The Send Pipelines as they are configured at the Send Ports
• The Send Port Groups as they are bound to the Orchestrations

The output is sorted on:
• Application
• Assembly
• Orchestration

Number of records in the MarkLog table
This query returns the number of records found in the MarkLog table of your BizTalk databases. This table is filled by the Backup BizTalk Server job, but is not cleaned up, unless you use the Terminator tool. Read this article to purge the records in the MarkLog table.

If you have more than 150.000 records in any of your MarkLog tables, you should use Terminator to purge these records.

The query can be easily extendedwith any custom databases which are also backed up by the Backup BizTalk Server job.

Show BizTalk Subscriptions
This download contains a .SQL query, which can be executed against your BizTalk Message Box. The results you get are the current subscriptions in the Message Box. Only the 'Equals Predicates' subscriptions are shown.
 
The following fields are returned:
• Application - Name of the BizTalk Application which contains the orchestration for this subscription
• Orchestration - Name of the orchestration which has the subscription
• Created On - The timestamp when the subscription was created
• Equals Predicate - The Equals Predicate (mostly the promoted property on a schema)
• Value - The value part from the predicate

Get Orchestration Instances, Counts and States
When this script is being run against a MessageBox it gives you the following information:

• Name of orchestration instances
• Number of orchestration instances
• State of the orchestration instances

The view is grouped by the first 2 columns and sorted on all 3 columns. To make it safe to execute this query against the MessageBoxDb, the WITH(NOLOCK) hint is used.

Number of Messages per MessageType in the TrackingDb
This query gives you insight in what information is stored in the BizTalk TrackingDb. When being run against it, the query shows the following columns:

• Schema
• Number of Messages

The result is grouped by and sorted on the field Schema. To make it safe to execute this query against the MessageBoxDb, the WITH(NOLOCK) hint is used. If you query the BizTalk databases yourself, never forget to add the WITH(NOLOCK) hint.


Hope these queries are helpfull for you. I'll certainly add more queries to TechNet Gallery.

Review of Steef-Jan's BizTalk Server 2010 Cookbook

At BizTalkAdminsBlogging I've written a review about Steef-Jan's first book.

You can find it here.

A flying start for BizTalkAdminsblogging.com

BizTalkadminsblogging has had a tremendous start. Within a week and a half 13 posts already and this will be number 14. An average almost two blog posts a day. This reflects the high demand for BizTalk administration related information, the need for sharing it and desire to create more exposure for this aspect of BizTalk Server. Jeroen and Joris have done an excellent job setting up this blog site, running and maintaining it. The recent posts have touched topics like PowerShell, Microsoft Operation Framework, and so on.

There is also a TechNet Wiki, where some of the bloggers and others have written articles on BizTalk Administration. I, Tord G. Nordahl, Sandro Pereira, Howard S. Edidin and Microsoft have written the following articles that are related to BizTalk administration:

• BizTalk Server 2010 - Tools
• BizTalk Server: Disaster Recovery
• BizTalk Monitoring Tools
• Administering BizTalk 2010 - Roles and Responsibilities
• BizTalk Server 2010: MSDTC
• Installing BizTalk Server 2010 in a Basic Multi-Computer Environment
• BizTalk Server 2010 Host Throttling
• BizTalk Server 2010: SQL Agent jobs
• BizTalk Server - Determine Version and Edition
• BizTalk Server: Host architecture
• BizTalk: List of Errors and Warnings, Causes and Solutions
• BizTalk Administrator Guidelines
• BizTalk Server: High Availability
• BizTalk Databases: Survival Guide
• BizTalk Health Check
• Microsoft BizTalk automatic throttling
• BizTalk Roles
• Recommendations for Installing, Sizing, Deploying, and Maintaining a BizTalk Server Solution
• BizTalk Administrator's Checklist Compiled by Microsoft BizTalk Support
• Common Issues and Resolutions with the BizTalk Server Administration Console
• BizTalk Server - Troubleshooting Problems with MSDTC
• Multiple Message Boxes in BizTalk (Scale out message box)

As you can see that is quite an extensive list (20+ articles).

In the near future Jeroen and I want to organize an event with the Dutch BizTalk User Group that is focused on BizTalk administration. A similar event has already taken place last week. The Ordina BizTalk Innovation Event with topic Administration and Monitoring has been a great success. More than 40 attendees were present at the event. This shows the growing commitment of the community towards BizTalk administration.

TechNet Wiki: The number of BizTalk articles is growing …

A couple of weeks ago I wrote a blog post for the Official blog of TechNet Wiki with the topic “Wiki-Ninjas on Technology: BizTalk Server”. In that blog post I wrote a story on contributions of the community and Microsoft employees for the TechNet Wiki. Mid December there were little of 40 BizTalk Wiki Articles and over a month’s time it has grown to little over 70. That is a tremendous growth of articles (around 30). It show increasing popularity of the TechNet Wiki as a channel of information on Microsoft technology and products.

The newly added articles on BizTalk Server have been written by myself, Tord Glad Nordahl, Sandro Pereira, Howard S. Edidin, and Mick Badran. All these articles (existing and new) are being improved every day by people mentioned here and others (Microsoft employees and the community). I expect the number will grow over time to over 100+ articles as more community members will probably join.

With the wealth of information on TechNet Wiki, MSDN and other Microsoft channels combined you can fully leverage the BizTalk Server platform. You can reach all the BizTalk related wiki articles through BizTalk Server Resources on the TechNet Wiki article.

Enjoy reading any of the articles and hopefully you will find them useful.

BizTalk Processing Monitor in Public Beta

Since this week BizTalk Processing Monitor is available as a Public Beta. It can be obtained here.

Why use BizTalk Processing Monitor
Depending on how you setup BizTalk Processing Monitor, it helps you, amongst others, with the following needs:

• Monitoring and notification of any message flow interruptions
• Get insight in how the BizTalk solution is setup
• Adds value when doing stress or load tests
• Informs you when your BizTalk system is throttling and why
• Informs you in how many messages (in number and in KB, MB or GB) are processed
• Informs you what kind of messages (Message types) are processed
• Informs you about transmission failures

Features
The most important features of BizTalk Processing Monitor are:

• Message flow monitoring with notifications
• Throttling monitoring with notifications
• Singleton Backlog monitoring
• Support for multiple environments
• Integration with other monitoring tools

Documentation
The following documentation is available:

• Getting started with BizTalk Processing Monitor (download)
• How to use the Traffic Light Monitoring (download)
• Overview Keyboard Shortcuts (download)
• Exporting and importing Diagrams (download)

Ordina BizTalk Innovation Event: Monitoring and Administration

I am organizing an event at Ordina on BizTalk Innovation with the topic “Monitoring and Administration”. This is the first event of a series of events under name “Ordina BizTalk Innovation” that will take place at my company Ordina. This and future events are open for customers, the community and Ordina professionals. The event on the 1st of February three speakers will do their presentations on BizTalk Monitoring and administration. During the event I will be the host.

Wouter Crooy, Senior BizTalk Consultant, will have a session on:

Custom Monitoring solutions for BizTalk, ESB Toolkit & WCF

Wouter will during his talk provide a number of custom solutions for monitoring a BizTalk solution and the ESB Toolkit . Using the standard tooling of BizTalk will get you a long way, still with some of the custom monitoring solutions you can have more insight in your own custom BizTalk solutions.

Saravana Kumar, BizTalk MVP, CEO of BizTalk360 will talk on:

Manage your BizTalk Server environment efficiently using BizTalk360

BizTalk 360 is a web based (Silverlight RIA) application primarily designed for supporting and monitoring Microsoft BizTalk Server environments. It addresses some of the common challenges organizations face on running the day to day operations of a BizTalk environment. Some of the key capabilities of BizTalk360 includes:

• Fine grained authorization
• Governance/Audit
• Proactive Monitoring/Notification capabilities
• Graphical Message Flow Viewer for Tracking data
• Various dashboards (Environment, Application, BizTalk Server, SQL Server, Host etc)
• Advanced Event Viewer
• Integrated BAM Portal
• Dynamic topology diagram
• Message Box Viewer (MBV) integration
• Knowledge base repository

There are various other features in addition to the above, that makes BizTalk 360 a must have application for any Microsoft BizTalk Server environments.

Lex Hegt, BizTalk Architect, will have a session on:

Lex will talk on BizTalk monitoring in general and provide an overview on existing tooling in context with BizTalk administration. He will also demonstrate the BizTalk Processing Monitor. This is a tool that, among other things, does (near) real-time monitoring of message flows through BizTalk systems enabling the administrator to quickly identifying issues.

You can register for the event here. The talks of Lex and Wouter will be in Dutch and Saravana’s talk in English. Also joining us during this event will be Tord Grad Nordahl BizTalk expert on BizTalk administration from Bouvet ASA (Norway).

Cheers.