XSLT performance when mapping large documents in BizTalk

Recently I had to map a document with many thousand rows. I could not split the document because before I could split it, the document’s nodes had to be sorted.

With such large files you generally test it using a small subset to avoid waiting for maps to complete, I built an XSLT which worked great, I thought.

When you use a select filter such as "not(KeyValue=preceding-sibling::row/ KeyValue)" you end up with a huge performance hit the larger the document gets. My map went from 2 seconds for 50 rows to 10 minutes for a few thousand.

How to improve performance when you have large XML files to map that you can’t split? Try using xsl:key instead, which builds an index of keys from which you can much more efficiently select.

Here is a sample XSLT that demonstrates how to use the xsl:key:

<?xml version="1.0" encoding="UTF-8" ?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0" xmlns:ns0="http://Conversion.schemas">

    <xsl:output method="xml" indent="no" />

    <xsl:key name="NumberKey" match="/*[local-name()='top' and namespace-uri()='http://biztalk/Conversion.schemas']/*[local-name()='row' and namespace-uri()='']"

        use="keyValue" />

    <xsl:template match="/">

        <ns0:Rows>

            <xsl:for-each select="/*[local-name()='top' and namespace-uri()='http://biztalk/Conversion.schemas']/*[local-name()='row' and namespace-uri()='' and generate-id(.) = generate-id(key('NumberKey', keyValue)[1])]">

                <xsl:variable name="current_Number" select="keyValue" />

                <Data>

                    <keyValue>

                        <xsl:value-of select="$current_Number" />

                    </keyValue>

                    <xsl:for-each select="//row[keyValue=$current_Number]">

                        <Part>

                            <PartID>

                                <xsl:value-of select="nr_data" />

                            </PartID>

                        </Part>

                    </xsl:for-each>

                </Data>

            </xsl:for-each>

        </ns0:Rows>

    </xsl:template>

</xsl:stylesheet>

Mapping complex conditions in BizTalk with XSLT

Recently I found myself in a situation where I needed to evaluate a set of complex conditions that I could not use the Business Rules Engine for. The BizTalk mapper was able to achieve the conditionality by chaining a series of about 21 functoids together but this became very unmanageable especially since I was combining false statements to see if both were false, then returning a true to an “And” functoid (&) which evaluated another branch as true and passed a result to the node in the destination schema, you get the idea.

An easier way is to plug in XSLT. There are disadvantages to doing this, the main one being its maintainability and transparency but when faced with the daunting set of functoids described above, it looked a decent option.

The XSLT function that I used was the <xsl:choose> statement, inside the statement was a set of conditions which, if true, would return a value/values to the destination schema. In the last part of the statement was the <xsl:otherwise> condition which could set values if the conditions presented were not valid and you had a default you wanted to present in that event.

<xsl:variable name="var:vAnnuity" select="count(/*[local-name()='InternalSupport' and namespace-uri()='http://internal_support']/*[local-name()='LoanPartItems' and namespace-uri()='' and Method='Annuity'])" />

<xsl:variable name="var:vLinear" select="count(/*[local-name()='InternalSupport' and namespace-uri()='http://internal_support']/*[local-name()='LoanPartItems' and namespace-uri()='' and Method='Linear'])" />

<xsl:variable name="var:vTotalCount" select="count(/*[local-name()='InternalSupport' and namespace-uri()='http://internal_support']/*[local-name()='PartItems' and namespace-uri()=''])" />

<xsl:variable name="var:vHasValidEndowment" select="count(/*[local-name()='InternalSupport' and namespace-uri()='http://internal_support']/*[local-name()='LoanPartItems' and namespace-uri()='' and isValid ='true'])" />

<xsl:variable name="var:vHasValidId" select="count(/*[local-name()='InternalSupport' and namespace-uri()='http://internal_support']/*[local-name()='Ready' and HasValidId ='true'])" />

<xsl:element name="LeningSoort">

    <xsl:choose>

        <xsl:when test="$var:vAnnuity = $var:vTotalCount">

            <xsl:value-of select="1" />

        </xsl:when>

        <xsl:when test="$var:vLinear = $var:vTotalCount">

            <xsl:value-of select="2" />

        </xsl:when>

        <xsl:when test="($var:vHasValidEndowment = $var:vTotalCount) and ($var:vHasValidId = $var:vTotalCount)">

            <xsl:value-of select="3" />

        </xsl:when>

        <xsl:otherwise>

            <xsl:value-of select="4" />

        </xsl:otherwise>

    </xsl:choose>

</xsl:element>

Of course each specific case requires different XSLT, but this should provide a framework of how to do this within a functoid.

To set up the XSLT:

• Create XSLT similar to the one above which meets your needs
• Drop a scripting functoid on your map
• Connect the output parameter to the node you want to create
• Note: do no include any input parameters, the XSLT is setup already to look them up
• Open the scripting functoid’s, click the (…) inside “Configure Functoid Script”
• Under “Script type” choose “Inline XSLT”
• Paste your XSLT snippet in the text box

Remove empty nodes in BizTalk by using XSLT

If you have ever had a map which begins to look unmanageable with tons of value mappings or extra functoids to simply manage empty nodes you don’t want in your output then you should consider using an XSLT mapping to clean the document up for you.

This involves actually making 2 mappings instead of one, but the upside is faster development with a nominal performance cost. Instead of adding functoids, testing and retesting the output, let the mapper generate the empty nodded and then add an extra mapping step with the following XSLT embedded in it:

<?xml version="1.0" ?>

<xsl:stylesheet xmlns:xsl="@@ YOUR NAMESPACE @@" version="1.0" xmlns:ns0="http://Stater.Isvcs.Iface.BO.GetLoanData.ElfV2">

    <xsl:output method="xml" indent="yes" />

    <xsl:template match="node()">

        <xsl:if test="count(descendant::text()[string-length(normalize-space(.))>0]|@*)">

            <xsl:copy>

                <xsl:apply-templates select="@*|node()" />

            </xsl:copy>

        </xsl:if>

    </xsl:template>

    <xsl:template match="@*">

        <xsl:copy />

    </xsl:template>

    <xsl:template match="text()">

        <xsl:value-of select="normalize-space(.)" />

    </xsl:template>

</xsl:stylesheet>

Save this snippet (or download it here) into your project (open the XSLT and change the namespace!) then create a new map. Make your source and destination schemas the same and in the map properties add your XSLT in the “Custom XSLT path” setting. Place the new map after the “dirty” map; the output should give you an XML document free of extra nodes and a map free of extra functoids.

Of course, if you are doing a strictly messaging based implementation then you are out of luck unless you chain ports together but within an orchestration using XSLT to remove empty nodes might work well for you.