Data validation using the BRE

 Uncategorized  Comments Off on Data validation using the BRE
Oct 292014
 

So we at Stott Creations often get requests to ensure that data is valid flowing through the BizTalk.

The out of the box functionality is pretty straight forward, erectile simply turn on XSD validation in the XML pipelines:

image

The sucky part about that is that it will only surface the first error. Now if your data is really bad, seek I have no desire to send the data in multiple times to get all of the errors.

What we have done is create a product that creates a comprehensive list of errors so you can send the entire list of errors.

So the first thing we have is a list of validation functions that will serve us to create the list of rules to validate the data against.

image

The second set of vocabulary items is the validation patterns:

image

To setup the actual policy, vialis 40mg we need to first setup the ability to Assert the list to the Rule engine. I create the rule as 1 Assert (so it shows up at the top of the list). I drag the run into the conditions pane. I right click the actions and choose assert and then drag List Enumerator into the fact. I also want this to run first, so I set the priority to 2.

image

Now I need to advance the Enumeration. I create 2 Advance (to sit below the 1 Assert). I drag the IEnumerator MoveNext into the Conditions pane. I then right click the actions and choose Assert and drag in the Current IEnumerator into the fact. I then right click the Actions and add update and drag IEnumerator into the fact. Because I want this to run second, I set the priority to 1.

image

Okay, so now we are ready to start creating validation rules. Each rule will run independent of each other, and I really don’t care which order it runs in, I just need them all to run. Each rule that I create in this policy is going to have a priority of 0 (which is the default). Let’s create a rule that checks a format.

I am going to check for two things:

  1. Check the XPath to see if the rule is going to be valid
  2. Check the format to see if it is going to be valid

image

Right click and choose Equal and drag in XPath Statement from the vocabulary

image

I then go to Visual Studio and open up the schema and choose the element or attribute I am attempting to check and copy the xpath statement.

image

And paste it in the right side of the rule

image

Then I drag the Check Format into the AND

image

I then drag Text Value into the first <empty string> slot

image

Now I go to the other Vocabulary and choose the date format I care to validate against, in this case Date with optional century indicator MMDDYYYY and I drag it into the second <empty string> slot. I then have two items, the first one is the regular expression value and the second one is the friendly version. I want to choose the regular expression.

image

The next part is if this is true, meaning, the xpath value is true, and it doesn’t match the regular expression check, I want to create an error. I drag the Create Format Error (Manual) into the actions

image

Now I need to fill in the error information: I need to supply the node type, which is either an attribute or element, I drag Node Type into the <enter a value> section and Node Name into the <empty string>

image

I then drag Text Item into the next <empty string> element, Line Number, and Position into the next two 0 places

image

I then drag from the Patterns vocabulary the Date with optional century indicator MMDDYYYY into the last <empty string> place holder, this time I choose the human readable description (because really, who understands Regular Expressions?)

image

I can continue creating rules this way.

However, there is an easier way:

I create the next rule, creating the If pane the same way, this time I am going to check the SSN. In the THEN, I drag in Create Format Error (Automatic)

image 

I then drag in Validation Information into the <null> (which has all of the details I need (Attribute/Element, node name, text, position, line number, etc)

image

And then drag the Pattern in the <empty string> and choose the friendly explanation.

image

There is tests for data lengths, min, max, and length ranges.

Now to execute it.

First we will show how to do it in .NET

Adding a reference to StottCreations.Validation in the GAC. I then create the following code

  1. I load an XML document
  2. I instantiate a new Record and pass in the xml document
  3. I set the facts (only one) to the Record.
  4. Because I might want to see the trace I new up the DebugTrackingInterceptor
  5. I setup the policy calling the Validation policy I just created.
  6. I then execute the policy.
  7. I also need to clear and dispose the policy (because we have found that the automated garbage collection within .NET is not fast enough).

image

 

When I run the test, I can look at the record, and this is what I see:

image

If I run a valid xml document through, this is the results

image

I can do the same for an orchestration:

image

In the Initialize Variables shape is the following code, again we don’t have the Call Rules shape because we need to dispose immediately.

image

and then in the decision shape (Good)

image

and in the Terminate shape:

image