What is a validation rule?
A validation rule is a.check of data in relation to other data in a message. If a single attribute needs to meet certain conditions (e.g. a minimum or a maximum value), a check on the schema is sufficient. But sometimes the value of an attribute depends on the value of one or more other attributes, as shown in the examples below. Conditions like these involve more than one attribute and can be checked with a validation rule.
Example 1
In case of a coverage for third party liability of a motorvehicle, an insurer can determine that a surplus to the premium is applicable, if the regular driver is under 24.
Example 2
In case of vehicle hull coverage, an insurer can determine that, if the insured sum is higher than 40.000, the deductible amount should be higher than 500.
Technical use validation rules
Every financial product or service contains a set of rules, applied to the data elements associated with the product or process. The AFD-definition Standard provides a framework and tooling to record specifications and checks, as well as checks at data element level. Validation rules in the AFD-definition Standard make use of XPath (AFD 1.0) and JMESPath (AFD 2.0) to ensure that a financial product is put into production accurately and efficiently.
Parties that have set up their own business rules engine environments, can convert validation rules to their own environment.
JMESPath (AFD 2.0)
JMESPath is a query language designed for JSON documents. Unlike XPath, which is tailored for XML, JMESPath is specifically crafted for navigating and querying JSON data. JMESPath expressions enable the extraction of specific elements or values from JSON documents, providing a standardized and efficient means of querying JSON data.
Reference: [JMESPath Documentation] (https://jmespath.org)
JMESPath is particularly valuable in scenarios where JSON is the primary data format, such as in web APIs and NoSQL databases. Its simplicity and flexibility make it a powerful tool for querying and transforming JSON data across various applications and platforms.
Ready-made libraries
The JMESPath specification is implemented in various languages, like Python or .Net. On the website of JMESPath you can find a list of JMESPath libraries that can be used by software developers (https://jmespath.org/libraries.html)
Testing JMESPath expressions
To make sure the validation rules themselves are correct, it is recommended to use an expression tester. For example:
- The original (single-line) JMESPath tester (https://jmespath.org/)
- A multi-line expression tester (https://mixedanalytics.com/tools/jmespath-expression-tester/
Validation rules within the AFD-definition Standard
JMESPath can be employed to establish unambiguous specifications, allowing for the examination of dependencies between multiple data elements within a single AFD message or database. Within the AFD-definition Standard, the JMESPath expression must always take the form of a conditional expression, yielding either a ‘true’ or ‘false’ result.
A JMESPath validation rule within the AFD-definition Standard is constructed as follows:
- A commonFunctional entity with the identifying characteristics of the document
- The actual AFD validation rules
An example of a validation rule is available in JSON Examples (JMESPath).
The actual validation rule contains the following elements:
- id: an element containing a rule ID of the validation rule.
- test: the validation rule written as a JMESPath (AFD 2.0) expression.
- message: an element containing the error message to be displayed in case the executed rule returns a ‘false’.
- source: an element containing documentation on the validation rule.
The validation rules within the AFD-definition Standard appear as a conditional expression:
if (‘logical expression’ then ‘logical expression’ else true())
Validation rules in JMESPath (AFD 2.0)
The validation rules for AFD 2.0 can be found in validationRules.json. A validation rule (in JMESPath) contains the following elements:
- rule id: JSON element containing a rule ID of the AFD 2.0 validation rule and the JMESPath-expression.
- test: the validation rule written as JMESPath expression.
- message: an optional element containing the error message to be displayed if the rule executed returns a ‘false’.
- source: an optional element containing a reference to documentation on the validation rule.
Below is an example in JMESPath within the AFD-definition Standard for the validation rule ‘If there is a coverage with entityType hullVehicle on the policy, there must also be an object with entityType motorVehicle on the policy, with the attribute initialListPrice set to a value greater than 0’.
"rule": [
{
"id": "Verbandscontrole 1",
"test": "!((policy[].coverage[? entityType == 'hullVehicle'])[] !=`[]`)||((policy[].object[? entityType == 'motorVehicle' && initialListPrice >`0`])[] !=`[]`)",
"message": "In case there is a coverage for hull vehicle, there must be a motor vehicle and the initial listprice must be greater than '0'.",
"source": "Manual par 1.4"
}
]
SIVI defined guidelines for the composition of AFD 2.0 validation rules. These are shown in the table below, using the example mentioned before.
Guidelines for the composition of AFD 2.0 validation rules
Name | Description |
Conditional expression | A validation rule should always be a conditional expression (yielding either ‘true’ or ‘false’). |
First condition | The condition !((policy[].coverage[? entityType ‘hullVehicle’])[] !=’[]’) signifies that if at least one coverage with the entityType ‘hullVehicle’ exists, the second condition should be evaluated. If no coverage with entityType ‘hullVehicle’ exists, the first condition is true, and consequently, the entire expression is true. Therefore, the second condition does not need to be validated further. |
Second condition | The second condition, ((policy[].object[? entityType ‘motorVehicle’ && initialListPrice > 0])[] !=’[]’), specifies that there must be an object with the entityType ‘motorVehicle’ and an attribute initialListPrice with a value greater than zero. If this condition is met, the expression evaluates to true. Conversely, if this condition is not met, the expression evaluates to ‘false’. |
Pipes || | This signifies ‘or’ and separates the two conditions. If either of the conditions is true, the entire expression evaluates to ‘true’. |
Arithmetic operators | Unlike XPath, arithmetic operators do not need to be ‘escaped’ in JMESPath. See the table below for options for arithmetic operators. |
Use of arithmetic operators in JMESPath
Arithmetic operator | Operator AFD 2.0 JMESPath |
> | > |
< | < |
>= | >= |
<= | <= |
AND | && |
OR | || |
NOT | ! |
EQUAL | == |
NOT EQUAL | != |
Examples of validation rules in AFD 2.0
Description | AFD 2.0 Validation rule |
If attribute A then attribute B If sumInsured greater than 100000, then the deductibleAmount should be greater than 500. |
!((policy[].coverage[? entityType == ‘hullVehicle’ && sumInsured >= `1000000`])[] !=`[]`) || ((policy[].coverage[? entityType == ‘hullVehicle’ && deductibleAmount == `500`])[] != `[]`) |
If attribute A and B then attribute C If sumInsured greater than 100000 and coverageAreaCode is equal to ‘E’, then the deductibleAmount should be greater than 500. |
!((policy[].coverage[? entityType == ‘hullVehicle’ && sumInsured >= `1000000`])[] !=`[]`) || ((policy[].coverage[? entityType == ‘hullVehicle’ && deductibleAmount == `500`])[] != `[]`) |
If attribute A then attribute B and C If coverageAreaCode is equal to ‘E’ then sumInsured should be greater than 30000 and deductibleAmount should be greater than 1000 |
!((policy[].coverage[? entityType == ‘hullVehicle’ && coverageAreaCode == ‘E’])[] !=`[]`) || ((policy[].coverage[? entityType == ‘hullVehicle’ && sumInsured > `30000` && deductibleAmount > `1000`])[] != `[]`) |
If entity A exists then entity B should exist. If premiumPayer exists, then a policyHolder should exist. |
!((policy[].party[? entityType == ‘premiumPayer’])[] !=`[]`) || ((policy[].party[? entityType == ‘policyHolder’])[] !=`[]`) |
If entity A exists then entity B should not exist. If mortgage exists, then lease should not exist. |
!((loan[? entityType == ‘mortgage’])[] !=`[]`) || ((loan[? entityType == ‘lease’])[] ==`[]`) |
Custom functions in JMESPath
The remainder of this paragraph describes how to specify a custom function, how to use it in JMESPath, and how to implement it in a .NET environment. It explains the application of JMESPath custom functions, categorized by target users: business analyst, architect, and software supplier.
It starts with a JMESPath custom functions introduction, intended for all users. The next section of this paragraph focuses on the business analyst and architect, providing a framework for the specification and use of custom functions. The last section is intended for the software supplier, illustrating how to apply custom functions.
Built-in vs. custom JMESPath function
JMESPath includes many built-in functions, such as abs, avg, starts_with, length
, etc. These are part of the JMESPath language and are therefore recognized by online query testers, like JMESPath Query Tester.
In addition to the JMESPath built-in functions, it is also possible to define custom function. Note that custom functions are not recognized by online tools because they are not part of the standard JMESPath language. Custom functions are useful in JMESPath-supported platforms, such as the .NET platform.
Example
This document describes the implementation of the custom function ageAtReferenceDate
. This function returns the difference in whole years between two date fields: a subject date and a reference date. Thus, the function provides the age of a person on a specific reference date, given a birthdate as the subject date.
Custom function specification
JMESPath built-in functions have a description, a signature definition, and a functional description of the function. The signature definition describes the input parameter type(s) and return type. The functional description outlines what the function does and the expected output for a given input. If necessary, this is further detailed. Finally, a specification includes one or more examples. Preferably, the specification language is English.
We can specify the custom function ageAtReferenceDate as follows
number ageAtReferenceDate(string $subjectdate, string $referencedate)
Returns the exact age in full years from the given $subjectdate to the specified $referencedate.
Both $subjectdate and $referencedate should be in the format ‘YYYY-MM-DD’. Below are some examples:
Given @ | Expression | Result |
‘2006-07-07’ | ageAtReferenceDate(@,‘2024-07-06’) | 17 |
‘2006-07-07’ | ageAtReferenceDate(@,‘2024-07-07’) | 18 |
‘2006-07-07’ | ageAtReferenceDate(@,‘2024-07-08’) | 18 |
In the example above, the custom function ageAtReferenceDate
has two parameters, type string, each with its own format. The first parameter is the current node, such as the birthdate of the subject whose age we want to determine. The second parameter is the reference date. This can be a fixed string value or a reference to another node.
The JMESPath community, tab Functions, contains more specifications (click here to get more infomation).
Use of custom functions within the JMESPath expression
An example of this custom function within a JMESPath expression looks as follows:
(policy[].party[?entityType == ‘policyHolder’ && birthDate.ageAtReferencedDate(@, $.commonFunctional0.referenceDate) >= `18`][] != `[]`)
This expression uses @ as the current-node token and $ as the root-node, which allows referencing a node outside the current node. For more information about these tokens, go to https://jmespath.site/main/#spec-examples-of-this-new-syntax.
Defining and managing custom functions
Key considerations when defining and managing custom functions are quality, consistency, and coherence. This ensures the creation of an intuitive library of custom functions. In addition, thorough documentation of the custom functions is essential, serving as a link between the business and software providers.
JMESPath for business analyst and architect
Example
From a business perspective, there is a need for a custom function that can determine the age in years of an object or person at a specific reference point of time. An additional requirement is the ability to retrieve the reference point of time from another message data.
Based on this functional description, a custom function specification should be drafted that answers the following questions:
- What is the name of the function?
- What parameters are required?
- What does the function return?
- What is the function signature?
- What is the description of the function?
- Are examples available?
- What is the complete specification of the function?
- How is this function applied in JMESPath?
Each question is described in the text below.
Custom function Name
The name of the function reflects what it returns/defines. In this case, it is the age on a reference date. Function names are always in camelCase; they always start with a lowercase letter, and subsequent words start with an uppercase letter. The name is a description of what the function provides (thus, not a verb form).
An obvious name in this case is: ageAtReferenceDate
(and not calculateAgeAtReferenceDate
)
Custom function Parameters
ageAtReferenceDate
is based on two dates; the date of the subject whose age needs to be determined and the reference date. Since in AFD 2.0 messages date fields always have the “YYYY-MM-DD” format, the parameters need to be of type string. In this case, the parameters are: string $subjectdate
and string $referencedate
.
Custom function Return value
In the example, an age is returned in whole years. To be able to perform calculations with the return value, such as determining that a person is at least 18 years old, the return type (number) is required in this case.
Custom function Signature
A function has a signature consisting of the name, the parameters, and the return value. In this case:
number ageAtReferenceDate(string $subjectdate, string $referencedate)
Custom function Description
The description of the function is what the function returns, optionally supplemented with format descriptions, for example:
Returns the exact age in full years from the given $subjectdate to the specified $referencedate.
Both $subjectdate and $referencedate should be in the format 'YYYY-MM-DD'.
Custom function Examples
To get a good insight, it is highly recommended to include one or more examples (similar to built-in JMESPath functions). The table under the next header contains some examples.
Custom function Specification
With all data collected before, it is now possible to compose the custom function Specification:
number ageAtReferenceDate(string $subjectdate, string $referencedate)
Returns the age in full years of argument $subjectdate at argument $referencedate.
Both $subjectdate and $referencedate are to be specified as ‘YYYY-MM-DD’. Below are some examples:
Given @ | Expression | Result |
‘2006-07-07’ | ageAtReferenceDate(@,‘2024-07-06’) | 17 |
‘2006-07-07’ | ageAtReferenceDate(@,‘2024-07-07’) | 18 |
‘2006-07-07’ | ageAtReferenceDate(@,‘2024-07-08’) | 18 |
Note: @ is the current-node
Remark: current-node means: the node of the subject. In the example under the next header, the current-node is the policyHolder
’s birthDate
Custom function application in JMESPath
Because examples are helpful to understand how to apply custom functions, two examples are presented below:
Example 1 – the second parameter has a fixed value
(policy[].party[?entityType == 'policyHolder' && birthDate.ageAtReferenceDate (@, '2024-07-07') >= `18`][] != `[]`)
Example 2 – the second parameter refers to another attribute in the message
(policy[].party[?entityType == 'policyHolder' && birthDate.ageAtReferenceDate(@, $.commonFunctional[0].referenceDate) >= `18`][] != `[]`)
JMESPath for the software supplier
The software supplier will apply custom functions in JMESPath expressions based on custom function specifications designed by the business analyst or architect. The software supplier will be able to implement custom functions into the according platform and use them in JMESPath expressions.
.NET implementation structure
JMESPath libraries are available for many platforms. This chapter demonstrates how to apply the example custom function ageAtReferenceDate
in a .NET platform (web server). Below is an elaboration of this example, based on the JmesPath.Net ‘Using functions’ example. More information about JMESPath.net is available on https://jdevillard.github.io/JmesPath.Net/. The remainder of this chapter provides a brief explanation of the JMESPath example and a .NET implementation of this example.
Each custom function has its own class
In .NET, each custom function has its own C# class. This class contains the following components
- a custom function definition (name and number of parameters)
- method
Validate()
contains input parameter checks. - method
Execute()
contains the actual code of the custom function.
Registration of the custom function
A custom function must be registered before it can be used.
var jmes = new JmesPath();
jmes.FunctionRepository
.Register<SubstringFunction>()
;
Custom function as part of a JMESPath expression
Just like other built-in JMESPath functions, a custom function can be part of a JMESPath expression, for example:
birthdate.ageAtReferenceDate(@) >= `18`
Use the # JmesPath.Transform() method to execute a JMESPath expression, for example:
var result = jmes.Transform(input, expression);
Explanation:
- input: the AFD message (api-payload)
- expression: the JMESPath expression, including the custom function
- result: the result of the expression
A .NET C# implementation
This section shows an implementation of the ageAtReferenceDate
example in a .NET C# application, based on the description in the section above.
Custom function class
Figure 3.4.1 Example of a custom function class (click on the image to open)
Custom function registration and use
This section shows a complete .NET C# web-api controller class, including the registration of the custom function and its usage. The two embedded test resources submitPolicy_new_request.json
and submitPolicy_new_request_schema.json
can be found under the header Embedded Resources.
Figure 3.4.2 Example of a custom function registration and use (click on the image to open)
The JMESPath custom function-specific The components, specific for JMESPath custom functions are highlighted in the subsequent sections.
Registration of custom function
Figure 3.4.3 Example of a custom function registration (click on the image to open)
JMESPath expression with custom function
Figure 3.4.4 Example of a JMESPath expression with custom function (click on the image to open)
Calling the JMESPath expression
Figure 3.4.5 Calling the JMESPath expression (click on the image to open)
Embedded resources
To test the web API controller, the following embedded test resources are included. In practice, these are the request payload and the AFD definition schema.
submitPolicy_new_request.json
Figure 3.4.6 Example of an embedded test resource for a request payload (click on the image to open)
submitPolicy_new_request_schema.json
Figure 3.4.7 Example of an embedded test resource for a request AFD-definition schema (click on the image to open)
Post your comment on this topic.