Validation rules in JMESPath - AFD-definition Standard

1. What is a validation rule?

A validation rule is a.check of data in relation to other data in a message. If a single attribute needs to meet certain conditions (e.g. a minimum or a maximum value), a check on the schema is sufficient. But sometimes the value of an attribute depends on the value of one or more other attributes, as shown in the examples below. Conditions like these involve more than one attribute and can be checked with a validation rule.

Example 1

In case of a coverage for third party liability of a motorvehicle, an insurer can determine that a surplus to the premium is applicable, if the regular driver is under 24.

Example 2

In case of vehicle hull coverage, an insurer can determine that, if the insured sum is higher than 40.000, the deductible amount should be higher than 500.

2. Technical use validation rules

Every financial product or service contains a set of rules, applied to the data elements associated with the product or process. The AFD-definition Standard provides a framework and tooling to record specifications and checks, as well as checks at data element level. Validation rules in the AFD-definition Standard make use of XPath (AFD 1.0) and JMESPath (AFD 2.0) to ensure that a financial product is put into production accurately and efficiently.

Parties that have set up their own business rules engine environments, can convert validation rules to their own environment.

JMESPath (AFD 2.0)

JMESPath is a query language designed for JSON documents. Unlike XPath, which is tailored for XML, JMESPath is specifically crafted for navigating and querying JSON data. JMESPath expressions enable the extraction of specific elements or values from JSON documents, providing a standardized and efficient means of querying JSON data.

Reference: [JMESPath Documentation] (https://jmespath.org)

JMESPath is particularly valuable in scenarios where JSON is the primary data format, such as in web APIs and NoSQL databases. Its simplicity and flexibility make it a powerful tool for querying and transforming JSON data across various applications and platforms.

Ready-made libraries

The JMESPath specification is implemented in various languages, like Python or .Net. On the website of JMESPath you can find a list of JMESPath libraries that can be used by software developers (https://jmespath.org/libraries.html)

Testing JMESPath expressions

To make sure the validation rules themselves are correct, it is recommended to use an expression tester. For example:

The original (single-line) JMESPath tester (https://jmespath.org/)
A multi-line expression tester (https://mixedanalytics.com/tools/jmespath-expression-tester/

3. Validation rules within the AFD-definition Standard

JMESPath can be employed to establish unambiguous specifications, allowing for the examination of dependencies between multiple data elements within a single AFD message or database. Within the AFD-definition Standard, the JMESPath expression must always take the form of a conditional expression, yielding either a ‘true’ or ‘false’ result.

A JMESPath validation rule within the AFD-definition Standard is constructed as follows:

A commonFunctional entity with the identifying characteristics of the document
The actual AFD validation rules

An example of a validation rule is available in JSON Examples (JMESPath).

The actual validation rule contains the following elements:

id: an element containing a rule ID of the validation rule.
test: the validation rule written as a JMESPath (AFD 2.0) expression.
message: an element containing the error message to be displayed in case the executed rule returns a ‘false’.
source: an element containing documentation on the validation rule.

The validation rules within the AFD-definition Standard appear as a conditional expression and consists of two parts:

The first condition A should be true.
The second condition B should be true, if the first condition is true

The validation rules within the AFD-definition Standard appear as a conditional expression:

if (‘logical expression A’ then ‘logical expression B’ else true())

4. Validation rules in JMESPath (AFD 2.0)

The validation rules for AFD 2.0 can be found in the file validationRules.json. A validation rule (in JMESPath) contains the following elements:

rule id: JSON element containing a rule ID of the AFD 2.0 validation rule and the JMESPath-expression.
test: the validation rule written as JMESPath expression.
message: an optional element containing the error message to be displayed if the rule executed returns a ‘false’.
source: an optional element containing a reference to documentation on the validation rule.

Below is an example in JMESPath within the AFD-definition Standard for the validation rule ‘If there is a coverage with entityType hullVehicle on the policy, there must also be an object with entityType motorVehicle on the policy, with the attribute initialListPrice set to a value greater than 0’.

"rule": [
    {
      "id": "Verbandscontrole 1",
      "test": "!((policy[].coverage[? entityType == 'hullVehicle'])[] !=`[]`)||((policy[].object[? entityType == 'motorVehicle' && initialListPrice >`0`])[] !=`[]`)",
      "message": "In case there is a coverage for hull vehicle, there must be a motor vehicle and the initial listprice must be greater than '0'.",
      "source": "Manual par 1.4"
    }
  ]

SIVI defined guidelines for the composition of AFD 2.0 validation rules. These are shown in the table below, using the example mentioned before.

Guidelines for the composition of AFD 2.0 validation rules

Name	Description
Conditional expression	A validation rule should always be a conditional expression (yielding either ‘true’ or ‘false’).
First condition	The condition !((policy[].coverage[? entityType ‘hullVehicle’])[] !=’[]’) signifies that if at least one coverage with the entityType ‘hullVehicle’ exists, the second condition should be evaluated. If no coverage with entityType ‘hullVehicle’ exists, the first condition is true, and consequently, the entire expression is true. Therefore, the second condition does not need to be validated further. For a valid validation rule apostrophes should be used instead of quotation marks.
Second condition	The second condition, ((policy[].object[? entityType ‘motorVehicle’ && initialListPrice > 0])[] !=’[]’), specifies that there must be an object with the entityType ‘motorVehicle’ and an attribute initialListPrice with a value greater than zero. If this condition is met, the expression evaluates to true. Conversely, if this condition is not met, the expression evaluates to ‘false’.
Pipes \|\|	This signifies ‘or’ and separates the two conditions. If either of the conditions is true, the entire expression evaluates to ‘true’.
Arithmetic operators	Unlike XPath, arithmetic operators do not need to be ‘escaped’ in JMESPath. See the table below for options for arithmetic operators.

Use of arithmetic operators in JMESPath

Arithmetic operator	Operator AFD 2.0 JMESPath
>	>
<	<
>=	>=
<=	<=
AND	&&
OR	\|\|
NOT	!
EQUAL	==
NOT EQUAL	!=

Examples of validation rules in AFD 2.0

Description	AFD 2.0 Validation rule
If attribute A then attribute B If sumInsured greater than 100000, then the deductibleAmount should be greater than 500.	!((policy[].coverage[? entityType == ´hullVehicle´ && sumInsured >= ´100000´])[] !=´[]´) \|\| ((policy[].coverage[? entityType == ´hullVehicle´ && deductibleAmount == ´500´])[] != ´[]´)
If attribute A and B then attribute C If sumInsured greater than 100000 and coverageAreaCode is equal to ‘E’, then the deductibleAmount should be greater than 500.	!((policy[].coverage[? entityType == ´hullVehicle´ && sumInsured >= ´100000´==&&== coverageAreaCode == ´E´])[] !=´[]´) \|\| ((policy[].coverage[? entityType == ´hullVehicle´ && deductibleAmount == ´500´])[] != ´[]´)
If attribute A then attribute B and C If coverageAreaCode is equal to ‘E’ then sumInsured should be greater than 30000 and deductibleAmount should be greater than 1000	!((policy[].coverage[? entityType == ´hullVehicle´ && coverageAreaCode == ´E´])[] !=´[]´) \|\| ((policy[].coverage[? entityType == ´hullVehicle´ && sumInsured > ´30000´ && deductibleAmount > ´1000´])[] != ´[]´)
If entity A exists then entity B should exist. If premiumPayer exists, then a policyHolder should exist.	!((policy[].party[? entityType == ´premiumPayer´])[] !=´[]´) \|\| ((policy[].party[? entityType == ´policyHolder´])[] !=´[]´)
If entity A exists then entity B should not exist. If mortgage exists, then lease should not exist.	!((loan[? entityType == ´mortgage´])[] !=´[]´) \|\| ((loan[? entityType == ´lease´])[] ==´[]´)

5. Use of custom functions in JMESPath

JMESPath includes a variety of built-in functions, such as abs, avg, starts_with and length. These built-in functions do not cover all the requirements needed for validation rules in the AFD-definition Standard. It is possible to solve this issue by using custom functions with JMESPath.
Users can create custom functions in JMESPath themselves using libraries in their own language, like Python or .Net. On the website of JMESPath you can find a list of JMESPath libraries that can be used by software developers (https://jmespath.org/libraries.html).

6. AFD custom functions in JMESPath

Developing custom functions takes time and can be complex. In addition, there is a risk that parties develop different custom functions for the same functionality. To prevent this, SIVI provides a number of standard custom functions. These are called AFD custom functions. SIVI will develop custom functions for the most common missing functionalities in JMESPath.

SIVI has collected and developed the specifications for the following custom functions:

acfCountDateDiff – calculates the difference between two date fields
acfDateAdd – adds a certain number of units to a date
acfSubString – extracts a specified number of contiguous characters from an attribute
acfTrim – removes specified characters from an attribute

A further elaboration of these AFD custom functions can be found at the bottom of this page

Please contact SIVI, if the custom function you need is not yet included in this chapter. SIVI will develop the custom function, if it is sufficiently generic.

Note that custom functions are not recognized by online tools since they are not part of the standard JMESPath language. However, they are useful in JMESPath-compatible platforms, such as the .NET platform.

This section describes how to define and use custom functions. It starts with an introduction to JMESPath custom functions, followed by a framework for specifying and implementing them.

AFD custom function specification

JMESPath built-in functions include a description, a signature definition, and a functional description. The signature definition specifies the input parameter type(s) and return type. The functional description outlines what the function does and the expected output for a given input with additional details provided if needed. Finally, a specification includes one or more examples.

For more specifications, visit the Functions tab on the JMESPath community (click here to get more infomation).

AFD custom function Name

The name of the function clearly reflects what it returns or defines. Function names are always presented in camelCase; they start with a lowercase letter, and subsequent words start with an uppercase letter. The name is a description of what the function provides.

AFD custom function Parameters

AFD custom functions contain five input parameters:

Item – Name of the input parameter in camelCase.
Description – A brief explanation of the input parameter
Type – Defines the data types the parameter can accept, such as numbers, string or dates.
Format – Defines how the data is presented, without altering its underlying precision (e.g. YYYY-MM-DD for dates).
Option list – Lists the variables that can be selected for this input parameter.

AFD Custom function Return value

An AFD custom function includes a single return value with the following characteristics:

Item – The name of the return value written in camelCase.
Description – A brief explanation of the return value.
Type – Defines the data types the return value can hold such as numbers, strings or dates.
Format – Defines how the return value is displayed, without affecting its underlying precision (e.g. YYYY-MM-DD for dates).

AFD custom function Signature

An AFD custom function has a signature consisting of the name, the parameters, and the return value.

AFD custom function Examples

To get a good insight, it is highly recommended to include one or more examples (similar to built-in JMESPath functions).

7. Currently used AFD custom functions

In this section the custom functions currently in use for AFD are described, in accordance with the layout used above.

acfCountDateDiff

AFD custom function specification

This AFD custom function calculates the difference between two date fields in a chosen calendar unit (day, week, month or year). The input parameters are calendar unit, subject date and reference date. The output parameter returns a number in a selected calendar unit, where the result is truncated downwards without rounding.

AFD custom function Name

The name of this AFD custom function is acfCountDateDiff

AFD custom function Parameters

Input parameters

Item	Description	Type	Format	Option list
calendarUnit	Calendar unit	string		day, week, month, year
subjectDate	Starting date	date	‘YYYY-MM-DD’
referenceDate	Reference date	date	‘YYYY-MM-DD’

AFD custom function Return value

Return value

Item	Description	Type	Format
acfCountDateDiff	Number of whole units in the selected calendar unit, truncated without rounding	number

AFD custom function Signature

The signature of acfCountDateDiff looks like this: number acfCountDateDiff (string $subjectDate, string $referenceDate, string $calendarUnit).

AFD custom function Examples

subjectDate (@)	Expression (subjectDate, referenceDate)	Return value
‘2006-07-07’	acfCountDateDiff (@,‘2024-07-06’, ‘year’)	17
‘2006-07-07’	acfCountDateDiff (@,‘2024-07-07’, ‘year’)	18
‘2006-07-07’	acfCountDateDiff (@,‘2024-07-08’, ‘year’)	18
‘2017-08-25’	acfCountDateDiff (@, ‘2011-08-25’, ‘month’)	-72
‘2020-08-01’	acfCountDateDiff (@, ‘2024-08-25’, ‘year’)	4
‘2021-08-01’	acfCountDateDiff (@, ‘2024-01-25’, ‘year’)	2

Remarks:

The sign ‘@’ refers to the current node within the data. This means that ‘@’ represents the current value of the element (in this case subjectDate).
If the referenceDate comes before the suibjectDate, the return value is negative.

AFD custom function Implementation in JMESPath

The required JMESPath-expression using an AFD custom function may look like the examples below:

Example 1 – the second parameter has a fixed value

(policy[].party[?entityType == 'policyHolder' && birthDate.acfCountDateDiff (@, '2024-07-07', ‘year’) >= `18`][] != `[]`)

Example 2 – the second parameter refers to another attribute in the message

In this example referenceDate refers to the attribute referenceDate which belongs to the entity commonFunctional.

(policy[].party[?entityType == 'policyHolder' && birthDate.acfCountDateDiff (@, $.commonFunctional[0].referenceDate, ‘year’) >= `18`][] != `[]`)

acfDateAdd

AFD custom function specification

This AFD custom function adds a certain number of units in a chosen calendar unit (day, week, month or year) to a date. The result is the calculated date. The attributes applied are calendar unit, number and subject date.

AFD custom function Name

The name of this AFD custom function is acfDateAdd.

AFD custom function Parameters

Input parameters

Item	Description	Type	Format	Option list
subjectDate	Starting date	date	‘YYYY-MM-DD’
calendarUnit	Calendar unit	string		day, week, month, year
number	Number of calendar units to add to a date. May also be negative.	number	integer

AFD custom function Return value

Return value

Item	Description	Type	Format
acfDateAdd	Calculated date from date (starting value) plus number of selected calendar unit (day, week, month, year)	string

AFD custom function Signature

The signature of acfDateAdd looks like this: string acfDateAdd (string $subjectDate, number $number, string $calendarUnit)

AFD custom function Examples

subjectDate (@)	Expression (subjectDate, referenceDate)	Return value
‘2024-10-09’	acfDateAdd (@, 2, year)	2026-10-09
‘2024-10-09’	acfCountDateDiff (@, -2, ‘year’)	2022-10-09
‘2024-12-12’	acfCountDateDiff (@, 100, ‘year’)	2025-03-21

Remarks:

The sign ‘@’ refers to the current node within the data. This means that ‘@’ represents the current value of the element (in this case subjectDate).
If the input parameter is a negative number, the calculated date acfDateAdd comes before the subjectDate.

AFD custom function Implementation in JMESPath

An example will follow soon.

acfSubString

AFD custom function specification

This AFD custom function extracts a specified number of contiguous characters from an attribute value. The starting position and the number of characters to extract are provided as input parameters.

AFD custom function Name

The name of this AFD custom function is acfSubString

AFD custom function Parameters

Input parameters

Item	Description	Type	Format	Option list
subjectText	Attribute value	string
startCharacter	Starting position of the character	number
numberOfCharacters	Number of characters	number

AFD custom function Return value

Return value

Item	Description	Type	Format
acfSubString	Number of contiguous characters from an attribute value	string

AFD custom function Signature

The signature of acfSubString looks like this: string acfSubString (string $subjectText, number $startCharacter, number $numberOfCharacters)

AFD custom function Examples

subjectText (@)	Expression (subjectText, startCharacter, numberOfCharacters)	Return value
‘customerDetails’’	acfSubString (@, 9, 7)	‘Details’
‘customerDetails’’	acfSubString (@, 1, 3)	‘cus’

Remarks:

The sign ‘@’ refers to the current node within the data. This means that ‘@’ represents the current value of the element (in this case subjectDate).

AFD custom function Implementation in JMESPath

The required JMESPath expression using an AFD custom function may look like the example below:

Example
The motorvehicle must have a constructionYear, that is no more than five years earlier than year of the effectiveDate:

policy[].object[?constructionYear < `-5` + $.policy[0].effectiveDate.acfSubString(@,1,4).to_number(@) ][] == `[]`

The logic ensures that the set of motor vehicles failing validation is empty. The result of the expression is true unless an object’s constructionYear is earlier than 2019 (assuming effectiveDate is in 2024).

acfTrim

AFD custom function specification

Each AFD custom function removes specified characters from an attribute. If no characters are specified, spaces are removed by default.

The first function removes the specified characters from both the beginning and the end.
The second function removes the specified characters only from the beginning.
The third function removes the specified characters only from the end.

AFD custom function Name

The names of these AFD custom functions are:

acfTrim – removes specified characters from start and end
acfTrimLeft – removes specified characters from the start
acfTrimRight – emoves specified characters from the end

AFD custom function Parameters

Input parameters

Item	Description	Type	Format	Option list
subject	Attribute value	string
[chars]	Characters	string

AFD custom function Return value

Return value

Item	Description	Type	Format
acfTrim	Attribute value without specified characters or spaces from the beginning and end.	string
acfTrimLeft	Attribute value without specified characters or spaces from the beginning.	string
acfTrimRight	Attribute value without specified characters or spaces from the end.	string

AFD custom function Signature

The AFD custom function Signature depends on the AFD custom function.

For acfTrim – string acfTrim(string $subject [, string $chars])
For acfTrimLeft – string acfTrimLeft(string $subject [, string $chars])
For acfTrimRight – string acfTrimRight(string $subject [, string $chars])

AFD custom function Examples

The tables below contain examples of the AFD custom functions for acfTrim, acfTrimLeft and acfTrimRight.

Remarks:

The sign ‘@’ refers to the current node within the data. This means that ‘@’ represents the current value of the element (in this case subject).

acfTrim

subject (@)	Expression	Return value
‘ subject string ‘	acfSTrim(@)	‘subject string’
‘ subject string ‘	acfSTrim(@, ‘’)	‘subject string’
‘ subject string ‘	acfSTrim(@, ‘ ‘)	‘subject string’
‘ subject string ‘	acfSTrim(@, ‘su’)	‘bject string’
‘ subject string ‘	acfSTrim(@, ‘su ‘)	‘bject string’
‘11WWGG’	acfSTrim(@, ‘0123456789’)	‘WWGG’

acfTrimLeft

subject (@)	Expression	Return value
‘ subject string ‘	acfTrimLeft(@)	‘subject string ‘
‘ subject string ‘	acfTrimLeft(@, ‘s’)	‘ subject string ‘
‘ subject string ‘	acfTrimLeft(@, ‘su’)	‘ subject string ‘
‘ subject string ‘	acfTrimLeft(@, ‘su ‘)	‘bject string ‘
‘ subject string ‘	acfTrimLeft(@, ‘gsu ‘)	‘bject string ‘
‘7BN123’	acfTrimLeft(@, ‘0123456789’)	‘BN123’

acfTrimRight

subject (@)	Expression	Return value
‘ subject string ‘	acfTrimRight(@)	‘ subject string’
‘ subject string ‘	acfTrimRight(@, ‘s’)	‘ subject string ‘
‘ subject string ‘	acfTrimRight(@, ‘su’)	‘ subject string ‘
‘ subject string ‘	acfTrimRight(@, ‘su ‘)	‘subject string ‘
‘ subject string ‘	acfTrimRight(@, ‘gsu ‘)	‘ subject strin’

AFD custom function Implementation in JMESPath

Example of a JMESPath query, where the first letter of a license plate may not contain V or B. The first letter does not have to start at the first position.

Example message

{
    "policy": [
        {
            "entityType": "policyDetails",
            "object":[
                {
                    "entityType": "motorVehicle",
                    "licensePlate": "7BN1234"
                }
            ]
        }
    ]
}

The JMESPath expression returns ‘true’ if the first letter is not equal to B or V:

!( (policy[].object[? entityType 'motorVehicle'])[] !=`[]` ) || ( (policy[].object[? entityType ‘motorVehicle’

&& !starts_with(licensePlate.acfTrimLeft(@,‘1234567890’), ‘B’) && !starts_with(licensePlate.acfTrimLeft(@,‘1234567890’), ‘V’) ])[] !=`[]`) )

3.3. Validation rules in XPath

4. How to create an AFD-definition?

Feedback

Post your comment on this topic.

3.4. Validation rules in JMESPath

1. What is a validation rule?

Example 1

Example 2

2. Technical use validation rules

JMESPath (AFD 2.0)

Ready-made libraries

Testing JMESPath expressions

3. Validation rules within the AFD-definition Standard

4. Validation rules in JMESPath (AFD 2.0)

5. Use of custom functions in JMESPath

6. AFD custom functions in JMESPath

AFD custom function specification

AFD custom function Name

AFD custom function Parameters

AFD Custom function Return value

AFD custom function Signature

AFD custom function Examples

7. Currently used AFD custom functions

AFD custom function specification

AFD custom function Name

AFD custom function Parameters

AFD custom function Return value

AFD custom function Signature

AFD custom function Examples

AFD custom function Implementation in JMESPath

AFD custom function specification

AFD custom function Name

AFD custom function Parameters

AFD custom function Return value

AFD custom function Signature

AFD custom function Examples

AFD custom function Implementation in JMESPath

AFD custom function specification

AFD custom function Name

AFD custom function Parameters

AFD custom function Return value

AFD custom function Signature

AFD custom function Examples

AFD custom function Implementation in JMESPath

AFD custom function specification

AFD custom function Name

AFD custom function Parameters

AFD custom function Return value

AFD custom function Signature

AFD custom function Examples

AFD custom function Implementation in JMESPath

Feedback