Technical Articles
Validate JSON message in API Management
Introduction
API Management provides MessageValidation Policy, which has the following functionalities:
-
Validate any XML message against an XSD schema.
- Validate SOAP messages against a WSDL definition.
- Confirm JSON or XML is well-formed, based on content-type (if <ResourceURL> element is omitted)
Obviously its limitation is to validate a json message based on a json schema. In the following content, a way to validate json message based on json schema has been introduced.
The policy template can be downloaded from github: here
Steps to Implement JSON Validation
1. Get JSON Schema.
There are many online web pages provides service generating json schema from a sample json message.
This is the one I’m using
https://jsonformatter.org/json-to-jsonschema
For example, you have a bellowing json message
{
"MessageHeader":{
"MessageId":"734bbd96-7dc0-4a03-9597-69eb4ff53c0d",
"SystemDateTime":"1638610670",
"SenderService":"POSTMAN"
},
"BusinessDocument":{
"DocumentId":"612",
"Description":"Senior",
"Creater":"Kautzer",
"Items":[
{
"ItemId":"00010",
"Material":"circuit",
"Quantity":"910"
},
{
"ItemId":"00020",
"Material":"capacitor",
"Quantity":"271"
}
]
}
}
Put the message into the web page as below
This is the json schema generated
{
"$schema": "http://json-schema.org/draft-06/schema#",
"$ref": "#/definitions/Welcome9",
"definitions": {
"Welcome9": {
"type": "object",
"additionalProperties": false,
"properties": {
"MessageHeader": {
"$ref": "#/definitions/MessageHeader"
},
"BusinessDocument": {
"$ref": "#/definitions/BusinessDocument"
}
},
"required": [
"BusinessDocument",
"MessageHeader"
],
"title": "Welcome9"
},
"BusinessDocument": {
"type": "object",
"additionalProperties": false,
"properties": {
"DocumentId": {
"type": "string",
"format": "integer"
},
"Description": {
"type": "string"
},
"Creater": {
"type": "string"
},
"Items": {
"type": "array",
"items": {
"$ref": "#/definitions/Item"
}
}
},
"required": [
"Creater",
"Description",
"DocumentId",
"Items"
],
"title": "BusinessDocument"
},
"Item": {
"type": "object",
"additionalProperties": false,
"properties": {
"ItemId": {
"type": "string"
},
"Material": {
"type": "string"
},
"Quantity": {
"type": "string",
"format": "integer"
}
},
"required": [
"ItemId",
"Material",
"Quantity"
],
"title": "Item"
},
"MessageHeader": {
"type": "object",
"additionalProperties": false,
"properties": {
"MessageId": {
"type": "string",
"format": "uuid"
},
"SystemDateTime": {
"type": "string",
"format": "integer"
},
"SenderService": {
"type": "string"
}
},
"required": [
"MessageId",
"SenderService",
"SystemDateTime"
],
"title": "MessageHeader"
}
}
}
Note: By default, the service sets all provided fields as mandatory.
2. Convert the Json schema into base64 string.
For some reasons I cannot explain, the Key Value Map service truncates the json schema. As a walkaround, its base64 string does work.
This is the base64 encode online service I used:
This is the base64 string generated.
ewogICAgIiRzY2hlbWEiOiAiaHR0cDovL2pzb24tc2NoZW1hLm9yZy9kcmFmdC0wNi9zY2hlbWEjIiwKICAgICIkcmVmIjogIiMvZGVmaW5pdGlvbnMvV2VsY29tZTkiLAogICAgImRlZmluaXRpb25zIjogewogICAgICAgICJXZWxjb21lOSI6IHsKICAgICAgICAgICAgInR5cGUiOiAib2JqZWN0IiwKICAgICAgICAgICAgImFkZGl0aW9uYWxQcm9wZXJ0aWVzIjogZmFsc2UsCiAgICAgICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgICAgICAgIk1lc3NhZ2VIZWFkZXIiOiB7CiAgICAgICAgICAgICAgICAgICAgIiRyZWYiOiAiIy9kZWZpbml0aW9ucy9NZXNzYWdlSGVhZGVyIgogICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICJCdXNpbmVzc0RvY3VtZW50IjogewogICAgICAgICAgICAgICAgICAgICIkcmVmIjogIiMvZGVmaW5pdGlvbnMvQnVzaW5lc3NEb2N1bWVudCIKICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgfSwKICAgICAgICAgICAgInJlcXVpcmVkIjogWwogICAgICAgICAgICAgICAgIkJ1c2luZXNzRG9jdW1lbnQiLAogICAgICAgICAgICAgICAgIk1lc3NhZ2VIZWFkZXIiCiAgICAgICAgICAgIF0sCiAgICAgICAgICAgICJ0aXRsZSI6ICJXZWxjb21lOSIKICAgICAgICB9LAogICAgICAgICJCdXNpbmVzc0RvY3VtZW50IjogewogICAgICAgICAgICAidHlwZSI6ICJvYmplY3QiLAogICAgICAgICAgICAiYWRkaXRpb25hbFByb3BlcnRpZXMiOiBmYWxzZSwKICAgICAgICAgICAgInByb3BlcnRpZXMiOiB7CiAgICAgICAgICAgICAgICAiRG9jdW1lbnRJZCI6IHsKICAgICAgICAgICAgICAgICAgICAidHlwZSI6ICJzdHJpbmciLAogICAgICAgICAgICAgICAgICAgICJmb3JtYXQiOiAiaW50ZWdlciIKICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiRGVzY3JpcHRpb24iOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIgogICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICJDcmVhdGVyIjogewogICAgICAgICAgICAgICAgICAgICJ0eXBlIjogInN0cmluZyIKICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiSXRlbXMiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAiYXJyYXkiLAogICAgICAgICAgICAgICAgICAgICJpdGVtcyI6IHsKICAgICAgICAgICAgICAgICAgICAgICAgIiRyZWYiOiAiIy9kZWZpbml0aW9ucy9JdGVtIgogICAgICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgfSwKICAgICAgICAgICAgInJlcXVpcmVkIjogWwogICAgICAgICAgICAgICAgIkNyZWF0ZXIiLAogICAgICAgICAgICAgICAgIkRlc2NyaXB0aW9uIiwKICAgICAgICAgICAgICAgICJEb2N1bWVudElkIiwKICAgICAgICAgICAgICAgICJJdGVtcyIKICAgICAgICAgICAgXSwKICAgICAgICAgICAgInRpdGxlIjogIkJ1c2luZXNzRG9jdW1lbnQiCiAgICAgICAgfSwKICAgICAgICAiSXRlbSI6IHsKICAgICAgICAgICAgInR5cGUiOiAib2JqZWN0IiwKICAgICAgICAgICAgImFkZGl0aW9uYWxQcm9wZXJ0aWVzIjogZmFsc2UsCiAgICAgICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgICAgICAgIkl0ZW1JZCI6IHsKICAgICAgICAgICAgICAgICAgICAidHlwZSI6ICJzdHJpbmciCiAgICAgICAgICAgICAgICB9LAogICAgICAgICAgICAgICAgIk1hdGVyaWFsIjogewogICAgICAgICAgICAgICAgICAgICJ0eXBlIjogInN0cmluZyIKICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiUXVhbnRpdHkiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIiwKICAgICAgICAgICAgICAgICAgICAiZm9ybWF0IjogImludGVnZXIiCiAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgIH0sCiAgICAgICAgICAgICJyZXF1aXJlZCI6IFsKICAgICAgICAgICAgICAgICJJdGVtSWQiLAogICAgICAgICAgICAgICAgIk1hdGVyaWFsIiwKICAgICAgICAgICAgICAgICJRdWFudGl0eSIKICAgICAgICAgICAgXSwKICAgICAgICAgICAgInRpdGxlIjogIkl0ZW0iCiAgICAgICAgfSwKICAgICAgICAiTWVzc2FnZUhlYWRlciI6IHsKICAgICAgICAgICAgInR5cGUiOiAib2JqZWN0IiwKICAgICAgICAgICAgImFkZGl0aW9uYWxQcm9wZXJ0aWVzIjogZmFsc2UsCiAgICAgICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgICAgICAgIk1lc3NhZ2VJZCI6IHsKICAgICAgICAgICAgICAgICAgICAidHlwZSI6ICJzdHJpbmciLAogICAgICAgICAgICAgICAgICAgICJmb3JtYXQiOiAidXVpZCIKICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiU3lzdGVtRGF0ZVRpbWUiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIiwKICAgICAgICAgICAgICAgICAgICAiZm9ybWF0IjogImludGVnZXIiCiAgICAgICAgICAgICAgICB9LAogICAgICAgICAgICAgICAgIlNlbmRlclNlcnZpY2UiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIgogICAgICAgICAgICAgICAgfQogICAgICAgICAgICB9LAogICAgICAgICAgICAicmVxdWlyZWQiOiBbCiAgICAgICAgICAgICAgICAiTWVzc2FnZUlkIiwKICAgICAgICAgICAgICAgICJTZW5kZXJTZXJ2aWNlIiwKICAgICAgICAgICAgICAgICJTeXN0ZW1EYXRlVGltZSIKICAgICAgICAgICAgXSwKICAgICAgICAgICAgInRpdGxlIjogIk1lc3NhZ2VIZWFkZXIiCiAgICAgICAgfQogICAgfQp9Cg==
3. Create a key value map to store json schema.
KVM name: JsonSchemaSandbox
key: schema
value: <the base64 json schema string generated above>
To make sure the configuration is consistent with the policies, please use the exactly the same names to create the KVM. or You need to change the KVM operation policy accordingly.
Key Valu Map storing Json Schema
4. Import the Policy Template and apply it to your API proxy
Once appied the template, the following policies and programs will be added to the proxy.
Step 1: CheckMalformedJson.
Confirm json message is malformed or not.
Step 2: Get the stored json schema from KVM in base64 format.
Step 3: decode base64 at first. Then validate the json by using tv4 function.
The full version of tv4 can be downloaded from github: here.
Step 4: raise exception if necessary.
Step 5: fill in the backend credential. Please adjust this step according to you specific environment.
Step 6: Basic Auth.
Unit Test
1. Positive Case
put the following json into POSTMAN. There are quite a few variables are used to generate different values each time to imitate a more real case.
{
"MessageHeader":{
"MessageId":"{{$guid}}",
"SystemDateTime":"{{$timestamp}}",
"SenderService":"POSTMAN"
},
"BusinessDocument":{
"DocumentId":"{{$randomInt}}",
"Description":"{{$randomJobDescriptor}}",
"Creater":"{{$randomLastName}}",
"Items":[
{
"ItemId":"00010",
"Material":"{{$randomNoun}}",
"Quantity":"{{$randomInt}}"
},
{
"ItemId":"00020",
"Material":"{{$randomNoun}}",
"Quantity":"{{$randomInt}}"
}
]
}
}
My backend service is a sort of echo, which responses what it has received. This is the result.
2. Negative Case with malformed Json
make a malformed json by removing the comma after the “MessageHeader” curved braket as below
{
"MessageHeader":{
"MessageId":"{{$guid}}",
"SystemDateTime":"{{$timestamp}}",
"SenderService":"POSTMAN"
}
"BusinessDocument":{
"DocumentId":"{{$randomInt}}",
"Description":"{{$randomJobDescriptor}}",
"Creater":"{{$randomLastName}}",
"Items":[
{
"ItemId":"00010",
"Material":"{{$randomNoun}}",
"Quantity":"{{$randomInt}}"
},
{
"ItemId":"00020",
"Material":"{{$randomNoun}}",
"Quantity":"{{$randomInt}}"
}
]
}
}
This is the normal functionality handled by the native MessageValidation Policy. This is the result.
we can clearly see the malformed issue detected by POSTMAN edit above and the error message responsed from API management, which is not that precise, but enough for us to fix the error.
3. Negative Case with Missing Compulsory Element
let’s remove the second line time number as below
{
"MessageHeader":{
"MessageId":"{{$guid}}",
"SystemDateTime":"{{$timestamp}}",
"SenderService":"POSTMAN"
},
"BusinessDocument":{
"DocumentId":"{{$randomInt}}",
"Description":"{{$randomJobDescriptor}}",
"Creater":"{{$randomLastName}}",
"Items":[
{
"ItemId":"00010",
"Material":"{{$randomNoun}}",
"Quantity":"{{$randomInt}}"
},
{
"Material":"{{$randomNoun}}",
"Quantity":"{{$randomInt}}"
}
]
}
}
This is the test result.
we can see in line 18, the compulsory element is missing. In the response message, Error Data Path says that
Conclusion
By using javascrip policy, we can fulfill the requirement validating json on schema.
Please feel free to download the policy template for your work or study: here.
Thank you for this great article. It's a good and easy solution to check json payloads within APIM. Thanks again!
cheers.
Hello
I am getting next error on validating:
Do you have any Ideas what is wrong?
Thanks