Skip to Content
Technical Articles
Author's profile photo stephen xue

Validate JSON message in API Management

Introduction

API Management provides MessageValidation Policy, which has the following functionalities:

  • Validate any XML message against an XSD schema.

  • Validate SOAP messages against a WSDL definition.
  • Confirm JSON or XML is well-formed, based on content-type (if <ResourceURL> element is omitted)

Obviously its limitation is to validate a json message based on a json schema. In the following content, a way to validate json message based on json schema has been introduced.

The policy template can be downloaded from github: here

Steps to Implement JSON Validation

1.  Get JSON Schema.

There are many online web pages provides service generating json schema from a sample json message.

This is the one I’m using

https://jsonformatter.org/json-to-jsonschema

For example, you have a bellowing json message

{
    "MessageHeader":{
        "MessageId":"734bbd96-7dc0-4a03-9597-69eb4ff53c0d",
        "SystemDateTime":"1638610670",
        "SenderService":"POSTMAN"
    },
    "BusinessDocument":{
        "DocumentId":"612",
        "Description":"Senior",
        "Creater":"Kautzer",
        "Items":[
            {
                "ItemId":"00010",
                "Material":"circuit",
                "Quantity":"910"
            },
            {
                "ItemId":"00020",
                "Material":"capacitor",
                "Quantity":"271"
            }
        ]
    }
}

Put the message into the web page as below

This is the json schema generated

{
    "$schema": "http://json-schema.org/draft-06/schema#",
    "$ref": "#/definitions/Welcome9",
    "definitions": {
        "Welcome9": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
                "MessageHeader": {
                    "$ref": "#/definitions/MessageHeader"
                },
                "BusinessDocument": {
                    "$ref": "#/definitions/BusinessDocument"
                }
            },
            "required": [
                "BusinessDocument",
                "MessageHeader"
            ],
            "title": "Welcome9"
        },
        "BusinessDocument": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
                "DocumentId": {
                    "type": "string",
                    "format": "integer"
                },
                "Description": {
                    "type": "string"
                },
                "Creater": {
                    "type": "string"
                },
                "Items": {
                    "type": "array",
                    "items": {
                        "$ref": "#/definitions/Item"
                    }
                }
            },
            "required": [
                "Creater",
                "Description",
                "DocumentId",
                "Items"
            ],
            "title": "BusinessDocument"
        },
        "Item": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
                "ItemId": {
                    "type": "string"
                },
                "Material": {
                    "type": "string"
                },
                "Quantity": {
                    "type": "string",
                    "format": "integer"
                }
            },
            "required": [
                "ItemId",
                "Material",
                "Quantity"
            ],
            "title": "Item"
        },
        "MessageHeader": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
                "MessageId": {
                    "type": "string",
                    "format": "uuid"
                },
                "SystemDateTime": {
                    "type": "string",
                    "format": "integer"
                },
                "SenderService": {
                    "type": "string"
                }
            },
            "required": [
                "MessageId",
                "SenderService",
                "SystemDateTime"
            ],
            "title": "MessageHeader"
        }
    }
}

Note: By default, the service sets all provided fields as mandatory.

2. Convert the Json schema into base64 string.

For some reasons I cannot explain, the Key Value Map service truncates the json schema. As a walkaround, its base64 string does work.

This is the base64 encode online service I used:

https://www.base64encode.org/

This is the base64 string generated.

ewogICAgIiRzY2hlbWEiOiAiaHR0cDovL2pzb24tc2NoZW1hLm9yZy9kcmFmdC0wNi9zY2hlbWEjIiwKICAgICIkcmVmIjogIiMvZGVmaW5pdGlvbnMvV2VsY29tZTkiLAogICAgImRlZmluaXRpb25zIjogewogICAgICAgICJXZWxjb21lOSI6IHsKICAgICAgICAgICAgInR5cGUiOiAib2JqZWN0IiwKICAgICAgICAgICAgImFkZGl0aW9uYWxQcm9wZXJ0aWVzIjogZmFsc2UsCiAgICAgICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgICAgICAgIk1lc3NhZ2VIZWFkZXIiOiB7CiAgICAgICAgICAgICAgICAgICAgIiRyZWYiOiAiIy9kZWZpbml0aW9ucy9NZXNzYWdlSGVhZGVyIgogICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICJCdXNpbmVzc0RvY3VtZW50IjogewogICAgICAgICAgICAgICAgICAgICIkcmVmIjogIiMvZGVmaW5pdGlvbnMvQnVzaW5lc3NEb2N1bWVudCIKICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgfSwKICAgICAgICAgICAgInJlcXVpcmVkIjogWwogICAgICAgICAgICAgICAgIkJ1c2luZXNzRG9jdW1lbnQiLAogICAgICAgICAgICAgICAgIk1lc3NhZ2VIZWFkZXIiCiAgICAgICAgICAgIF0sCiAgICAgICAgICAgICJ0aXRsZSI6ICJXZWxjb21lOSIKICAgICAgICB9LAogICAgICAgICJCdXNpbmVzc0RvY3VtZW50IjogewogICAgICAgICAgICAidHlwZSI6ICJvYmplY3QiLAogICAgICAgICAgICAiYWRkaXRpb25hbFByb3BlcnRpZXMiOiBmYWxzZSwKICAgICAgICAgICAgInByb3BlcnRpZXMiOiB7CiAgICAgICAgICAgICAgICAiRG9jdW1lbnRJZCI6IHsKICAgICAgICAgICAgICAgICAgICAidHlwZSI6ICJzdHJpbmciLAogICAgICAgICAgICAgICAgICAgICJmb3JtYXQiOiAiaW50ZWdlciIKICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiRGVzY3JpcHRpb24iOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIgogICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICJDcmVhdGVyIjogewogICAgICAgICAgICAgICAgICAgICJ0eXBlIjogInN0cmluZyIKICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiSXRlbXMiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAiYXJyYXkiLAogICAgICAgICAgICAgICAgICAgICJpdGVtcyI6IHsKICAgICAgICAgICAgICAgICAgICAgICAgIiRyZWYiOiAiIy9kZWZpbml0aW9ucy9JdGVtIgogICAgICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgfSwKICAgICAgICAgICAgInJlcXVpcmVkIjogWwogICAgICAgICAgICAgICAgIkNyZWF0ZXIiLAogICAgICAgICAgICAgICAgIkRlc2NyaXB0aW9uIiwKICAgICAgICAgICAgICAgICJEb2N1bWVudElkIiwKICAgICAgICAgICAgICAgICJJdGVtcyIKICAgICAgICAgICAgXSwKICAgICAgICAgICAgInRpdGxlIjogIkJ1c2luZXNzRG9jdW1lbnQiCiAgICAgICAgfSwKICAgICAgICAiSXRlbSI6IHsKICAgICAgICAgICAgInR5cGUiOiAib2JqZWN0IiwKICAgICAgICAgICAgImFkZGl0aW9uYWxQcm9wZXJ0aWVzIjogZmFsc2UsCiAgICAgICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgICAgICAgIkl0ZW1JZCI6IHsKICAgICAgICAgICAgICAgICAgICAidHlwZSI6ICJzdHJpbmciCiAgICAgICAgICAgICAgICB9LAogICAgICAgICAgICAgICAgIk1hdGVyaWFsIjogewogICAgICAgICAgICAgICAgICAgICJ0eXBlIjogInN0cmluZyIKICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiUXVhbnRpdHkiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIiwKICAgICAgICAgICAgICAgICAgICAiZm9ybWF0IjogImludGVnZXIiCiAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgIH0sCiAgICAgICAgICAgICJyZXF1aXJlZCI6IFsKICAgICAgICAgICAgICAgICJJdGVtSWQiLAogICAgICAgICAgICAgICAgIk1hdGVyaWFsIiwKICAgICAgICAgICAgICAgICJRdWFudGl0eSIKICAgICAgICAgICAgXSwKICAgICAgICAgICAgInRpdGxlIjogIkl0ZW0iCiAgICAgICAgfSwKICAgICAgICAiTWVzc2FnZUhlYWRlciI6IHsKICAgICAgICAgICAgInR5cGUiOiAib2JqZWN0IiwKICAgICAgICAgICAgImFkZGl0aW9uYWxQcm9wZXJ0aWVzIjogZmFsc2UsCiAgICAgICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgICAgICAgIk1lc3NhZ2VJZCI6IHsKICAgICAgICAgICAgICAgICAgICAidHlwZSI6ICJzdHJpbmciLAogICAgICAgICAgICAgICAgICAgICJmb3JtYXQiOiAidXVpZCIKICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAiU3lzdGVtRGF0ZVRpbWUiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIiwKICAgICAgICAgICAgICAgICAgICAiZm9ybWF0IjogImludGVnZXIiCiAgICAgICAgICAgICAgICB9LAogICAgICAgICAgICAgICAgIlNlbmRlclNlcnZpY2UiOiB7CiAgICAgICAgICAgICAgICAgICAgInR5cGUiOiAic3RyaW5nIgogICAgICAgICAgICAgICAgfQogICAgICAgICAgICB9LAogICAgICAgICAgICAicmVxdWlyZWQiOiBbCiAgICAgICAgICAgICAgICAiTWVzc2FnZUlkIiwKICAgICAgICAgICAgICAgICJTZW5kZXJTZXJ2aWNlIiwKICAgICAgICAgICAgICAgICJTeXN0ZW1EYXRlVGltZSIKICAgICAgICAgICAgXSwKICAgICAgICAgICAgInRpdGxlIjogIk1lc3NhZ2VIZWFkZXIiCiAgICAgICAgfQogICAgfQp9Cg==

3. Create a key value map to store json schema.

KVM name: JsonSchemaSandbox

key: schema

value: <the base64 json schema string generated above>

To make sure the configuration is consistent with the policies, please use the exactly the same names to create the KVM. or You need to change the KVM operation policy accordingly.

Key%20Valu%20Map%20storing%20Json%20Schema

Key Valu Map storing Json Schema

4. Import the Policy Template and apply it to your API proxy

Once appied the template, the following policies and programs will be added to the proxy.

Polities

Step 1: CheckMalformedJson.

Confirm json message is malformed or not.

Step 2: Get the stored json schema from KVM in base64 format.

Step 3: decode base64 at first. Then validate the json by using tv4 function.

The full version of tv4 can be downloaded from github: here.

Step 4: raise exception if necessary.

Step 5: fill in the backend credential. Please adjust this step according to you specific environment.

Step 6: Basic Auth.

Unit Test

1.  Positive Case

put the following json into POSTMAN. There are quite a few variables are used to generate different values each time to imitate a more real case.

{
    "MessageHeader":{
        "MessageId":"{{$guid}}",
        "SystemDateTime":"{{$timestamp}}",
        "SenderService":"POSTMAN"
    },
    "BusinessDocument":{
        "DocumentId":"{{$randomInt}}",
        "Description":"{{$randomJobDescriptor}}",
        "Creater":"{{$randomLastName}}",
        "Items":[
            {
                "ItemId":"00010",
                "Material":"{{$randomNoun}}",
                "Quantity":"{{$randomInt}}"
            },
            {
                "ItemId":"00020",
                "Material":"{{$randomNoun}}",
                "Quantity":"{{$randomInt}}"
            }
        ]
    }
}

My backend service is a sort of echo, which responses what it has received. This is the result.

2. Negative Case with malformed Json

make a malformed json by removing the comma after the “MessageHeader” curved braket as below

{
    "MessageHeader":{
        "MessageId":"{{$guid}}",
        "SystemDateTime":"{{$timestamp}}",
        "SenderService":"POSTMAN"
    }
    "BusinessDocument":{
        "DocumentId":"{{$randomInt}}",
        "Description":"{{$randomJobDescriptor}}",
        "Creater":"{{$randomLastName}}",
        "Items":[
            {
                "ItemId":"00010",
                "Material":"{{$randomNoun}}",
                "Quantity":"{{$randomInt}}"
            },
            {
                "ItemId":"00020",
                "Material":"{{$randomNoun}}",
                "Quantity":"{{$randomInt}}"
            }
        ]
    }
}

This is the normal functionality handled by the native MessageValidation Policy. This is the result.

we can clearly see the malformed issue detected by POSTMAN edit above and the error message responsed from API management, which is not that precise, but enough for us to fix the error.

3. Negative Case with Missing Compulsory Element

let’s remove the second line time number as below

{
    "MessageHeader":{
        "MessageId":"{{$guid}}",
        "SystemDateTime":"{{$timestamp}}",
        "SenderService":"POSTMAN"
    },
    "BusinessDocument":{
        "DocumentId":"{{$randomInt}}",
        "Description":"{{$randomJobDescriptor}}",
        "Creater":"{{$randomLastName}}",
        "Items":[
            {
                "ItemId":"00010",
                "Material":"{{$randomNoun}}",
                "Quantity":"{{$randomInt}}"
            },
            {

                "Material":"{{$randomNoun}}",
                "Quantity":"{{$randomInt}}"
            }
        ]
    }
}

This is the test result.

we can see in line 18, the compulsory element is missing. In the response message, Error Data Path says that

“ErrorDataPath”: “/BusinessDocument/Items/1”.
Consider in the json world, all arrays starts from 0, and 1 means the second item of the array. The provided information is comprehensive enough for us to deterimine the issue.

Conclusion

By using javascrip policy, we can fulfill the requirement validating json on schema.

Please feel free to download the policy template for your work or study: here.

Assigned Tags

      3 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Cedric Heisel
      Cedric Heisel

      Thank you for this great article. It's a good and easy solution to check json payloads within APIM. Thanks again!

      Author's profile photo stephen xue
      stephen xue
      Blog Post Author

      cheers.

      Author's profile photo Dzianis Monich
      Dzianis Monich

      Hello

       

      I am getting next error on validating:

      Do you have any Ideas what is wrong?

       

      Thanks