Testing newly trained Machine Learning models on Production with Service Ticket Intelligence

Testing a new Machine Learning model before it replaces the old one for productive usage is key to ensuring model quality and business continuity. How can we achieve this continuous testing and deployment without interrupting the existing Machine Learning model functions and operations for your business?

In the context of Service Ticket Intelligence, the service only allows for one active model per language per scenario and there’s no way that you can test out newly trained models without having to de-activate the current active production model.

With the 2001 release, we introduced a test model support where you will now be able to execute tests against any trained models that are in READY (model successfully trained) state simply by specifying a model_id of interest in the inference request body payload.

Given the scenario below whereby you already have 3 Machine Learning models in your account:

• Model A, id xxxA – activated and used in production via ticket application
• Model B, id xxxB – newly trained model and to be tested/evaluated by an administrator
• Model C, id xxxC – model not trained and not testable

To test out the new Machine learning model (model B, id xxxB) here’s how you should structure your inference request body payload to make use of this feature. You will only need to provide an additional “model_id” field, referencing to the model of interest.

{
  "model_id": "xxxB",
  "business_object": "ticket",
  "messages": [
    {
      "id": 2001,
      "contents": [
        {
          "field": "text",
          "value": "you customer service is so good"
        }
      ]
    }
  ]
}

Test model support is applicable to both classification and recommendation models of Service Ticket Intelligence.

Service Ticket Intelligence is also available on SAP Cloud Platform as a free trial. Find out how you can set up your own trial account with Service Ticket Intelligence here and give the test model support feature a try today!