Machine learning / Ticket intelligence 1802
1. Description of features and functions of Deal intelligence
a. General overview
- Service agents spend a large amount of time to manually classify tickets
- No reference to best practices/accuracy based on historical data sets
- Large customer service request queues
Service Ticket Intelligence automatically categorizes incoming customer enquiries to deliver best-in-class customer service.
Automatic categorization should help to:
- Improved service response times
- Improve efficiency of service team
- Increase in agent productivity
- Better prioritization of incoming tickets
- Automatic classification based on model accuracy
How does it work?
Historical data is used to train the machine learning model. As a result we get a customer specific predictive model which is applied to provide an automatic categorization of tickets.
Historic data is taken from existing tickets in C4C.
b. Detailed features & functions
i. End user experience
The category is one of the most important information in the ticket.
Based on the right categorization of a ticket, C4C offers multiple standard features based on the category to streamline the ticket handling. Main features are:
- Team assignment
- Knowledge base article determination
- Workflow rules
The ticket intelligence functionality automatically categorizes incoming tickets.
The categorization is done based on the subject and message body of an incoming email (subject & description if a ticket is manually created)
It is not mandatory to have both -subject AND description- in order to get a prediction, but with regards to an accurate description both should be filled.
The main use case is that customers send emails to your companies support email address/adresses and automatically a ticket is created in C4C. These tickets are categorized automatically.
The categorization is technically done in two steps.
First step is the prediction of a category based on the machine learning model. The predicted category is passed into the ticket – but in the first step only in a machine learning specific field on the tab ‘solution center’ in the ticket is filled.
In a second step this value is copied to the standard field ‘Service category’
Also a confidence level is coming from the prediction model.
If the confidence level is above the threshold set in the configuration of the model, then the predicted category will be copied to the standard category field (service category) of the ticket. All actions dependent on the category (workflowrules, routing etc.) are then triggered.
If it is below the threshold it is not copied and just serves as a proposal.
Currently the prediction is executed on every save of the ticket- but if the standard category field is already filled it is not overwritten by the category which was determined on save. This new category is only visible in the solution center.
So this means if a ticket is updated via email or manually- the standard category field will not be changed if it contains a value. (filled in manually or automatically via the prediction)
Also manually created tickets are categorized automatically ones they are saved. (in this case categories must not be mandatory in the Quick create of the ticket). There is a slight delay until you see the predicted category since the webservice to the Leonardo machine learning is asynchronous for performance reasons. In order to see the category in the standard category field you have to refresh the ticket.
Predicted categories can be changed manually.
In release 1802 on ticket intelligence will be able to automatically predict level 1 (Service category) of the category catalog.
In future releases also the second level category will be predicted.
On request SAP can switch the determination on a project base from level 1 to level 2. The machine learning model will then predict the second level category and the first level is derived from the result based on the assignment of the second level in the catalog. Please discuss this option with your implementation contact if required.
Currently there is no authorization/user specific configuration possible.
Once ticket intelligence is switched on, it applies to all users who work with tickets and to all tickets which are created manually or automatically
iii. Supported languages
English, German, French, Chinese,Spanish, Russian, portugese, Japanese. If you send emails with other languages, the prediction will deliver random results.
2. Prerequisites for the implementation
a. Customer must have a C4C Enterprise License
Please check your SAP C4C Contract for your license type.
b. Data volume
The more historical data the better.
To create the prediction model, the tickets of the last 12 month are used.
The absolute minimum needed to create the prediction model is 5000 tickets. The number is heavily dependent on the factors listed next.
c. Data history
No history data based on the change log of a ticket is required for the prediction of the category. In the future other fields are also planned to be predicted (for example priority) then the history will become important.
d. Data quality
The machine learning for tickets is based on the subject and description. So all tickets used for the model training should have properly filled the subject and description along with the proper category value. The more structured the data is in these fields the more accurate the prediction.
For example if the mail which is sent to C4C is based on a template in a webform the model accuracy will be higher than for emails which just contain any free texts written by the sender.
e. Data consistency
Ticket categories need to correspond to the categories in the active catalog.
Please make sure that all categories which are in the tickets used for the model training are part of the active category catalog.
If not a mapping is necessary -> inform your implementation contact!
f. Data balance
The data balance plays a significant role. If for example 90% of the tickets have the same category machine learning might not make sense since it always determines the same category. However this could still be a benefit since the user has not to make this first decision himself.
The more even the category assignment is in the tickets used for the model training, the higher is the benefit of using Ticket intelligence. Example you have category A,B,C,D and 10.000 tickets:
A distribution of the categories like
Each of the categories A,B,C,D are used in 2500 tickets is better than
9900 tickets have category A and B,C,D are only used in 100.
g. Customization/Custom development
If you have custom development related to the ticket category, it needs to be checked if it still is consistent once ticket intelligence is switched on.
In general the implementation can be divided into the following phases:
a. Precheck data
Check customer data with regards to topic 2 a-g
In a future release SAP will deliver automatic checks.
b. Activate/configure Ticket intelligence
i. Tenant decision
There are currently to ways to bring your ticket data into the SAP Leonardo component.
The standard way in 1802 : manually download ticket data from Production tenant and upload manually when you define your model in C4C.
If you choose this option SAP will download and upload the data for you.
If you choose this option the production data can be used to train your machine learning model in the test tenant. The feature can then be tested in the test tenant based on productive data.
As long as the upload/download process is in place, the tenant decision is clear – testing can be done in the current testtenant.
However- the manual upload/download is additional effort and can only be done with the help of SAP.
Therefore, in the future the manual download/upload will be obsolete and replaced by an automatic transfer of data from C4C to the Leonardo component.
In this case – before activating the Ticket intelligence feature it needs to be decided in which tenant the first implementation is done.
Full copy of Production tenant:
Recommended solution: Request a full copy of your production tenant. Set up the feature, load historical ticket data into the Leonardo machine learning component (will be described below) and test in this tenant. Be aware that a separate tenant might be subject to additional cost based on your contract.
use existing test tenant:
Since the historical data from the production tenant is required in order to train the machine learning model, test data from the test tenant will not do it.
If the recommended option above is not used, you could load your production data from production to Sap Leonardo machine learning- then request SAP to connect your test tenant to your productive SAP Leonardo connection and test based on the productive data.
The downside is that once you go live you have to switch the connection back and you have no test environment anymore. This option is only available as BETA version in 1802.
Also the switching of connections might cause time delays.
You can not request a new model after data validation since there is no data in test.
Contact your implemetation contact at SAP for this option.
The Ticket intelligence feature has to be activated by SAP.
An incident has to be created in the tenant where it should be activated.
Please request the activation of the scenario:
‘Machine lerning scenario: Ticket Intelligence’
from the tenant where you want the Machine learning scenario to be activated.
In a future release the feature can be switched on via scoping.
c. Train machine learning model
Before you can make use of the Ticket intelligence feature, you have to create your customer specific trained model which is used to predict the categories. To train your custom model go to the admin workcenter and choose the view ‘prediction services’ ‘configure ’.
The view has to be assigned to your admin user role in order to be visible in the admin workcenter.
The creation of the model is done in simple steps
1. Choose Ticket intelligence
2. Add a new model
- as a result you see a new model with your chosen description
- if you choose auto extraction as data source no further actions are required
- If you choose File upload, you have to upload the ticket data.
3. Mark your new model and choose ‘train’
- As a result the historical Ticket data is sent to the Leonardo machine learning component and your customer specific model is trained/created based on your historic data.
- This may take some time- wait until the status is ‘finished’
d. Validate model training results
As a result of the model training the SAP Leonardo component returns an accuracy which tells you how good your model is.
Any accuracy above 60 is considered as good.- the higher the better.
It is necessary to activate the model once the validation has been done and the accuracy is satisfying.
Once the model is active you can test either by creating tickets via emails or by creating manually tickets and check if the right categories are predicted.
f. Go live
In case you used your test tenant with a switched connection to the prod SAP Leonardo component, the connection has to be switched back.
In in case you started with a copy of your production tenant or with the test tenant you have to repeat all steps in your production tenant.
Once testing is complete you can make the feature available for all users.
4. Productive usage
If required, you can retrain your model by creating a new model and activating it.
When/Why would I create a new model?
A retraining of the model could be necessary if your Ticket data history changes over the course of time for example:
- After uploading new Tickets caused by a data migration
- Enrolling new processes for handling Tickets in your company
- Different user behavior
- Change of your category structure in the catalog.
Will the activation of a new model deactivate the old one?
Yes, only one model can be active at one point in time.
Can I reactivate an older model? Yes