Technical Articles
Prompt Engineering for Advanced Text Processing on Customer Messages
This blog post is part of a series of technical enablement sessions on SAP BTP for Industries. Check the full calendar here to watch the recordings of past sessions and register for the upcoming ones! The replay of this session available here.
Authors: Yatsea Li, Alice Magnani, Edward Neveux, Jacob Tan
Disclaimer: SAP notes that posts about potential uses of generative AI and large language models are merely the individual poster’s ideas and opinions, and do not represent SAP’s official position or future development roadmap. SAP has no legal obligation or other commitment to pursue any course of business, or develop or release any functionality, mentioned in any post or related content on this website.
Introduction
Large Language Models (LLMs) like ChatGPT and GPT-4 have gained significant popularity with emergent capabilities. In this fourth session of the series, we will explore the potentials of GPT in SAP ecosystem, and discuss how SAP partners can leverage this powerful AI technology in combination with SAP technologies to accelerate their solution development processes, and create more intelligent solutions on SAP Business Technology Platform (SAP BTP).
Here list the full blog post series of Exploring the potential of GPT in SAP ecosystem:
In our previous blog about GPT as code assistance for SAP CAP and RAP Application development, we have created a basic Intelligent Ticketing Solution in SAP CAP with GPT. We will continue enhancing it by integrating with GPT through API for Advanced Text Processing on Customer Message.
From this blog post onward, we are going to implement the second solution requirement from SALifestyle under the two personas from the SAP partner, Christine the Consultant, and Daniel the Developer.
To fulfill this requirement, we will proceed with implementing the LLM Proxy Service in the architecture diagram, which integrates the Intelligent Ticketing Solution with GPT through APIs.
Before implementing LLM Proxy Service of Intelligent ticketing Solution for through GPT APIs, in this blog post, let us step back a bit to fulfill the same requirement through prompt engineering.
Prompt Engineering for Advanced Text Analysis and Processing on Customer Messages
To improve customer satisfaction with the products and services of SALifestyle, an inbound customer message needs to be analyzed and classified by an AI algorithm upon submission for appropriate action.
Specifically, the solution should generate a title and a short summary of the message.
It should also extract entities mentioned in the message, such as product, customer, and order etc, which can be used for further process automation.
Additionally, sentiment analysis should be performed to determine if the message is positive, negative, or neutral.
Furthermore, the solution should classify the message into different categories, such as technical issues requiring troubleshooting assistance or complaints needing immediate attention etc.
Prompt Deconstruct
Let’s recap the structure of a typical prompt with an example below. We’ll follow the exact structure in later’s prompt engineering for advanced text analysis and processing on customer message, such as text summarization, sentiment analysis, and entities extraction. Additionally, the target output of prompt should be in JSON, which can be processed with NodeJS in SAP CAP for integration through GPT’s API.
Prompt Engineering for Text Summarization
First, let us start with Text Summarization for Inbound Customer Message.
As an example, a customer has submitted a message expressing their dissatisfaction with receiving a broken popcorn maker during delivery.
A title is generated as Broken popcorn maker, along with a generated summary providing a concise overview of the issue.
Now, let’s have a look at the demo recording about how Christine and Daniel collaborate to achieve this through prompt engineering.
Here is the prompt of summarizing an inbound customer message into a title and a short summary as JSON output.
Here you have the prompts of text summarization showed in the video.
| Use Case#1 | Text Summarization for Inbound Customer Message |
| System Message | You are an AI assistant that helps to summarize the input text into a title not more than 100 characters and a short summary not more than 200 characters. |
| User Message | Input text: Everything was working fine one day I went to make a shot of coffee it stopped brewing after 3 seconds Then I tried the milk frother it stopped after 3 seconds again I took it back they fixed it under warranty but it’s happening again I don’t see this machine lasting more then 2 years to be honest I’m spewing I actually really like the machine It’s almost like it’s losing pressure somewhere, they wouldn’t tell my what the problem was when they fixed it.. Purchased at Harvey Norman for $1,349. Product is used: Several times a weekTitle: Summary: |
| Use Case#2 | Text Summarization for Inbound Customer Message in Ingestible JSON format |
| System Message |
You are an AI assistant that helps to summarize the input text into a title not more than 100 characters and a short summary not more than 200 characters. Expected output in JSON as below { \”title\”: \”{{Generated title}}\”,\”summary\”: \”{{Generated summary}}\” } |
| User Message | Input text: Everything was working fine one day I went to make a shot of coffee it stopped brewing after 3 seconds Then I tried the milk frother it stopped after 3 seconds again I took it back they fixed it under warranty but it’s happening again I don’t see this machine lasting more then 2 years to be honest I’m spewing I actually really like the machine It’s almost like it’s losing pressure somewhere, they wouldn’t tell my what the problem was when they fixed it.. Purchased at Harvey Norman for $1,349. Product is used: Several times a weekJSON: |
2. Prompt Engineering for Sentiment Analysis
Similarly, understanding the sentiment expressed in a customer message can greatly assist the service agents at SALifestyle in prioritizing and addressing the customer’s needs effectively, ultimately leading to an enhanced customer experience.
Let’s have a look at the demo recording about how Christine and Daniel analyze the sentiment of customer’s text message through prompt engineering.
Here you have the prompts of sentiment analysis showed in the video.
| Use Case#1 | Sentiment Analysis for Inbound Customer Message |
| System Message | You are an AI assistant that help to analyze the sentiment on the input text message coming from a service ticketing system |
| User Message | Input text: Everything was working fine one day I went to make a shot of coffee it stopped brewing after 3 seconds Then I tried the milk frother it stopped after 3 seconds again I took it back they fixed it under warranty but it’s happening again I don’t see this machine lasting more then 2 years to be honest I’m spewing I actually really like the machine It’s almost like it’s losing pressure somewhere, they wouldn’t tell my what the problem was when they fixed it.. Purchased at Harvey Norman for $1,349. Product is used: Several times a weekSentiment: |
| Use Case#2 | Sentiment Analysis for Inbound Customer Message in Ingestible JSON format |
| System Message |
You are an AI assistant that help to analyze the sentiment on the input text message coming from a service ticketing system. Expected output in JSON as below { \”sentiment\”: \”{{Positive/Neutral/Negative}}\” } |
| User Message | Input text: Everything was working fine one day I went to make a shot of coffee it stopped brewing after 3 seconds Then I tried the milk frother it stopped after 3 seconds again I took it back they fixed it under warranty but it’s happening again I don’t see this machine lasting more then 2 years to be honest I’m spewing I actually really like the machine It’s almost like it’s losing pressure somewhere, they wouldn’t tell my what the problem was when they fixed it.. Purchased at Harvey Norman for $1,349. Product is used: Several times a weekJSON: |
3. Prompt Engineering for Entities Extraction
Additionally, extracting entities from a customer message, such as product, customer, and order etc. not only enable the service agents of SALifestyle to easily identify the relevant information but also enables further process automation, which can improve process efficiency and ultimately lead to higher customer satisfaction.
Let us have a look at the demo recording about how Christine and Daniel extract the entities from a customer’s text message through prompt engineering.
Here you have the prompts of extracting entities showed in the video.
| Use Case#1 | Entities Extraction in list for Inbound Customer Message |
| System Message | You are an AI assistant that helps to extract a list of entities from input text. The target entities are customer_no and product_name, issue_description etc. Output a list of extracted entities with format as entity_name: entity_value and one entity one line. |
| User Message |
Input text: Everything was working fine one day I went to make a shot of coffee it stopped brewing after 3 seconds Then I tried the milk frother it stopped after 3 seconds again I took it back they fixed it under warranty but it’s happening again I don’t see this machine lasting more then 2 years to be honest I’m spewing I actually really like the machine It’s almost like it’s losing pressure somewhere, they wouldn’t tell my what the problem was when they fixed it.. Purchased at Harvey Norman for $1,349. Product is used: Several times a week Extracted entities: |
| Use Case#2 | Entities Extraction for Inbound Customer Message in Ingestible JSON format |
| System Message |
You are an AI assistant that helps to extract a list of entities from input text. An entity is made of a field and its value, such as customer_no, product_name etc. The field name follows snakecase naming conversion. Expected output in JSON as below {\”entities:\” [ {\”field\”: \”{{the identified field}}\”,\”value\”: \”{{the extracted value of the field}}\” }]} |
| User Message | Input text: Everything was working fine one day I went to make a shot of coffee it stopped brewing after 3 seconds Then I tried the milk frother it stopped after 3 seconds again I took it back they fixed it under warranty but it’s happening again I don’t see this machine lasting more then 2 years to be honest I’m spewing I actually really like the machine It’s almost like it’s losing pressure somewhere, they wouldn’t tell my what the problem was when they fixed it.. Purchased at Harvey Norman for $1,349. Product is used: Several times a weekJSON: |
4. Prompt Engineering for processing the Inbound Customer Message in one short
Finally, putting the pieces together into one comprehensive prompt is what SALifestyle ultimately aims for when processing inbound customer messages.
let’s have a look at the demo recording about how Christine and Daniel collaborate to achieve this through prompt engineering.
Here you have the prompts of processing customer message showed in the video.
| Use Case#1 | Processing Inbound Customer Message in Ingestible JSON format |
| System Message | You are an AI assistant to process the input text. Here are your tasks on the text. 1.Apply Sentiment Analysis 2.Generate a title less than 100 characters,and summarize the text into a short description less than 200 characters 3.Extract the entities such as customer,product,order,delivery,invoice etc from the text Here is a preliminary list of the target entity fields and description. Please extract all the identifiable entities even not in the list below. Don’t include any field with unknown value. -customer_no: alias customer number, customer id, account id, account number which could be used to identify a customer. -customer_name: customer name, account name -customer_phone: customer contact number. -product_no: product number, product id -product_name -order_no: sales order number, order id -order_date -delivery_no: delivery number, delivery id -delivery_date: delivery date, shipping date -invoice_no: alias invoice number, invoice id, receipt number, receipt id etc. which can be used to locate a invoice. -invoice_date: invoice date, purchase date -store_name -store_location etc.For those fields not in list must follow the Snakecase name conversation like product_name, no space allow.Output expected in JSON format as below: {\”sentiment\”:\”{{Positive/Neutral/Negative}}\”,\”title\”:\”{{The generated title based on the input text less than 100 characters}}\”,\”summary\”:\”{{The generated summary based on the input text less than 300 characters}}\”,\”entities\”:[{\”field\”:\”{{the extracted fields such as product_name listed above}}\”,\”value\”:\”{{the extracted value of the field}}\”}]} |
| User Message | Input text: Everything was working fine one day I went to make a shot of coffee it stopped brewing after 3 seconds Then I tried the milk frother it stopped after 3 seconds again I took it back they fixed it under warranty but it’s happening again I don’t see this machine lasting more then 2 years to be honest I’m spewing I actually really like the machine It’s almost like it’s losing pressure somewhere, they wouldn’t tell my what the problem was when they fixed it.. Purchased at Harvey Norman for $1,349. Product is used: Several times a weekJSON: |
Further resources
- The source code of the sample Intelligent Ticketing Solution
- SAP and generative AI
- Best practices for prompt engineering with OpenAI API
Wrap up
We have seen that GPT can be helpful in the advanced text analysis and processing on customer message process, such as text summarization, sentiment analysis and entities extraction etc, in next blog post we’ll see how to integrate this prompt into the intelligent ticketing solution via GTP Chat API. We hope you have enjoyed the reading.
Good job, Yatsea!
Thanks Jay.
Great Article! It is quite informative and helpful.