Managing the Chatbot Lifecycle with Versions

Last year, SAP Conversational AI introduced versioning as part of its enterprise-grade offering to help you implement changes to your bot when it’s already live. To make it easier for new developers to follow a robust workflow, all bots are now versioned by default.

If you’re new to versioning or don’t know how to use it effectively, this post will explain the basic concepts. If you’re already familiar with versioning, this post should help you adapt it to your development process and get even more out of versioning in teams with dedicated roles.

Note: This article assumes that you are familiar with the fundamentals of bot building. If you haven’t built a bot yet, check out this tutorial to learn how to build your first bot with SAP Conversational AI.

Why – Iterate on Your Chatbot with Zero Downtime

Consider the following scenario: You just released your bot, and users are engaging regularly. You want to improve your bot, so you create a new intent, but you don’t realize that it conflicts with an existing skill trigger. Someone talks to your bot and suddenly… no response. Making live edits like this can have unpredictable side effects and break your bot.

As a workaround, you could create a copy of your bot (via forking) every time you implement a change. But what happens when you want to make the updated bot available to your users (or roll back to a previous working bot after discovering a bug)? Your user-facing channels are still connected to the previous bot, so you need to re-point them – and this means digging into your application code, manually changing the bot token, and redeploying. Quite an involved process!

Concepts – What are Versions and Environments?

Luckily, you can streamline this process with versions and environments. Versions allow you to update your bot without exposing working drafts to users, while environments make it easy to deploy a new version or roll back to a previous version of your chatbot.

A version is a separate copy of your bot’s intents, entities, skills, and configuration within the same bot project. You can view and edit these versions independently. When working on a version of your bot, all changes are automatically saved to this version only, without affecting others. This allows you to have separate copies for production (the live version that users are talking to) and development (the unfinished version that you are working on).

Environments allow you to control how these separate versions are exposed to users. An environment is essentially a pointer to a version. It connects your bot’s entry point (channel) with the version that it points to.

Of course, you can easily re-point an environment to a different version – for example, when you want to promote a version from development to production.

Workflow – How to Use Versions and Environments

How can we put these concepts into practice? Let’s look at a workflow that we think works well.

Every new bot that you create or copy comes with a v1 version and a development (Dev) environment pointing to this version. Note: If you’re already using versioning for your bots, you might have a custom set of versions and environments instead.

For this example, we’ll build on the default setup and extend it as follows:

Here we’ve introduced additional Prod and Test environments, both connected to the same set of channels (Webchat and Slack). New versions move through the development process as shown below:

Let’s break this process down step-by-step:

1. Start by making changes to your Dev version – for example, implement a new skill, improve your bot’s training data, or fix a bug.

2. When you’re happy with the changes to your Dev version, promote your version from Dev to Test. Your Test environment should now point to the same version that the Dev environment is pointing to.

2.a If you don’t have a Test environment, create it along with the channels that your bot will be available from. Generally, your Test environment should mirror your production environment. This means that if users can talk to your bot on Slack, you should test your bot with a Slack channel too.

3. Run tests to make sure your bot is working as intended. If it is not, fix any bugs in your Dev version and rerun tests. (Depending on your bot’s complexity, this may be a manual or automated step.)

3.a Pro Tip: You can use the token of your Test environment to make a request to your bot from an external script (see Dialog endpoints).

4. Optionally, run a benchmark on your test version to determine the quality of your bot’s training data. Please note that you need to have at least 4 intents with 30 expressions each to use this feature. You can learn more about training analytics here.

4.a Pro Tip: When your benchmark is completed, compare the performance of your test version against your production version. This will help you catch any issues and show you how much your bot has improved since the last iteration.

5. When you are ready to release, first create a new version as a snapshot of your tested version – depending on the size of your bot this might take a few seconds. Assign it to your Dev environment so you can continue the next iteration of your bot from there.

5.a You can give any name to your new version, but it’s a good idea to pick a naming scheme and stick with it. (In this article we use linear numbering.)

6. Finally, promote the version from your Test environment to production to make it available to all users. Your Prod environment should now point to the same version that the Test environment is pointing to. The previous Prod version remains available in your version history, but no environment is currently pointing to it.

6.a If you don’t have a production environment yet, create it along with the channels that your bot will be available from.

7. Repeat steps 1 through 6 to iterate on your bot.

The benefit of this approach is that you end up with a linear version history where every version represents a stable release (except the most recent Dev version). This allows you to easily inspect previous versions and roll back if needed.

If you are familiar with version control and Git, conceptually we are using versions like tags here to mark release points in the bot’s history. You can also use versions like commits to creating recovery points during development. In this case, it’s a good idea to name your versions in a way that allows you to differentiate stable releases from development checkpoints. For example:

Versions and environments are flexible concepts. We encourage you to find and define a workflow that works best for your development process and team.

Advanced Lifecycle Scenario

Finally, let’s look at a more complex example that shows how we can go beyond convention and enforce a stricter workflow in teams with dedicated roles.

Let’s say we assign a person to monitor user requests and improve our training dataset over time, typically a subject matter expert with limited technical knowledge. We need to give them access to our production bot so they can monitor incoming user requests, but we don’t want them to be able to take down our live bot. What do we do?

First, let’s set up permissions so that the subject matter expert (SME) can contribute to a development version but not mess with our production version. It’s necessary that your bot is part of an organization for this. Note: You can learn more about organizations here.

1. Create two teams in your organization, one for the SME and another for the bot admin. Assign users to each team.

2. Make sure that organization-level permissions for the SME team are set to read-only. We will refine this at the bot level.

3. Go to your bot and set read+write permissions on the Dev environment for the SME team.

That’s it! With these permissions, the SME will be able to make changes only to the current Dev version, but no other version.

How does it work? When we set permissions on an environment, they take precedence over bot-level or organization-level permissions for the version that the environment is pointing to. In our example, write permissions on the Dev environment override the organization-level read-only permissions for the SME team. For versions that are assigned to other environments (or no environment at all), the read-only org-level permission still applies. You can learn more about permissions here.

Now let’s see what happens when the SME identifies an utterance in the Log Feed that they want to teach the bot:

When assigning an utterance from the Log Feed, you specify the intent and the version that you want to add to. Remember that a version is just a copy of the bot’s resources, so each version will have its own set of intents. If the utterance belongs to a new intent that doesn’t exist yet, you need to create the intent for the respective version first. You can learn more about the Log Feed here.

In our example, the SME is assigning an utterance to the Dev version of the bot. Since we’ve set up write permissions for the Dev environment, the utterance is added to the intent successfully.

But what happens when the SME tries to assign an utterance to the production version?

Because no specific permissions are defined for the production environment, it inherits the read-only org-level permissions – and the change is denied. Awesome, we just stopped the SME from making a change to the production version and breaking our live bot!

This example illustrates how you can take your versioning workflow to the next level – by protecting your versions with environment permissions and using Log Feed to continuously improve the training data of your development version with messages from your users.

If you are ready to use versioning like a pro, head over to your chatbot on SAP Conversational AI or sign up for free here!