My colleague Carina Kothe, who is part of the Kyma team, has been looking into AutoGen. (Disclaimer: There are no plans to integrate AutoGen into Kyma). During her research, she came across my A Chess Game with AutoGen and ChatGPT blog post and asked me if I had used AutoGen with SAP AI Core.
For an internal hackathon, I had setup SAP AI Core based on these SAP AI Core tutorials. However, I haven’t had the opportunity to experiment with the integration in AutoGen. But just the day before Carina’s question, I had been exploring how Ollama could be used locally with Autogen. Therefore, I knew that there were more options available than just OpenAI or Azure OpenAI.
Challenge Accepted
SAP AI Core has specific operational requirements. It requires a bearer token to authenticate the user and provide them access to the system. In addition, the custom header field ‘AI-Resource-Group’ is necessary. This field is used to select the pre-configured resource group. Finally, an ‘api-version’ is needed in the query string.
My initial approach was to ask ChatGPT. It suggested a promising idea, but unfortunately, it didn't work out. However, I learned a valuable lesson: if you're ever tasked with extending an interface, you can ask ChatGPT for help. Just assume that the extension you're looking for exists, and ChatGPT will likely hallucinate a wonderful interface.
Despite ChatGPT's suggestion, I couldn’t find any "auth" or "oauth" details in the AutoGen code. For this reason, I created a proxy to better understand the data sent by AutoGen. This proxy also had the advantage of allowing me to test the interaction between AutoGen and SAP AI Core. I was able to identify the necessary data that AutoGen needed to send in order to communicate directly with SAP AI Core.
Header Data
With the proxy I analyzed the header data that AutoGen was sending:
Host: localhost:8080
Accept-Encoding: gzip, deflate
Connection: keep-alive
Accept: application/json
Content-Type: application/json
User-Agent: OpenAI/Python 1.14.3
X-Stainless-Lang: python
X-Stainless-Package-Version: 1.14.3
X-Stainless-Os: MacOS
X-Stainless-Arch: arm64
X-Stainless-Runtime: CPython
X-Stainless-Runtime-Version: 3.11.6
Authorization: Bearer ollama
X-Stainless-Async: false
Content-Length: 4884
The first thing I saw was that AutoGen was setting the authorization. I had the API key from the Ollama test in the config_list configuration which was included in the header data. I just replace the Ollama API key with the bearer token for SAP AI Core.
The X-Stainless headers provided another piece of the puzzle. They guided me towards the custom headers and I was able to find them within the code. With some extra searching I found the extra_headers within llm_config, which is defined when the AssistantAgent class is initialized. The same for the extra_query for the api-version query.
Code
This is the code which sets the bearer token, extra header and extra query:
from autogen import AssistantAgent, UserProxyAgent
import os
import oauth
token = oauth.get_bearer_token()
base_url = os.getenv('AI_CORE_BASE_PATH')
config_list = [
{
"model": "egal",
"base_url": base_url,
"api_key": token,
}
]
llm_config={
"cache_seed": None,
"config_list": config_list,
"temperature": 0,
"extra_headers": {
"AI-Resource-Group": 'default',
},
"extra_query": {
"api-version": "2023-05-15"
}
}
# Create an AssistantAgent with extra headers and query parameters
assistant = AssistantAgent("assistant", llm_config=llm_config)
# Create a UserProxyAgent that can execute code
user_proxy = UserProxyAgent(
"user_proxy",
code_execution_config={"work_dir": "coding", "use_docker": False})
user_proxy.initiate_chat(assistant, message="""
You are a useful agent and create only the titles of innovative ideas.
You don’t answer questions or give any suggestions.
You only respond strictly with an innovative title and a short one-line description
""")
The oauth.get_bearer_token() function is defined in another file which can be found also at the autogen-aicore repository. The agent itself creates innovations but answers strictly with an innovation title and short description.
Summary
The extensibility of Autogen, together with SAP AI Core, allows different agents to be used for tasks such as automating jobs, using collaborative agents, analyzing data or making new AI models. And my agent creates with "unicorn at the coffee machine" this innovation:
Title: "Unicorn Brew: The Magic Coffee Experience"
Description: An AI-powered coffee machine that personalizes your coffee based on your mood, weather, and schedule.
Lets innovate!
