This blog post is a follow up to Getting your head into Cloud Application Programming model multitenancy. A Cloud Application Programming(CAP) model sample code project that implements multitenancy via the CDS-MTX library in SAP Cloud Platform Cloud Foundry and illustrates the use-case of having a common container that is joined against the run-time containers created by the ServiceManager during the subscription process can be found here.
Video= In 1 Take : HiddenLife
The Service Manager is in itself a service that manages other services. This sounds kinda redundant but the broader goal is to support a more generalized way to interact with services across various hypervisors/landscapes. For us it brings for the first time a supported way to manage HANA containers in multitenant applications. Using the service manager in this way only exercises a small portion of it's capabilities, but for our purposes, that's OK, we just need something that can create HANA containers at run-time as opposed to deploy-time.
When you create an application in Cloud Foundry, most every example you find will describe using a service called hana with a plantype of hdi-shared. This provides for the creation of a static set of HANA containers at deploy-time that the application can use to store it's data. While this could be used to set up a large pool of containers for use serving multiple customers in a multitenant application, this would be a cumbersome way to manage data separation. What happens when you need to add one more customer that you'd expected? A much more efficient way is to dynamically create run-time containers as your customers are subscribing to your multitenant application.
The problem is that all the existing development tooling is only designed to allow you to explore, interact with, manage deploy-time created containers as described above. Containers that are created at run-time are controlled by the ServiceManager and use a different API to interact with them and the current tooling has not been enhanced to cover these run-time types of containers.
Technically, if you know all the connection details for a run-time container, you can enter them into the DB Explorer as if you were connecting to a distinct HANA server, but that option is very cumbersome and error prone since all the information about the container is system generated and finding it is also not trivial.
Fine you say. I'll just wait for SAP to get around to improving their tooling to handle the multitenant use-case and then start my multitenant project development. But what if you want to get started now? Maybe you want to be first-mover in a hot new industry segment and capture all the glory and profits!
The good new is that SAP also promotes as one of its core tenants, openness in its tooling and solutions that allow you to not have to wait if you're willing to put in some extra effort. The net effect is that we're not boxed in to a proprietary tool set and can extend things as needed.
Let's see what we can pull together to bridge this current functionality gap. What I'll describe below is valid as of early February 2021. Even if this blog post ages rapidly with new tooling releases, the concepts should be a guide to help you not be afraid to try to extend things to fit your needs no matter what SAP provides.
We'll start with Business Application Studio(BAS) and configure it for our needs. If you need to set up BAS, check out this tutorial.
Fire up your BAS subscription url and click Create Dev Space.
Give your DevSpace a name and use the "SAP Cloud Business Application" type and select the SAP HANA Tools extension at a minimum. Click the create button.
The reason we used "SAP Cloud Business Application" and not "SAP HANA Native Application" is that the cloud business app type includes a 3rd party extension called SQLTools where the other does not. This might change in the future but for now it's the case. Installing the SQLTools into a DevSpace that originated as a HANA native app is problematic.
Once it's started, click on its name.
From here I won't show a screen shot for each step. Just follow along.
In my case I won't go through the steps to deploy this app. I already have so I'll just focus on the pertinent parts.
So I can run "cf a" and "cf s" to see my deployed application.
Notice that my application has 2 deploy-time containers defined with names CAPMT_HDI and CAPMT_SUB. CAPMT_HDI is intended to store data that all subscribers will have access to in one place. CAPMT_SUB is actually a copy of the container a subscriber would get upon subscription. I do this so that I can use the existing tooling to at least check to see that the tables/views are getting defined properly. Also notice that my application has a service instance called CAPMT_SMC of type service-manager and plan container. This is where my app can get the information needed to interact with the service manager.
I'll expand the PROJECTS tree view and also the SAP HANA PROJECTS section and move it up to give us some more space to work with it. Note: For some reason you have to expand the section before you're able to move it.
Click on the green plug bind icon.
Select the CAPMT_HDI existing deploy-time container.
Notice now the little stack of blocks icon is available. Clicking on this brings up the DB Explorer.
The DB Explorer is launched and you can browse tables/views and the data within them.
All good so far. Now let's subscribe to our application. They have changed how they look a bit but the mechanism is the same.
OK, we see now that subscription has completed.
The example app we're using here leverages the Cloud Application Programming(CAP) development model. One of the benefits of using this approach is that it takes care of much of the complexity of building a multitenant application. Out of the box it provides the needed callback methods that are required to interact with the system's subscription mechanism. See the docs on the saas-registry service for more details on this. The CAP multitenant library (cds-mtx) uses the database source code provided in the db module to populate run-time containers created via the subscription process. Since we have CAPMT_SUB deploy-time container in our project we can inspect it to see if we're getting the desired structure and expect that the run-time containers will also be working, but how do we prove that this is the case?
When you're in the heat of a development effort things can and do go wrong and it's always best when you have an independent way to verify your assumptions.
The subscription produces no evidence of the run-time container that is supposed to be created. How do we know it exists? Did it get created properly? Does it contain the right customer specific data? All of these questions are difficult to answer given the current state of the tooling.
In order to figure out what the service manager is doing, you need to use it's API to query it and ask for a list of hana/hdi-shared instances that it's created. You could use a REST tool(like Postman). This is in fact what I did to get started understanding how the service-manager API behaved. I took my learnings and put them into a CF CLI plugin. You can install the plugin with this command.
As of this writing, this will give you version 1.1.1. Plugins provide sub-commands and this one provides a shorthand sub-command called smsi. Run it like this.
Without any additional flags, it will product something like this. In this case I have 2 subscriptions running.
It's now waiting for you to select one. Type in a number 1-3 to get the result. If you only have 1 subscription it won't prompt you, but will just give you the result.
Now if you're paying close attention, you'll notice that the Tenant: is the same GUID as the subaccount GUID from which the subscription was made.
If you run the command again but pass the -credentials flag, you'll get all the needed details to make a connection to the container.
At this point you could cut/paste the Host,Port, User, Password, Certificate into the DB Explorer and get a connection to the run-time container.
Go back to the DB Explorer and click on the + to add a new connection.
When you connect this way, you'll find that the DB Explorer opens with the wrong schema selected. You'll need to click on the schema button and select the proper short(er) version.
Not the one ending _RT.
Finally, you can browse the tables/views and open the contents.
Wow that a lot of steps and a lot of opportunities to make a cut/paste error. Believe me I've make my share.
I've added a flag to smsi sub-command to do all this work for you. Try it again but this time with the -m. flag.
Now go to the SQLTools icon, expand the CONNECTIONS. You'll see the container name is already there.
On the right end of the container name, click on the green plug icon.
Once the connection is made, you can expand the tables/views and right click on one and open it's contents.
Much faster and convenient for run-time created containers.
I don't know about you, but my brain doesn't remember GUIDs very well. What I really wanted was to have a list of subscribed subaccount that I could remember easily and use those names. I updated the callback handler in my application to add an additional label to my run-time container during the subscription processing that I could use in the smsi plugin.
At the time of this writing, I hadn't been able to publish this latest version(1.2.4) but there is another way.
Often you want to get you development environment "just so" with the right tools and versions and configuration that fits your needs. This is especially important when you develop in teams and everyone need to "be on the same page" in regard to the tooling they are using.
Since BAS follows the Theia extension format,, you can use that same extension format to create your own extensions. I've done this to collect up a set of installers that I find useful for my development and hope you find them useful too. I've published them on the OpenVSX marketplace where BAS gets it's extensions.
Nothing happened! Well, actually something did, but it is a bit hidden. A bunch of new commands were registered and are now available to be run.
Go to View -> FInd Command... and type "BAS:" the list will be filtered to this set of Install commands.
Pick the Install CF SMSI Plugin.
This will get you the latest plugin directly. Again, it doesn't seem like anything really happened. You can check the output view of the command to check.
Now, when you run the command with -m, you'll get connections with the subdomain name and not the GUID.
Much better. Now we have nice human readable connections to our run-time containers without interfering with the deploy-time container functionality.
As nice as this is, there are some drawbacks. SQLTools doesn't know much about how HANA databases are structured. True you can browse tables/views and see their contents, it doesn't(currently) know anything about synonyms, calculation views, sequences or other HANA specific DB artifacts. Maybe this will change but since SQLTools is a 3rd party extension, you can always ask the developer to add a feature or you can continue to blaze your own path.
Sometimes you just need to verify something quickly without the fuss of a graphical interface. This is especially true when you want to do unit testing or manage your projects with CI/CD tools. Thankfully an esteemed colleague of mine thomas.jung has developed just such a tool(hana-cli) and it doesn't take much to leverage it.
Let's install it from the BAS Installer Extension. Again, look for this under View -> Find Commands...
Let's also install a second CF plugin called DefaultEnv. This time from philip.mugglestone.
Now we can leverage both these together. Change into the tools directory of the sample project.
From here we'll create connection files called default-env.json with the "cf de" command and pick them up with the hana-cli command and run various features of hana-cli.
To create a default-env.json file for a deploy-time container, use "cf de " with the app name that created the container.
The smsi plugin now had a similar option to output default-env.json files but since there is the possibility to have many subscribers and thus run-time containers, then the connections are stored in a default-env-subdomain.json file to keep them unique. Then you can copy the one you want to default-env.json or I've written a script in the tools folder that does this for you called smsi-cli.
Notice that it gives you some example ways to invoke smsi-cli even as a wrapper to hana-cli.
Hana-cli does understand many of the HANA specific artifacts. One is synonyms. Check out the hana-cli help for more things to try.
Many thanks to Thomas for his hard work on hana-cli.
Hopefully this blog post has "lifted the veil" on service-manager created run-time containers. At some point the tooling will be updated to include this functionality inherently.
But until then we at least have ways of getting around these kinds of current limitations.
My continued hope is that this post along with the accompanying code sample put you and your project on the path to success.
Let me know if you have and questions or issues by leaving me a question below or better yet, asking it on the SAP community.
-Andrew
Note: This blog post was written prior to the account upgrade effort to "Feature Set B". As such screen grabs may not appear the same as what you're currently experiencing. That doesn't mean that the info contained here isn't useful, just keep this in mind.
Video= In 1 Take : HiddenLife
What is the ServiceManager?
The Service Manager is in itself a service that manages other services. This sounds kinda redundant but the broader goal is to support a more generalized way to interact with services across various hypervisors/landscapes. For us it brings for the first time a supported way to manage HANA containers in multitenant applications. Using the service manager in this way only exercises a small portion of it's capabilities, but for our purposes, that's OK, we just need something that can create HANA containers at run-time as opposed to deploy-time.
How is this different?
When you create an application in Cloud Foundry, most every example you find will describe using a service called hana with a plantype of hdi-shared. This provides for the creation of a static set of HANA containers at deploy-time that the application can use to store it's data. While this could be used to set up a large pool of containers for use serving multiple customers in a multitenant application, this would be a cumbersome way to manage data separation. What happens when you need to add one more customer that you'd expected? A much more efficient way is to dynamically create run-time containers as your customers are subscribing to your multitenant application.
So what's the problem?
The problem is that all the existing development tooling is only designed to allow you to explore, interact with, manage deploy-time created containers as described above. Containers that are created at run-time are controlled by the ServiceManager and use a different API to interact with them and the current tooling has not been enhanced to cover these run-time types of containers.
Technically, if you know all the connection details for a run-time container, you can enter them into the DB Explorer as if you were connecting to a distinct HANA server, but that option is very cumbersome and error prone since all the information about the container is system generated and finding it is also not trivial.
The virtue of being impatient.
Fine you say. I'll just wait for SAP to get around to improving their tooling to handle the multitenant use-case and then start my multitenant project development. But what if you want to get started now? Maybe you want to be first-mover in a hot new industry segment and capture all the glory and profits!
The good new is that SAP also promotes as one of its core tenants, openness in its tooling and solutions that allow you to not have to wait if you're willing to put in some extra effort. The net effect is that we're not boxed in to a proprietary tool set and can extend things as needed.
What we can do now?
Let's see what we can pull together to bridge this current functionality gap. What I'll describe below is valid as of early February 2021. Even if this blog post ages rapidly with new tooling releases, the concepts should be a guide to help you not be afraid to try to extend things to fit your needs no matter what SAP provides.
We'll start with Business Application Studio(BAS) and configure it for our needs. If you need to set up BAS, check out this tutorial.
Fire up your BAS subscription url and click Create Dev Space.
Give your DevSpace a name and use the "SAP Cloud Business Application" type and select the SAP HANA Tools extension at a minimum. Click the create button.
The reason we used "SAP Cloud Business Application" and not "SAP HANA Native Application" is that the cloud business app type includes a 3rd party extension called SQLTools where the other does not. This might change in the future but for now it's the case. Installing the SQLTools into a DevSpace that originated as a HANA native app is problematic.
Once it's started, click on its name.
From here I won't show a screen shot for each step. Just follow along.
- Open the workspace and select the projects folder.
- Open a new terminal window. Clone my multitenant sample project.
git clone https://github.com/SAP-samples/cloud-cap-multitenancy.git- Set your Cloud Foundry organization and space to where you have/will deploy the application.
In my case I won't go through the steps to deploy this app. I already have so I'll just focus on the pertinent parts.
So I can run "cf a" and "cf s" to see my deployed application.
Notice that my application has 2 deploy-time containers defined with names CAPMT_HDI and CAPMT_SUB. CAPMT_HDI is intended to store data that all subscribers will have access to in one place. CAPMT_SUB is actually a copy of the container a subscriber would get upon subscription. I do this so that I can use the existing tooling to at least check to see that the tables/views are getting defined properly. Also notice that my application has a service instance called CAPMT_SMC of type service-manager and plan container. This is where my app can get the information needed to interact with the service manager.
I'll expand the PROJECTS tree view and also the SAP HANA PROJECTS section and move it up to give us some more space to work with it. Note: For some reason you have to expand the section before you're able to move it.
- Expand the db_comm and db sections and their Database Connections.
Click on the green plug bind icon.
Select the CAPMT_HDI existing deploy-time container.
Notice now the little stack of blocks icon is available. Clicking on this brings up the DB Explorer.
The DB Explorer is launched and you can browse tables/views and the data within them.
All good so far. Now let's subscribe to our application. They have changed how they look a bit but the mechanism is the same.
- Go to a subaccount within the global account where the application is deployed and create a subscription to the capmt-partner app.
- Click Create.
OK, we see now that subscription has completed.
I don't believe there's a CAP on creativity and passion.
-Deadmau5
The example app we're using here leverages the Cloud Application Programming(CAP) development model. One of the benefits of using this approach is that it takes care of much of the complexity of building a multitenant application. Out of the box it provides the needed callback methods that are required to interact with the system's subscription mechanism. See the docs on the saas-registry service for more details on this. The CAP multitenant library (cds-mtx) uses the database source code provided in the db module to populate run-time containers created via the subscription process. Since we have CAPMT_SUB deploy-time container in our project we can inspect it to see if we're getting the desired structure and expect that the run-time containers will also be working, but how do we prove that this is the case?
When you're in the heat of a development effort things can and do go wrong and it's always best when you have an independent way to verify your assumptions.
The subscription produces no evidence of the run-time container that is supposed to be created. How do we know it exists? Did it get created properly? Does it contain the right customer specific data? All of these questions are difficult to answer given the current state of the tooling.
Extending the CF CLI.
In order to figure out what the service manager is doing, you need to use it's API to query it and ask for a list of hana/hdi-shared instances that it's created. You could use a REST tool(like Postman). This is in fact what I did to get started understanding how the service-manager API behaved. I took my learnings and put them into a CF CLI plugin. You can install the plugin with this command.
cf install-plugin -r CF-Community "service-management"As of this writing, this will give you version 1.1.1. Plugins provide sub-commands and this one provides a shorthand sub-command called smsi. Run it like this.
cf smsiWithout any additional flags, it will product something like this. In this case I have 2 subscriptions running.
It's now waiting for you to select one. Type in a number 1-3 to get the result. If you only have 1 subscription it won't prompt you, but will just give you the result.
Now if you're paying close attention, you'll notice that the Tenant: is the same GUID as the subaccount GUID from which the subscription was made.
If you run the command again but pass the -credentials flag, you'll get all the needed details to make a connection to the container.
At this point you could cut/paste the Host,Port, User, Password, Certificate into the DB Explorer and get a connection to the run-time container.
Go back to the DB Explorer and click on the + to add a new connection.
When you connect this way, you'll find that the DB Explorer opens with the wrong schema selected. You'll need to click on the schema button and select the proper short(er) version.
Not the one ending _RT.
Finally, you can browse the tables/views and open the contents.
Wow that a lot of steps and a lot of opportunities to make a cut/paste error. Believe me I've make my share.
SQLTools to the rescue!
I've added a flag to smsi sub-command to do all this work for you. Try it again but this time with the -m. flag.
cf smsi -mNow go to the SQLTools icon, expand the CONNECTIONS. You'll see the container name is already there.
On the right end of the container name, click on the green plug icon.
Once the connection is made, you can expand the tables/views and right click on one and open it's contents.
Much faster and convenient for run-time created containers.
Even better!
I don't know about you, but my brain doesn't remember GUIDs very well. What I really wanted was to have a list of subscribed subaccount that I could remember easily and use those names. I updated the callback handler in my application to add an additional label to my run-time container during the subscription processing that I could use in the smsi plugin.
At the time of this writing, I hadn't been able to publish this latest version(1.2.4) but there is another way.
Custom Extension to harmonize development environments.
Often you want to get you development environment "just so" with the right tools and versions and configuration that fits your needs. This is especially important when you develop in teams and everyone need to "be on the same page" in regard to the tooling they are using.
Since BAS follows the Theia extension format,, you can use that same extension format to create your own extensions. I've done this to collect up a set of installers that I find useful for my development and hope you find them useful too. I've published them on the OpenVSX marketplace where BAS gets it's extensions.
- Go to the extensions and agree to the disclaimer and type in Partners in the search box.
Nothing happened! Well, actually something did, but it is a bit hidden. A bunch of new commands were registered and are now available to be run.
Go to View -> FInd Command... and type "BAS:" the list will be filtered to this set of Install commands.
Pick the Install CF SMSI Plugin.
This will get you the latest plugin directly. Again, it doesn't seem like anything really happened. You can check the output view of the command to check.
Now, when you run the command with -m, you'll get connections with the subdomain name and not the GUID.
Much better. Now we have nice human readable connections to our run-time containers without interfering with the deploy-time container functionality.
But what if?
As nice as this is, there are some drawbacks. SQLTools doesn't know much about how HANA databases are structured. True you can browse tables/views and see their contents, it doesn't(currently) know anything about synonyms, calculation views, sequences or other HANA specific DB artifacts. Maybe this will change but since SQLTools is a 3rd party extension, you can always ask the developer to add a feature or you can continue to blaze your own path.
Sometimes you just need to verify something quickly without the fuss of a graphical interface. This is especially true when you want to do unit testing or manage your projects with CI/CD tools. Thankfully an esteemed colleague of mine thomas.jung has developed just such a tool(hana-cli) and it doesn't take much to leverage it.
Let's install it from the BAS Installer Extension. Again, look for this under View -> Find Commands...
Let's also install a second CF plugin called DefaultEnv. This time from philip.mugglestone.
Now we can leverage both these together. Change into the tools directory of the sample project.
cd /home/user/projects/cloud-cap-multitenancy/toolsFrom here we'll create connection files called default-env.json with the "cf de" command and pick them up with the hana-cli command and run various features of hana-cli.
To create a default-env.json file for a deploy-time container, use "cf de " with the app name that created the container.
But what about run-time containers?
The smsi plugin now had a similar option to output default-env.json files but since there is the possibility to have many subscribers and thus run-time containers, then the connections are stored in a default-env-subdomain.json file to keep them unique. Then you can copy the one you want to default-env.json or I've written a script in the tools folder that does this for you called smsi-cli.
Notice that it gives you some example ways to invoke smsi-cli even as a wrapper to hana-cli.
./smsi-cli -s partner-subb tablesHana-cli does understand many of the HANA specific artifacts. One is synonyms. Check out the hana-cli help for more things to try.
./smsi-cli -s partner-subb synonymsMany thanks to Thomas for his hard work on hana-cli.
Conclusion
Hopefully this blog post has "lifted the veil" on service-manager created run-time containers. At some point the tooling will be updated to include this functionality inherently.
But until then we at least have ways of getting around these kinds of current limitations.
My continued hope is that this post along with the accompanying code sample put you and your project on the path to success.
Let me know if you have and questions or issues by leaving me a question below or better yet, asking it on the SAP community.
-Andrew
| Partners: If you have a question, click here to ask it in the SAP Community . Be sure to tag it with Partnership and leave your company name in the question so that we can better assist you. |
- SAP Managed Tags:
15 Comments
