SAP Web IDE workspace and HDI container isolation, or how to work on one schema with multiple developers
By default, the SAP Web IDE for HANA, and the SAP Web IDE Full-Stack, write HDB modules into developer specific schemas. This blog describes why, and how you can change this behavior.
When you logon to the SAP Web IDE, you see the tree structure with your projects, folders and files. Those files are physically stored in your own little section sitting somewhere in Cloud Foundry or your XSA server. We call that space a workspace. Instead of having a shared workspace for all developers, the Web IDE works with a developer-isolated workspace. This means that each developer has his own workspace, and whatever the developer does within his workspace stays within that workspace, unless he distributes the content to other developers. Such distribution would most commonly be done through a centralized Git repository, or alternatively by exporting and importing (parts of) a project. The below figure illustrates this behavior.
Figure 1: DW development with Web IDE, where each developer is working in his own workspace, and syncs design time changes via the Git repository
Having an isolated workspace makes a lot of sense, as this allows developers to isolate their work from other developers, so they can concentrate on their own work without being distracted by changes that other people make. But how is this isolated behaviour reflected when you create objects that need to be pushed to HANA?
Inside a workspace, you can set up projects that contain HANA Database (HDB) modules. By default, the Web IDE builds each HDB module into a developer specific HANA Deployment Infrastructure (HDI) container, which on the database is represented by one main schema, and several other objects that hold metadata. In this blog, we only discuss the main schema, which will hold the runtime versions of your design time objects, such as tables and views.
Default behavior: developer-isolated schemas
By default, when the first developer builds an HDB module from within the Web IDE and has defined a schema name in his yaml file, a schema with that name is created in HANA and populated with its objects. However, if a second developer is working on the same project in his own workspace, and executes a build from his Web IDE, a new schema is created that is postfixed with “_1”, and the next developer would create one with postfix “_2”, and so forth. This means that besides an isolated workspace that holds all design time files for each developer, there is also an isolated schema or set of schemas, that hold the runtime objects for each developer. The below illustration depicts that. Please note at time of writing, this is the default behavior for the latest version of the Web IDE on both XSA and CloudFoundry. In older versions the Web IDE would start with the first schema already postfixed by “_1”. A deployment of a project, not using the Web IDE but using xs deploy or cf deploy, would always take the defined schema name without any postfix.
Figure 2: isolated workspace, isolated schema
Alternative behavior: share the runtime schema with multiple developers
It is possible to configure your HDB modules to write into a fixed schema name, without creating a unique schema for each developers’ HDB module. This way, developers share the same schema. But be aware that workspaces of developers cannot be shared. Therefore, sharing the schema comes at a risk of developers overwriting each other’s changes. This simply happens when one developer changes his design time objects inside an HDB module, and then builds these into the runtime schema. Since his design time object will deviate from other developers’ design time objects, the runtime version of objects will be out of sync for other developers. As soon as another developer builds again his design time objects, the runtime versions are again overwritten.
Figure 3: isolated workspace, shared schema
Therefore, it is strongly recommended against having modules of different Git feature branches build into the same schema. After all, the design time objects will differ from branch to branch, so with each build of a branch you will be overwriting the runtime state of another branch. You can consider though to allow different developers who work on the same project and same branch, to build their modules into the same schema. This still comes at the risk of overwrites, as the design time files are not automatically shared across the developers’ individual workspaces. However, you could mitigate that risk in two ways. First, by only building objects that you have changed, instead of doing a full build of the HDB module. Second, by syncing the workspace changes often via the Git repository.
There is one other risk of sharing the schema, namely that one developer bases his objects on runtime objects of another developer, who has not pushed the design time versions to Git. For example, developer A defines a calculation view on a table that was build by developer B. If developer B has not pushed his design time table to Git, a Web IDE build would still succeed, because the dependency checks are done on the runtime schema and the table exists there. But a deploy of the calculation view, based on the design time files sitting in Git, to another system would fail, because the underlying table is not part of the design time files and therefore will not be available in the runtime schema. This risk can be mitigated by automated test deploys that would provide you with an early warning.
Code example of how to share the schema
In the following example, I created a new project based on the SAP HANA Database Application template. That provides me with a project that holds just one HDB module. I’m using the default settings, which means the schema will be isolated.
Figure 4: Project structure and yaml file after creation from template
When building the HDB module, this generates an HDI container with main schema STRAWBERRY, as you can see in the below screenshot. Also note, that a service name is generated.
Figure 5: Web IDE build log excerpt for the first developer
To show what happens when a second developer builds the same HDB module, for testing purposes, I just export the project and import it under a different name. Now I have two projects with the same design time content. A build provides me with a different service name and a different schema name.
Figure 6: Web IDE build log excerpt for the second developer
To configure writing into the same schema instead, three changes are needed:
- In the YAML file, add the parameter “makeUniqueName” and set it to “false”;
- In the YAML file, add a parameter to define a fixed service name. Otherwise, still a unique service name will be created for each developer, and that service will not be allowed to share the schema of another service;
- If you have already built your HDB module, you have to remove the existing service, either using the Web IDE Resource Manager, or using the XSA or CloudFoundry Cockpit. Otherwise, the service and schema name will stay as it was defined before. Note that this will remove the entire HDI container including your data.
The code in the YAML file now looks as follows:
Figure 7: YAML file with fixed service name and fixed schema name
After removal of the existing services, a new build shows the following: a fixed service name, and a fixed schema name. Any other developer that builds the same project, will now connect to the same service and build into the same schema.
Figure 8: Web IDE build log excerpt for any developer
Working with the SAP Web IDE and HDB modules allows you to work in isolated mode, both design time in your workspace, and runtime on the SAP HANA database. However, in certain cases it can be beneficial to share the schema among multiple developers. This comes with the risk of developers overwriting each others’ objects, so you should have carefully considered if those risks can be well enough mitigated for your specific use case.
Clear and useful as usual, Sefan. Thanks!
Another great blog.
Thanks Sefan . Great blog simple to understand, well written !
Thanks Sefan, great post!
Say you were to implement an SQL DWH on HANA, which of the approaches would you recommend? If I understand it correctly, in the first approach with an isolated HDI Container for every developer, every developer basically builds his own development DWH. This has advantages when it comes to overwriting other developers objects, as you pointed out. On the other hand, in a large DWH with a lot of developers, this can create a lot schemas & tables and use a lot space very fast, right? Additionally, how would you make sure that each developer has data in his isolated container to test his objects with, and isnt working with empty tables?
Thanks again & BR,
The choice between isolating or sharing the schema should depend on a number of factors. The number of developers is one, but think also of the extent of dependency on other features being developed, or if it is easy for you to create a clean and rightly sized test data set. I hope to publish a more detailed blog that provides guidance on that topic.
When you think about the options, consider also a combination of isolation and sharing. For example, split your DW in a persistent part and a virtualized part (calcviews etc.). Share the persistent part (and keep that part simple) as one or more HDI containers and isolate the virtualized part in another. That's a solution that doesn't cost any extra space, but gives you a lot of flexibility.
Thanks for the reply! Splitting the DW sounds like an interesting idea as well.
Lokking forward to your next blog!
First of all, thanks for your blog.
I have a situation where we got stuck.
We have a landscape for xsa with dev->acc-> prod.
1. First we did our build in personal workspace pointing to dev space. Please note that we have added 'make unique schema: false' statement in yaml to avoid generating multiple schemas.
2. Next we did build of project and schema got generated. Say schema name is ITSCHEMA and it's corresponding container as we have used db module.
3. Next when we requested basis to deploy the mtar file to dev space , they mentioned that deployment failed with reason schema ITSCHEMA is already created, there is already container with this name.
We are not sure if we are missing something, can you help on the same.
Do you also have a service name defined in your YAML, and did you have that service name BEFORE you did the first build or deploy? That is the prerequisite. Otherwise, two different services are trying to build into the same schema, which is not allowed.
You mentioned XS Deploy should write to the actual schema but however when I am trying to do it says either unique constraint error or service cannot bind error.
Could not bind application "db" to service "hdi_db": 400 Bad Request
Any idea on this?
have you got this resolved? Having similar issues
See response to Praveen above.
Hi, is this resolved!
Thanks Stefan, this was really helpful
I am new to this world and I have a series of doubts.
What do you refer to developers? and if 2 schemes are created in the same workspace with the suffix _1 and _2, how can I configure the development space with one or the other scheme?
That is, I enter and my objects are in the _1 scheme and when trying to modify an object I get an error since when building it is looking in the _2 scheme.