Technical Articles
Automatically change HDI artifacts before building & deploying in Business Application Studio with GULP
Introduction
Initially the idea why we used this is to change remote source schemas of a .hdbreptask artifact before building and deploying it to the productive space.
The reason being is that these .hdbreptasks have a hardcoded reference to the source system schema, which looks like this:
<SourceObject RemoteObjectUniqueName=""SCHEMA_DEV"."TABLE_NAME"" SourceDisplayName="TABLE_NAME">
However once you deploy the MTAR into your productive space the remote source schema might not be “SCHEMA_DEV” but “SCHEMA_PROD”. If the schemaname is identical there is no issue with simply deploying the replication task.
Deployment from Dev to Prod System
Error message upon deployment in production
Deploying "src/replication/RT_BLOG_DEMO.hdbreptask"...
Error: com.sap.hana.di.reptask: Could not activate replication task: Database error 476: invalid remote object name: Could not find table, view or procedure named ["SCHEMA_DEV"."BLOG_DEMO"]. [8257007]
at "src/replication/RT_BLOG_DEMO.hdbreptask" (0:0)
Error: com.sap.hana.di.reptask: Deploying "src/replication/RT_BLOG_DEMO.hdbreptask"... failed [8212145]
at "src/replication/RT_BLOG_DEMO.hdbreptask" (0:0)
Warning: Worker 1 running the "com.sap.hana.di.reptask" plugin has encountered an error while deploying 1 objects [8212030]
Maybe initially you have the urge, as did we, to manually change the replication task files before building/deploying.
This is tedious (and also unnecessary as it turns out) plus you may end up breaking your development codeline if not properly managing your code repository.
Solution
For the initial solution we used Gulp.
Gulp is a cross-platform, streaming task runner that lets developers automate many development tasks. At a high level, gulp reads files as streams and pipes the streams to different tasks. These tasks are code-based and use plugins. The tasks modify the files, building source files into production files. To get an idea of what gulp can do check the list of gulp recipes on GitHub. [1]
What the script will help us with is executing the changes on the replication tasks files automatically.
You simply run the script instead of the classic build command and it will do the magic for you.
- add the gulpfile.js to your workspace
- add the following lines to your package.json
"scripts": { "make_development": "gulp --schema SCHEMA_DEV", "make_production": "gulp --schema SCHEMA_PROD" },
if you dont have gulp installed the following error will appear when calling the script:
user: GULP_DEMO$ npm run make_production
npm WARN lifecycle The node binary used for scripts is /extbin/bin/node but npm is using /opt/nodejs/node-v14.16.0-linux-x64/bin/node itself. Use the `--scripts-prepend-node-path` option to include the path for the node binary npm was executed with.
> deploy@ make_production /home/user/projects/SQL_Statements
> gulp --schema SCHEMA_PROD
sh: 1: gulp: not found
install gulp and additional modules using npm
npm install -save gulp gulp-replace del minimist
The actual script (gulpfile.js)
const log = require('fancy-log')
const { src, dest, series } = require('gulp')
const replace = require('gulp-replace')
const argv = require('minimist')(process.argv.slice(2))
const del = require('del')
const exec = require('child_process').exec;
function handleError(msg, done){
log.error(msg)
done()
}
function clean(done) {
return del([
'./dist'
])
}
function dist(done) {
return src([
'./db/**',
'./db/**/.*',
'mta.yaml'
],{base: './'})
.pipe(dest('dist'))
}
function changeSchema(done) {
//Check if schema name is passed as argument
if(!argv.hasOwnProperty("schema"))
handleError("ERR: Please specify a valid schema name", done)
return src('dist/db/src/**/*.hdbreptask', {base: './'})
.pipe(replace(/(?<=RemoteObjectUniqueName="")(.+?)(?=")/g, argv.schema))
.pipe(dest('.'));
}
function build(done){
exec('mbt build -p=cf -s=dist -t=../mta_archives', (err, stdout, stderr) => {
done(err);
});
}
exports.changeSchema = changeSchema
exports.default = series(clean, dist, changeSchema, build, clean)
What the script will do: (see the last line exports.default = series (…))
- Cleanup previous folders from older executions
- Copy the original files into a destination folder to only change a copy
- Change the schema value in the copied objects
- build the MTAR from the schema adjusted copy
- Cleanup afterwards
Executing the script
Open a terminal in Business Application Studio and execute
npm run make_production
This will generate the mtar with the changes to the replication tasks for you:
Better solution
Instead of the script above (which can still be useful in so many ways) what we could leverage is the functionality of schema alias replacement, which can be configured in the remote source configuration.
Here is the official documentation for the HANA adapter[2] check if the feature is existing for the adapter type you are using.
The configuration for our example remote source config in the production target DB above would look like this:
Remote source configuration for schema alias replacement in production
This would lead to a lookup onto the schema SCHEMA_PROD incase the SCHEMA_DEV is used.
According to the documentation:
“The value of this parameter can be changed when the remote source is suspended.”
alter remote source "Remote Source Name" suspend capture;
-- change remote source alias replacement
alter remote source "Remote Source Name" resume capture;
After this change the deployment into producton runs through successfully and the data can be accessed using the runtime catalog objects.
Infographic: Schema Alias Replacement and Suspend + Resume RS
Query on the generated virtual table from the replication task reads from the production schema:
query result on the generated virtual table
[3]
Altough the runtime objects point to the dev schema:
Special thanks go to Ingo Neumman for coming up with the idea of this script!
If you have further questions please ask them in the community here:
tl;dr
This blog post contains a script & howto guide on how you can make changes to any file you have in your Business Application Studio workspace. Specifically we used it for the purpose of renaming the .hdbreptask remote source schema definition but found out its unnecessary later on.
Cheers,
Yves
Additional resources
[1] https://developers.google.com/web/ilt/pwa/introduction-to-gulp
[2] https://help.sap.com/viewer/7952ef28a6914997abc01745fef1b607/2.0_SPS05/en-US/5d667b85725e4960b80550a0f75442ac.html