When I joined SAP five years ago I became a member of a small team of developers who were hired to obey one single commandment:
innovate! But the initial group was so small compared to the size of their goals that an important corollary was immediately appended:
automate everything! The zeal with which the team followed the directive made it possible to quickly grow and deliver
SAP Vora,
SAP Data Hub, and
SAP Data Intelligence in a remarkably short time.
Automation quickly spread across all the development and certification processes: as a result, SAP Data Intelligence itself became a treasure chest full of services and APIs for a complete programmatic control of the product. In the two previous episodes of Zen and the Art of SAP Data Intelligence, we presented two of these services: the Modeler REST API (
Episode 1) and the Connection Manager service (
Episode 2). In this post we will discover the most powerful member of the family: the
SAP Data Intelligence System Management Command-Line Client (aka
vctl), recently
released for productive usage with
SAP Data Hub 2.7 .
Prerequisites
- The steps of this post have been performed with Data Hub 2.7.3 and Data Intelligence 1910.1.11, but should be valid for any Data Hub ≥ 2.7 and any Data Intelligence ≥ 1910 versions.
- Recommended browser: Chrome.
- A basic knowledge of Unix commands is required.
- The execution of the snippets proposed in this blog post require a Unix terminal from where the Data Intelligence or Data Hub endpoints are reachable over the network.
- To complete Step 5, the admin credentials of the system tenant are needed, which is usually possible only if you are running an on-prem version of Data Hub or Data Intelligence.
Step 1: where is the System Management Command-Line Client?
This is the most difficult question of the entire post:
where is vctl? If you own an on-prem version of Data Hub or Data Intelligence the answer is a no-brainer: vctl can be downloaded with one click from the System Management application page.
The same executable can be found in the installation package you got from the SAP Market Place. As you can see from the example below, theinstallation package contains two versions of the client: one for Linux (vctl) and the other for Mac (vctl-darwin).
SAPDataHub-2.7.155-Foundation> ls -1 tools/
cert-driver
checkpoint_admin.sh
checkpoint_config.sh
common.sh
config_helper.py
dh.pem
diagnostics_collector.sh
export.sh
hl-hana-replication.jar
idp_configuration.py
images.sh
import_license.sh
import.sh
json_parser.py
log_collector.py
vctl
vctl-darwin
vsystem.sh
But what if you are using the Data Intelligence service on
SCP? In this case no download option and no installation package, hence no vctl executable. The reason is clear: vctl is a tool mostly meant for system administration and dev ops. Nevertheless, vctl works on the cloud just like it does on prem and I can think of many cases where it can be handy either way.
While waiting for SAP to make vctl available on SCP, cloud users can still try to find a compatible version of the client to install on their local machines. One possibility might be to
setup a trial version by following this
excellent tutorial by
anishagupta . In any case, you can always ask your SAP team for support.
Step 2: my first vctl command
Blame it on my being Italian, but I am firmly convinced that the first thing to do when you approach a new task is to ask for help. And vctl makes no exception:
./tools/vctl --help
SAP Data Hub System Management CLI
More information at https://help.sap.com/viewer/p/SAP_DATA_HUB
Usage:
vctl [flags]
vctl <command>
Available Commands:
completion Generate completion scripts for bash or zsh
help Help about any command
js Execute javascript
login Log in to SAP Data Hub System Management
parameter Manage application parameters (only available to admins)
passwd Changes the password of the current user
policy Manage policies
scheduler Manage application templates and instances
secret Manage secrets (only available to admins)
tenant Manage tenants and tenant strategies (only available to admins)
user Manage users
version Display the client and server version of SAP Data Hub System Management
vrep Interact with the file repository
whoami Print the tenant, username and role associated with the current user.
Flags:
--conn-file string Path to the connection file (default "/home/d064337/.vsystem/.vsystem_conn")
-h, --help help for vctl
-q, --quiet disable logging (overrides --verbose)
-v, --verbose verbose logging
Use "vctl <command> --help" for more information about a command.
If you get the output above, then you are all set to proceed to the next step. If not, just drop me a comment and we can check together if it is the case to open an SAP support ticket.
Step 3: login with your DI/DH credentials
Whatever you want to achieve with vctl, it all starts with authentication. You need the usual four items to successfully compete the login:
- vsystem ingress URL
- tenant
- user
- password
The login command is pretty straightforward to use:
./tools/vctl login --help
Log in to SAP Data Hub System Management at host and port specified in `<address>` using credentials `<tenant>` `<user>` and `<password>`.
If the -p option is not specified, you will be prompted for the password.
Usage:
vctl login <address> <tenant> <user> [-p <password>] [flags]
Flags:
--cert stringArray CA certificate for validating the SAP Data Hub System Management peer
-h, --help help for login
--insecure Skip validating SSL connections with installed CA bundles
--no-proxy Do not use proxies, even if the proxy environment variables are set
-p, --password string Password to login with
Global Flags:
--conn-file string Path to the connection file (default "/home/d064337/.vsystem/.vsystem_conn")
-q, --quiet disable logging (overrides --verbose)
-v, --verbose verbose logging
Better to input your password in the interactive mode instead of using the --password flag:
export cluster_url=<vsystem-ingress-url>
export tenant=<tenant-name>
export user=<user-name>
./tools/vctl login $cluster_url $tenant $user
Enter password: *************
Step 4: configure a tenant with vctl
You can use vctl to manage and configure pretty much every aspect of your system. You can create and delete users, change passwords, assign policies, manage your applications, and much more. As an example, the snippet below shows how to create three new
member users in the
default tenant.
#! /bin/bash -f
export role="member"
export tenant="default"
export pwd="PwdT0Ch@ng3"
for username in user01 user02 user03
do
echo creating user $username
./tools/vctl user create $tenant $username $pwd $role
done
To delete the users you can use the following:
#! /bin/bash -f
export role="member"
export tenant="default"
export pwd="T0Ch@ng3$$"
for username in user01 user02 user03
do
echo deleting user $username
./tools/vctl user delete $tenant $username
done
Step 5: configure the whole cluster with vctl
Needless to say that the vctl can perform only what the authenticated user is authorized to do. The two snippets in
Step 4, for instance, can be successfully executed only by a tenant administrator.
For on-prem installations, there is a special tenant called
system whose administrators have super powers on the whole cluster, especially if they use vctl. In the example below, the system administrator creates the new tenant called
development based on the default strategy
strat-default-2.7.155, then create the administrator of this tenant, and finally the member users.
export cluster_url=<vsystem-ingress-url>
export tenant="system"
export user=<system-admin-name>
./tools/vctl --verbose login $cluster_url $tenant $user
export strategy="strat-default-2.7.155"
export newtenant=<new-tenant-name>
export user=<new-user-name>
export role="tenantAdmin"
echo creating tenant $newtenant
./tools/vctl tenant create $newtenant
./tools/vctl tenant set-strategy $newtenant $strategy
echo creating new administrator user $user for tenant $newtenant
./tools/vctl user create $newtenant $username $pwd $role
export role="member"
export pwd="PwdT0Ch@ng3"
for username in user01 user02 user03
do
echo creating user $username
./tools/vctl user create $newtenant $username $pwd $role
done
Epilog
In this third episode of "
Zen and the art of SAP Data Intelligence" we learned how to use the
SAP Data Intelligence System Management command-line client. This tool is the key to automate and scale the system administration and the operations on your Data Intelligence on-prem cluster. I use it extensively to establish a CI/CD process across my Data Intelligence landscapes. But this is the topic for another post.
I encourage you to download vctl and try it out with the examples above. They are just scratching the surface of what this small executable can do. With a bit of patience and elbow grease you can unlock its full power, and believe me: it is worth the investment.
If you would like to use vctl for something more ambitious than the snippets above and wouldn't mind a blog post on that, just add your request in the comments. As usual: all feedbacks are more than welcome.
Thank you for reading!
For the philomaths
Further information about the topics treated in this blog post can be found in the following references: