Technical Articles
Zen and the Art of SAP Data Intelligence. Episode 3: vctl, the hidden pearl you must know
When I joined SAP five years ago I became a member of a small team of developers who were hired to obey one single commandment: innovate! But the initial group was so small compared to the size of their goals that an important corollary was immediately appended: automate everything! The zeal with which the team followed the directive made it possible to quickly grow and deliver SAP Vora, SAP Data Hub, and SAP Data Intelligence in a remarkably short time.
Automation quickly spread across all the development and certification processes: as a result, SAP Data Intelligence itself became a treasure chest full of services and APIs for a complete programmatic control of the product. In the two previous episodes of Zen and the Art of SAP Data Intelligence, we presented two of these services: the Modeler REST API (Episode 1) and the Connection Manager service (Episode 2). In this post we will discover the most powerful member of the family: the SAP Data Intelligence System Management Command-Line Client (aka vctl), recently released for productive usage with SAP Data Hub 2.7 .
Prerequisites
- The steps of this post have been performed with Data Hub 2.7.3 and Data Intelligence 1910.1.11, but should be valid for any Data Hub ≥ 2.7 and any Data Intelligence ≥ 1910 versions.
- Recommended browser: Chrome.
- A basic knowledge of Unix commands is required.
- The execution of the snippets proposed in this blog post require a Unix terminal from where the Data Intelligence or Data Hub endpoints are reachable over the network.
- To complete Step 5, the admin credentials of the system tenant are needed, which is usually possible only if you are running an on-prem version of Data Hub or Data Intelligence.
Step 1: where is the System Management Command-Line Client?
This is the most difficult question of the entire post: where is vctl? If you own an on-prem version of Data Hub or Data Intelligence the answer is a no-brainer: vctl can be downloaded with one click from the System Management application page.
The same executable can be found in the installation package you got from the SAP Market Place. As you can see from the example below, theinstallation package contains two versions of the client: one for Linux (vctl) and the other for Mac (vctl-darwin).
SAPDataHub-2.7.155-Foundation> ls -1 tools/ cert-driver checkpoint_admin.sh checkpoint_config.sh common.sh config_helper.py dh.pem diagnostics_collector.sh export.sh hl-hana-replication.jar idp_configuration.py images.sh import_license.sh import.sh json_parser.py log_collector.py vctl vctl-darwin vsystem.sh
But what if you are using the Data Intelligence service on SCP? In this case no download option and no installation package, hence no vctl executable. The reason is clear: vctl is a tool mostly meant for system administration and dev ops. Nevertheless, vctl works on the cloud just like it does on prem and I can think of many cases where it can be handy either way.
While waiting for SAP to make vctl available on SCP, cloud users can still try to find a compatible version of the client to install on their local machines. One possibility might be to setup a trial version by following this excellent tutorial by anisha gupta . In any case, you can always ask your SAP team for support.
Step 2: my first vctl command
Blame it on my being Italian, but I am firmly convinced that the first thing to do when you approach a new task is to ask for help. And vctl makes no exception:
./tools/vctl --help SAP Data Hub System Management CLI More information at https://help.sap.com/viewer/p/SAP_DATA_HUB Usage: vctl [flags] vctl <command> Available Commands: completion Generate completion scripts for bash or zsh help Help about any command js Execute javascript login Log in to SAP Data Hub System Management parameter Manage application parameters (only available to admins) passwd Changes the password of the current user policy Manage policies scheduler Manage application templates and instances secret Manage secrets (only available to admins) tenant Manage tenants and tenant strategies (only available to admins) user Manage users version Display the client and server version of SAP Data Hub System Management vrep Interact with the file repository whoami Print the tenant, username and role associated with the current user. Flags: --conn-file string Path to the connection file (default "/home/d064337/.vsystem/.vsystem_conn") -h, --help help for vctl -q, --quiet disable logging (overrides --verbose) -v, --verbose verbose logging Use "vctl <command> --help" for more information about a command.
If you get the output above, then you are all set to proceed to the next step. If not, just drop me a comment and we can check together if it is the case to open an SAP support ticket.
Step 3: login with your DI/DH credentials
Whatever you want to achieve with vctl, it all starts with authentication. You need the usual four items to successfully compete the login:
- vsystem ingress URL
- tenant
- user
- password
The login command is pretty straightforward to use:
./tools/vctl login --help Log in to SAP Data Hub System Management at host and port specified in `<address>` using credentials `<tenant>` `<user>` and `<password>`. If the -p option is not specified, you will be prompted for the password. Usage: vctl login <address> <tenant> <user> [-p <password>] [flags] Flags: --cert stringArray CA certificate for validating the SAP Data Hub System Management peer -h, --help help for login --insecure Skip validating SSL connections with installed CA bundles --no-proxy Do not use proxies, even if the proxy environment variables are set -p, --password string Password to login with Global Flags: --conn-file string Path to the connection file (default "/home/d064337/.vsystem/.vsystem_conn") -q, --quiet disable logging (overrides --verbose) -v, --verbose verbose logging
Better to input your password in the interactive mode instead of using the –password flag:
export cluster_url=<vsystem-ingress-url> export tenant=<tenant-name> export user=<user-name> ./tools/vctl login $cluster_url $tenant $user Enter password: *************
Step 4: configure a tenant with vctl
You can use vctl to manage and configure pretty much every aspect of your system. You can create and delete users, change passwords, assign policies, manage your applications, and much more. As an example, the snippet below shows how to create three new member users in the default tenant.
#! /bin/bash -f export role="member" export tenant="default" export pwd="PwdT0Ch@ng3" for username in user01 user02 user03 do echo creating user $username ./tools/vctl user create $tenant $username $pwd $role done
To delete the users you can use the following:
#! /bin/bash -f export role="member" export tenant="default" export pwd="T0Ch@ng3$$" for username in user01 user02 user03 do echo deleting user $username ./tools/vctl user delete $tenant $username done
Step 5: configure the whole cluster with vctl
Needless to say that the vctl can perform only what the authenticated user is authorized to do. The two snippets in Step 4, for instance, can be successfully executed only by a tenant administrator.
For on-prem installations, there is a special tenant called system whose administrators have super powers on the whole cluster, especially if they use vctl. In the example below, the system administrator creates the new tenant called development based on the default strategy strat-default-2.7.155, then create the administrator of this tenant, and finally the member users.
export cluster_url=<vsystem-ingress-url> export tenant="system" export user=<system-admin-name> ./tools/vctl --verbose login $cluster_url $tenant $user export strategy="strat-default-2.7.155" export newtenant=<new-tenant-name> export user=<new-user-name> export role="tenantAdmin" echo creating tenant $newtenant ./tools/vctl tenant create $newtenant ./tools/vctl tenant set-strategy $newtenant $strategy echo creating new administrator user $user for tenant $newtenant ./tools/vctl user create $newtenant $username $pwd $role export role="member" export pwd="PwdT0Ch@ng3" for username in user01 user02 user03 do echo creating user $username ./tools/vctl user create $newtenant $username $pwd $role done
Epilog
In this third episode of “Zen and the art of SAP Data Intelligence” we learned how to use the SAP Data Intelligence System Management command-line client. This tool is the key to automate and scale the system administration and the operations on your Data Intelligence on-prem cluster. I use it extensively to establish a CI/CD process across my Data Intelligence landscapes. But this is the topic for another post.
I encourage you to download vctl and try it out with the examples above. They are just scratching the surface of what this small executable can do. With a bit of patience and elbow grease you can unlock its full power, and believe me: it is worth the investment.
If you would like to use vctl for something more ambitious than the snippets above and wouldn’t mind a blog post on that, just add your request in the comments. As usual: all feedbacks are more than welcome.
Thank you for reading!
For the philomaths
Further information about the topics treated in this blog post can be found in the following references:
- The SAP Data Intelligence System Management command-line client is described in the official System Management Command-Line Client Reference.
- For the administration of the on-prem SAP Data Hub systems, you can check the official Administration Guide for SAP Data Hub.
- For the administration of the SAP Data Intelligence cloud services, you can check the official Administration Guide for SAP Data Intelligence.
- The concept of strategies in SAP Data Intelligence and SAP Data Hub is explained here.
Hello Gianluca
we are also using vctl with the latest "On-Premise" Version of SAP Datahub 2.7.3 (2.7.155) in our Landscapes.
See some additional findings in the Blog: prepare the Installation Host for the SLC Bridge
Best Regards Roland
Thank you for the link Roland, great post and lots of info therein.
Hello Gianluca,
Thank you for providing good stuffs...
Hi Gianluca
What are the steps equivalent to Step5 for Cloud version then ?
Regards
Leena
Hello Gianluca,
I have a cloud version of SAP DI and have tenant admin rights, but I couldn't see the Download VCTL Tool options.