Skip to Content
Technical Articles

Monitoring of ABAP System on AWS Using CloudWatch – Part I

This Article is referenced from the approach mentioned by Marcel Toerpe at SAP monitoring: A serverless approach using Amazon CloudWatch on SAP on AWS blogs. Check also his Github Repository for latest information on this solution

In this Article, we are going to discuss about an serverless approach of monitoring SAP based on Amazon CloudWatch and AWS Lambda. This approach describes us to effectively monitor SAP ABAP-based environments at a low cost without the need to deploy or manage any additional servers or agents using AWS Services. The solution can be deployed seamlessly with the AWS Serverless Application Repository or AWS CloudFormation. This allows us to easily publish our own metrics, such as SAP application-level performance data, and create thresholds and alarms in CloudWatch.

SAP has provided several notes which support SAP products on AWS:-

This Article will concentrate on the information more about AWS Services that can be used to monitor SAP, but relevant link has been provided for further information

1. AWS SERVICES

1.1 Amazon VPC

Amazon VPC helps us to define a virtual network in our own way, in a logically isolated area within the AWS Cloud. We can launch your AWS resources, such as SAP instances on EC2, into this VPC.  We can configure our VPC; we can select its IP address range, create subnets, and configure route tables, network gateways, and security settings.

1.2 AWS Lambda

AWS Lambda is a serverless compute service that lets us run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes. With Lambda, we can run code for virtually any type of application or backend service – all with zero administration.

1.3 Amazon CloudWatch

Amazon CloudWatch is a monitoring and observability service built for developers, site reliability engineers (SREs), and IT managers. CloudWatch provides us the data and actionable insights to monitor SAP application. CloudWatch collects SAP monitoring and operational data in the form of logs, metrics, and events, providing us with a unified view of SAP system that run on AWS and on-premises servers. We can use CloudWatch to detect anomalous behavior in our SAP environments, set alarms, visualize logs and metrics side by side, take automated actions, troubleshoot issues, and discover insights to keep our SAP applications running smoothly.

1.4 SAPJCO

The SAP Java Connector (SAP JCo) is a development library that helps a Java application to communicate with SAP systems via SAP’s RFC protocol. It combines an easy-to-use API with unprecedented flexibility and performance. The SAP JCo supports both communication directions: inbound Remote Function Calls (Java calls ABAP) as well as outbound Remote Function Calls (ABAP calls Java). SAP provides below notes for further information:

1.5 AWS Secrets Manager

AWS Secrets Manager helps us protecting secrets/credentials needed to access our SAP applications, services, and IT resources. The service enables us to easily rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle. Users and applications retrieve SAP secrets/credentials with a call to Secrets Manager APIs, eliminating the need to hardcode sensitive information in plain text.

1.6 AWS CloudFormation

AWS CloudFormation gives us an easy way to model a collection of related AWS and third-party resources, provision them quickly and consistently, and manage them throughout their lifecycles, by treating infrastructure as code. We can use a template to create, update, and delete an entire stack as a single unit, as often as we need to, instead of managing resources individually.

1.7 Amazon SNS

Using Amazon SNS topics, our publisher systems can fanout messages to many subscriber systems including Amazon SQS queues, AWS Lambda functions and HTTPS endpoints, for parallel processing, and Amazon Kinesis Data Firehose. The A2P functionality enables us to send messages to users at scale via SMS, mobile push, and email.

2. ARCHITECTURE

The following architecture diagram shows the monitoring setup and Flow that gets deployed with AWS CloudFormation

The generated Amazon CloudWatch rule triggers a Java-based Lambda function every minute and performs multiple Remote Function Calls (RFC) to the SAP system through the official SAP JCo Library, provided as Lambda layer. The connectivity is created by adding the Lambda to our existing Amazon Virtual Private Cloud (VPC) using also the latest improvements on networking for AWS Lambda.

The SAP RFC credentials, and connection information are stored securely inside AWS Secrets Manager and read on-demand to establish connectivity. The Lambda function extracts the SAP application level metrics, adds the respective custom namespace, and pushes it to CloudWatch. Users can then create and use their own custom dashboards inside CloudWatch.

CloudWatch also provide us feature to create Alarms based on Metrics which can be forwarded to Amazon Simple Notification Service (Amazon SNS) by which we can trigger email/SMS as per the settings.

3. PREREQUISITES

For implementing the above architecture below are some of the pre-requisite that needs to be done

3.1 SAP NetWeaver ABAP

SAP Engine to which Lambda function is going to connect must have SAP NetWeaver ABAP 740 or higher version with components could be any of ECC, BW, S4, SRM, APO etc.

We can check SAP’s installation guides for implementing or upgrading the same version on the SAP landscape.

System’s Status can be checked for version of SAP that has been implemented on the system.

3.2 ST-PI

For the lambda function to fetch data from the ABAP instance using the SAP JCo is possible only if SAP Component ST-PI Release 740 SP 08 or higher is installed on the system. As this release contains some of the important Functional modules such as SWNC_GET_WORKLOAD_SNAPSHOT, which is called by the java code for collection of data which is delivered by SAP later to the mentioned ST-PI Release. SAP Notes 907035 can be checked for further information.

System’s Status can be checked for version of ST-PI that has been implemented on the system.

3.3 Statistical Records

All configuration related to storing all the Statistical data of SAP system should be enabled. Data in T-code STAD and ST03 must be updating regularly with the latest data. SAP has provided guided procedure to keep Statistical data up to date. For more details below notes can be checked:-

We can check the Last Measurement Time to check if the data is getting updated in the system.

3.4 SAP RFC User

To allow AWS Lambda function to connect to SAP Instance with the required authorization, it is recommender to create a separate user for fetching such data from SAP. To do so, below steps needs to be done

Go to T-code PFCG and Upload the below Custom Role:-

DATE                                              20200423                                          153553
RELEASE                                           753
LOADED_AGRS                                       ZSAPMONITOR
AGR_DEFINE                                        000ZSAPMONITOR                                                 DEVELOPER   20200408085327000000000000000DEVELOPER   20200423153520000000000000000
AGR_TCODES                                        000ZSAPMONITOR                   TR/SDF/SMON                                        X 00000
AGR_TCODES                                        000ZSAPMONITOR                   TRSM50                                             X 00000
AGR_TCODES                                        000ZSAPMONITOR                   TRST03                                             X 00000
AGR_1250                                          000ZSAPMONITOR                   000001S_ADMI_FCDT-HB53000100    M  O000000
AGR_1250                                          000ZSAPMONITOR                   000002S_APPL_LOGT-HB53000100    SX O000000
AGR_1250                                          000ZSAPMONITOR                   000003S_BTCH_ADMT-HB53000100    U  O000000
AGR_1250                                          000ZSAPMONITOR                   000004S_DATASET T-HB53000100    SX O000000
AGR_1250                                          000ZSAPMONITOR                   000005S_RFC     T-HB53000100    U  O000000
AGR_1250                                          000ZSAPMONITOR                   000006S_TCODE   T-HB53000100    S  O000000
AGR_1250                                          000ZSAPMONITOR                   000007S_TOOLS_EXT-HB53000100    G  O000000
AGR_1250                                          000ZSAPMONITOR                   000008S_USER_GRPT-HB53000100    U  O000000
AGR_1251                                          000ZSAPMONITOR                   000001S_ADMI_FCDT-HB53000100    S_ADMI_FCDPADM                                                                            M  O000000
AGR_1251                                          000ZSAPMONITOR                   000002S_RFC     T-HB53000100    RFC_TYPE  FUNC                                                                            U  O000000
AGR_1251                                          000ZSAPMONITOR                   000003S_RFC     T-HB53000100    RFC_NAME  SWNC_GET_WORKLOAD_SNAPSHOT                                                      U  O000000
AGR_1251                                          000ZSAPMONITOR                   000004S_RFC     T-HB53000100    RFC_NAME  RFC_GET_FUNCTION_INTERFACE                                                      U  O000000
AGR_1251                                          000ZSAPMONITOR                   000005S_RFC     T-HB53000100    RFC_NAME  RFCPING                                                                         U  O000000
AGR_1251                                          000ZSAPMONITOR                   000006S_RFC     T-HB53000100    RFC_NAME  DDIF_FIELDINFO_GET                                                              U  O000000
AGR_1251                                          000ZSAPMONITOR                   000007S_RFC     T-HB53000100    RFC_NAME  /SDF/SMON_GET_SMON_RUNS                                                         U  O000000
AGR_1251                                          000ZSAPMONITOR                   000008S_RFC     T-HB53000100    RFC_NAME  /SDF/SMON_ANALYSIS_START                                                        U  O000000
AGR_1251                                          000ZSAPMONITOR                   000009S_RFC     T-HB53000100    RFC_NAME  /SDF/SMON_ANALYSIS_READ                                                         U  O000000
AGR_1251                                          000ZSAPMONITOR                   000010S_TCODE   T-HB53000100    TCD       /SDF/SMON                                                                       S  O000000
AGR_1251                                          000ZSAPMONITOR                   000011S_TCODE   T-HB53000100    TCD       ST03                                                                            S  O000000
AGR_1251                                          000ZSAPMONITOR                   000012S_ADMI_FCDT-HB53000100    S_ADMI_FCDST0R                                                                            M  O000000
AGR_1251                                          000ZSAPMONITOR                   000013S_TOOLS_EXT-HB53000100    AUTH      S_TOOLS_EX_A                                                                    G  O000000
AGR_1251                                          000ZSAPMONITOR                   000014S_RFC     T-HB53000100    ACTVT     16                                                                              U  O000000
AGR_1251                                          000ZSAPMONITOR                   000015S_USER_GRPT-HB53000100    CLASS     *                                                                               U  O000000
AGR_1251                                          000ZSAPMONITOR                   000016S_ADMI_FCDT-HB53000100    S_ADMI_FCDST0M                                                                            M  O000000
AGR_1251                                          000ZSAPMONITOR                   000017S_USER_GRPT-HB53000100    ACTVT     03                                                                              U  O000000
AGR_1251                                          000ZSAPMONITOR                   000018S_APPL_LOGT-HB53000100    ACTVT     03                                                                              SX O000000
AGR_1251                                          000ZSAPMONITOR                   000019S_APPL_LOGT-HB53000100    ALG_OBJECTBCSGLWL                                                                         SX O000000
AGR_1251                                          000ZSAPMONITOR                   000020S_APPL_LOGT-HB53000100    ALG_OBJECTBCSWNC                                                                          SX O000000
AGR_1251                                          000ZSAPMONITOR                   000021S_APPL_LOGT-HB53000100    ALG_SUBOBJ                                                                                SX O000000
AGR_1251                                          000ZSAPMONITOR                   000022S_TCODE   T-HB53000100    TCD       SM50                                                                            S  O000000
AGR_1251                                          000ZSAPMONITOR                   000023S_BTCH_ADMT-HB53000100    BTCADMIN  D                                                                               U  O000000
AGR_1251                                          000ZSAPMONITOR                   000024S_DATASET T-HB53000100    PROGRAM   SAPLSDEB                                                                        SX O000000
AGR_1251                                          000ZSAPMONITOR                   000025S_DATASET T-HB53000100    PROGRAM   SAPLCRFC                                                                        SX O000000
AGR_1251                                          000ZSAPMONITOR                   000026S_DATASET T-HB53000100    FILENAME                                                                                  SX O000000
AGR_1251                                          000ZSAPMONITOR                   000027S_DATASET T-HB53000100    ACTVT                                                                                     SX O000000
AGR_1251                                          000ZSAPMONITOR                   000028S_BTCH_ADMT-HB53000100    BTCADMIN  Y                                                                               U  O000000
AGR_1251                                          000ZSAPMONITOR                   000029S_RFC     T-HB53000100    RFC_NAME  BDL_GET_CENTRAL_TIMESTAMP                                                       U  O000000
AGR_1251                                          000ZSAPMONITOR                   000030S_RFC     T-HB53000100    RFC_NAME  RFC_METADATA_GET                                                                U  O000000
AGR_TEXTS                                         000ZSAPMONITOR                   E00000SAP Monitoring via Amazon CloudWatch
AGR_FLAGS                                         000ZSAPMONITOR                   COLL_AGR  DEVELOPER   20200408085327DEVELOPER   20200408085327
AGR_FLAGS                                         000ZSAPMONITOR                   DEVCLASS  DEVELOPER   20200408085327DEVELOPER   20200408085327
AGR_FLAGS                                         000ZSAPMONITOR                   MASTER_LANDEVELOPER   20200408085327DEVELOPER   20200408085327E
AGR_FLAGS                                         000ZSAPMONITOR                   RESP_USER DEVELOPER   20200408085327DEVELOPER   20200408085327
AGR_FLAGS                                         000ZSAPMONITOR                   FORCE_MIX DEVELOPER   20200408085718DEVELOPER   20200423153518
AGR_HIER                                          000ZSAPMONITOR                   0000000200000001 00000010TR/SDF/SMON                                       01                                X                                         0000000000
AGR_HIER                                          000ZSAPMONITOR                   0000000300000001 00000020TRST03                                            01                                X                                         0000000000
AGR_HIER                                          000ZSAPMONITOR                   0000000400000001 00000030TRSM50                                            01                                X                                         0000000000
AGR_HIERT                                         000ZSAPMONITOR                   D00000002Snapshot-Monitor
AGR_HIERT                                         000ZSAPMONITOR                   E00000002Snapshot Monitor
AGR_HIERT                                         000ZSAPMONITOR                   D00000003Systemlast u. Perform. Statistik
AGR_HIERT                                         000ZSAPMONITOR                   E00000003Workload and Performance Statistics
AGR_HIERT                                         000ZSAPMONITOR                   D00000004Workprozesse einer AS-Instanz
AGR_HIERT                                         000ZSAPMONITOR                   E00000004Work Processes of AS Instance
AGR_HIERT                                         000ZSAPMONITOR                   F00000004Synthèse des processus
AGR_HIERT                                         000ZSAPMONITOR                   I00000004Riepilogo processo di lavoro
AGR_HIERT                                         000ZSAPMONITOR                   N00000004Work-process-overzicht
AGR_TIME                                          000ZSAPMONITOR                   MENU      DEVELOPER   20200408085435000000000000000DEVELOPER   20200408093202000000000000000
AGR_TIME                                          000ZSAPMONITOR                   PROFILE   DEVELOPER   20200408085718000000000000000DEVELOPER   20200423153518000000000000000
AGR_LSD                                           000ZSAPMONITOR        

Check the list of Roles that is inside the file and then proceed

Since the role is uploaded explicitly hence it is mandatory to generate the profiles for the same, to do so we need to go to Authorization Data

And then click Generate to populate the profiles for the role

After successful generation of profiles, we can see that same in the Role properties

Assign this role to the user which will be used by Lambda Function for fetching the Statistical data, we can either choose to create new user or existing user. It is recommended to use user as type System, as this User doesn’t need SAPGUI to fetch the data.

User Comparison needs to be done for completing the role assignment process

3.5 SAP JCo Lambda Layer

For connecting Lambda function to SAP Instance, we need to use Java code which will use SAP JCo Library

3.5.1 Prepare SAP JCo Library

To create SAP JCo Library Lambda Layer, we first need to download Latest SAP JAVA Connector library from SAP Marketplace. More information can be checked on below notes:

This will download the file as zip to the local drive of the machine

Extract all the files which are available in the zip

Extraction has below files which are provided by SAP as part of Java Connector

Create a new folder java with sub-folder lib and add the below highlighted files

Then create a zip of java directory using standard OS tool

Cross check the files inside the java.zip using the OS standard tool

3.5.2 Create Lambda Layer

Go to AWS Lambda from AWS Console → click on Layer then click on Create Layer

Enter name of the Lambda Layer and upload the java.zip file. Also, specify the Compatible runtimes as JAVA 11 and 8. Click on Create to proceed with the creation of the lambda Layer

Below screen will appear after a successful creation of the Lambda Layer

This is end of Part – I, in next part Monitoring of ABAP System on AWS Using CloudWatch – Part II of this article you can find more information about Deployment, Testing, Post Configuration, Dashboard etc… of Monitoring ABAP Systems via AWS.

Be the first to leave a comment
You must be Logged on to comment or reply to a post.