This is in continuation of my previous blog post Backup and Recovery of SAP HANA Database Directly to AWS S3 Bucket using AWS Backint Agent – Part II
To enable SAP HANA backups using AWS Backint Agent, it is required to set up SAP HANA parameters. If chosen option was “modify” for the global.ini file update, the SSM document or AWS Backint installer adds or updates the following backup related SAP HANA parameters in global.ini for the system database. If option chosen was “sql”, then it is required to run the SQL statements manually specified in the modify_global_ini.sql file to update these parameters.
In A YAML file location in the /<installation directory>/aws-backint-agent/ directory has all the AWS Backint Agent configuration parameters. The name of the configuration file is aws-backint-agent-config.yaml. The following table has the configuration parameters added as part of the AWS Backint Agent installation process, and additional parameters which can be added as per the requirements.
Default values during installation of AWS Backint Agent is already optimize the performance of backup and restore operations. But still if further optimize needs to be done for the performance of backup and restore operations, then paramters UploadChannelSize, UploadConcurrency and MaximumConcurrentFilesForRestore can be adjusted accordingly. Make sure of using the right instance type and storage configurations to get the best performance, as AWS Backint Agent is constrained by the resources available in the instance.
After adjusting(up to the allowed maximum), these 3 parameters needs to be added in aws-backint-agent-config.yaml file.
It is recommended to test both the backup and recovery operations after any change, to make sure that there is no unintended impact to backup and restore operations, as well as to other standard operations.
Additionally, to get the best performance during backup and restore operations, make surethat SAP HANA data and log volumes are configured following the best practices from AWS.
Below mentioned are limited number of examples of SQL statements which are only specific to backint that can be used to perform backup and recovery. SAP HANA Administration, or SQL Reference guides needs to be followed to find the required syntax of all the other options for specific SAP HANA version.
SAP has provided note 2705942 - How-To: Backup tenant from HANA Studio using SQL and 2658020 - List of SAP HANA SQL Error Codes for further suppport in SQL for HANA.
To initiate a full data backup of the system database, below SQL can be used--
To initiate a full data backup of the tenant database, below SQL can be used--
To initiate a differential data backup of the tenant database, below SQL can be used:
To initiate a differential data backup of the tenant database, below SQL can be used:
To initiate an incremental data backup of the tenant database, below SQL can be used:
To recover tenant database to a point in time, below SQL can be used:
To recover tenant database with a specific data backup using catalogs stored in S3, below SQL can be used:
To recover tenant database with a specific data backup without using a catalog, below SQL can be used:
In addition , SAP HANA Studio can also be used to backup and recover the HANA database from AWS S3 Bucket. This process internally use AWS Backint Agent to perform the operations. For further information, please check SAP guided procedure to use HANA studio for Backup and Recovery.
In addition , SAP HANA Cockpit can also be used to backup and recover the HANA database from AWS S3 Bucket. This process internally use AWS Backint Agent to perform the operations. For further information, please check SAP guided procedure to use HANA studio for Backup and Recovery.
SAP has provided below notes to support HANA Cockpit in terms of backup:-
Backup files can be verified in Amazon S3 bucket from the Amazon S3 console or by using APIs. AWS Backint Agent stores the backup files using a designated folder structure within Amazon S3 bucket.
During backup and restore, SAP HANA uses the below folder structure to stream data into a pipe so that Backint agent can read and write. So, AWS Backint Agent maintains this same folder structure in the Amazon S3 bucket as well. It is recommended that not to change this structure after the back up of the files. Changing this folder structure can cause issues during the restore operation and impact recoverability.
For system and tenant databases, data, log, and catalog backups can be found in the following locations. Data backups will include an additional prefix that was used during the backup.
Data and Log backups can be found inside the databases
HANA backups can be scheduled using ABAP transaction DBACOCKPIT, SAP has provided multiple documentation which can be used to Schedule the backups via DBA Planning Calendar
HANA Backups can also be scheduled using Scripts via OS schedulers such as crontab, SAP has provided scripts which we can use for scheduling the backups
HANA backups can also be scheduled using HANA Cockpit, SAP has provided step by step guide which can be used to schedule backups
HANA backups can also be scheduled using inbuild HANA XS. Both SAP HANA Extended Services classic model and advanced model provide job-scheduling features so that you can run routine administration tasks in the background at planned intervals.
Beginning with SAP HANA 2 SPS 03, SAP HANA Cockpit can be used to set the retention policies for SAP HANA database backups. Based on the retention policies, SAP HANA Cockpit can automatically trigger jobs to delete old backups from catalogs, as well as the physical backups. This process also automatically deletes backup files stored in Amazon S3 buckets.
SAP has provided note 2950547 - How to delete old data backup and log backup automatically?
AWS S3 Bucket Lifecycle can be used to expire certain objects based on the prefix rules and same can be deleted after certain interval of time. Todo so need to follow the below procedure:
Goto Amazon S3 → select bucket → Management → Click Create Lifecycle Rule
Specify the Rule name and prefix to expire the objects in the bucket
Specify the Lifecycle rules and number of days after which lifecycle rule will apply
Cross-check summary and click Save
Create all the other rules which respect to different databases for log and data
Below is the location where AWS Backing Agent Log can be found:
Below is the location where Backup or Recovery Logs of the HANA System database can be found:
Below is the location where Backup or Recovery Logs of the HANA Tenant databases can be found:
Now SAP HANA installed on AWS EC2 is getting backed up directly to Amazon S3 Bucket. BEST OF LUCK!!
In this 3 parts of blog post, we have discussed how we can install, configure and manager Backup and Recovery of SAP HANA Database Directly to AWS S3 Bucket using AWS Backint Agent.
THANK YOU..
5. CONFIGRUATIONS
5.1 Backint-related SAP HANA Parameters
To enable SAP HANA backups using AWS Backint Agent, it is required to set up SAP HANA parameters. If chosen option was “modify” for the global.ini file update, the SSM document or AWS Backint installer adds or updates the following backup related SAP HANA parameters in global.ini for the system database. If option chosen was “sql”, then it is required to run the SQL statements manually specified in the modify_global_ini.sql file to update these parameters.
SSH Console
5.2 AWS Backint Agent configuration parameters
In A YAML file location in the /<installation directory>/aws-backint-agent/ directory has all the AWS Backint Agent configuration parameters. The name of the configuration file is aws-backint-agent-config.yaml. The following table has the configuration parameters added as part of the AWS Backint Agent installation process, and additional parameters which can be added as per the requirements.
| Name of the parameter | Description | Default value |
| S3StorageClass | This parameter specifies the S3 Storage Class type that AWS Backint Agent can use while storing the backup files. Options allowed STANDARD, STANDARD_IA, or ONEZONE_IA. | STANDARD |
| EnableTagging | This parameter is used to enable or disable the default Object tagging for backups files stored in S3. Tagging helps to identify the AWS Backint version and SAP HANA version used during the backup. Options allowed: true or false. | True |
| LogLevel | This parameter used for the logging level for agent logs. Options allowed: info and debug. | Info |
| UploadConcurrency | This parameter used to define the number of S3 threads that can work in parallel during backup. Options Allowed: 1 to 200. | 100 |
| UploadChannelSize | This parameter used to define the number of files that can be uploaded in parallel to the S3 bucket during the backups. Options allowed: 1 to 32. | 10 |
| MaximumConcurrentFilesForRestore | This parameter used to define the number of files that can be downloaded in parallel from S3 during the restore. Options allowed: 1 to 32. | 5 |
5.3 Parameter Tuning
Default values during installation of AWS Backint Agent is already optimize the performance of backup and restore operations. But still if further optimize needs to be done for the performance of backup and restore operations, then paramters UploadChannelSize, UploadConcurrency and MaximumConcurrentFilesForRestore can be adjusted accordingly. Make sure of using the right instance type and storage configurations to get the best performance, as AWS Backint Agent is constrained by the resources available in the instance.
After adjusting(up to the allowed maximum), these 3 parameters needs to be added in aws-backint-agent-config.yaml file.
It is recommended to test both the backup and recovery operations after any change, to make sure that there is no unintended impact to backup and restore operations, as well as to other standard operations.
Additionally, to get the best performance during backup and restore operations, make surethat SAP HANA data and log volumes are configured following the best practices from AWS.
6. BACKUP AND RECOVERY
6.1 Using SQL Statements
Below mentioned are limited number of examples of SQL statements which are only specific to backint that can be used to perform backup and recovery. SAP HANA Administration, or SQL Reference guides needs to be followed to find the required syntax of all the other options for specific SAP HANA version.
SAP has provided note 2705942 - How-To: Backup tenant from HANA Studio using SQL and 2658020 - List of SAP HANA SQL Error Codes for further suppport in SQL for HANA.
To initiate a full data backup of the system database, below SQL can be used--
BACKUP DATA USING BACKINT ('/usr/sap/<SID>/SYS/global/hdb/backint/SYSTEMDB/<MY_PREFIX>')To initiate a full data backup of the tenant database, below SQL can be used--
BACKUP DATA FOR <TENANT DB ID> USING BACKINT ('/usr/sap/<SID>/SYS/global/hdb/backint/DB_<TENANT DB ID>/<MY_PREFIX >')To initiate a differential data backup of the tenant database, below SQL can be used:
BACKUP DATA DIFFERENTIAL FOR <TENANT DB ID> USING BACKINT ('/usr/sap/<SID>/SYS/global/hdb/backint/DB_<TENANT DB ID>/<MY_PREFIX >')To initiate a differential data backup of the tenant database, below SQL can be used:
BACKUP DATA DIFFERENTIAL FOR <TENANT DB ID> USING BACKINT ('/usr/sap/<SID>/SYS/global/hdb/backint/DB_<TENANT DB ID>/<MY_PREFIX >')To initiate an incremental data backup of the tenant database, below SQL can be used:
BACKUP DATA INCREMENTAL FOR <TENANT DB ID> USING BACKINT ('/usr/sap/<SID>/SYS/global/hdb/backint/DB_<TENANT DB ID>/<MY_PREFIX >')To recover tenant database to a point in time, below SQL can be used:
RECOVER DATABASE FOR <TENANT DB ID> UNTIL TIMESTAMP 'YYYY-MM-DD HH:MM:SS' USING DATA PATH ('/usr/sap/<SID>/SYS/global/hdb/backint/DB_<TENANT DB ID>/') USING LOG PATH ('/usr/sap/<SID>/SYS/global/hdb/backint/DB_<TENANT DB ID>') USING BACKUP_ID 1234567890123 CHECK ACCESS USING BACKINTTo recover tenant database with a specific data backup using catalogs stored in S3, below SQL can be used:
RECOVER DATA FOR <TENANT DB ID> USING BACKUP_ID 1234567890123 USING CATALOG BACKINT USING DATA PATH ('/usr/sap/<SID>/SYS/global/hdb/backint/DB_<TENANT DB ID>/') CLEAR LOGTo recover tenant database with a specific data backup without using a catalog, below SQL can be used:
RECOVER DATA FOR <TENANT DB ID> USING BACKINT ('/usr/sap/<SID>/SYS/global/hdb/backint/DB_<TENANT DB ID>/<MY_PREFIX >') CLEAR LOG6.1 Using HANA Studio
In addition , SAP HANA Studio can also be used to backup and recover the HANA database from AWS S3 Bucket. This process internally use AWS Backint Agent to perform the operations. For further information, please check SAP guided procedure to use HANA studio for Backup and Recovery.
6.2 Using HANA Cockpit
In addition , SAP HANA Cockpit can also be used to backup and recover the HANA database from AWS S3 Bucket. This process internally use AWS Backint Agent to perform the operations. For further information, please check SAP guided procedure to use HANA studio for Backup and Recovery.
SAP has provided below notes to support HANA Cockpit in terms of backup:-
- 2699762 - Backup and Recovery: Software Requirements for Scheduling Backups in SAP HANA Cockpit
- 2484147 - Schedule backups for multi tenant HANA cockpit 2.0
- 2185556 - FAQ: SAP HANA Cockpit (delivered with SAP HANA 1.0)
6.3 Search Backups in S3 Bucket
Backup files can be verified in Amazon S3 bucket from the Amazon S3 console or by using APIs. AWS Backint Agent stores the backup files using a designated folder structure within Amazon S3 bucket.
During backup and restore, SAP HANA uses the below folder structure to stream data into a pipe so that Backint agent can read and write. So, AWS Backint Agent maintains this same folder structure in the Amazon S3 bucket as well. It is recommended that not to change this structure after the back up of the files. Changing this folder structure can cause issues during the restore operation and impact recoverability.
For system and tenant databases, data, log, and catalog backups can be found in the following locations. Data backups will include an additional prefix that was used during the backup.
<awsdoc-example-bucket>/<optional-my-folder>/<SID>/usr/sap/<SID>/SYS/global/hdb/backint/SYSTEMDB/
<awsdoc-example-bucket>/<optional-my-folder>/<SID>/usr/sap/<SID>/SYS/global/hdb/backint/DB_<Tenant ID>/AWS S3 Console
Data and Log backups can be found inside the databases
AWS S3 Console
7. MANAGING BACKUPS
7.1 Scheduling Backups
7.1.1 Using ABAP DBACockpit
HANA backups can be scheduled using ABAP transaction DBACOCKPIT, SAP has provided multiple documentation which can be used to Schedule the backups via DBA Planning Calendar
- 2164096 - Schedule backups for SAP HANA multiple-container systems with DB13
- https://help.sap.com/viewer/6b8fe8492ce14d24af5855c3d10701e3/117/en-US/1619b9a6e71f4e2b8f70e403be3ca...
- https://blogs.sap.com/2016/09/12/schedule-backups-for-sap-hana-multitenant-database-containers-with-...
7.1.2 Using Backup Scripts
HANA Backups can also be scheduled using Scripts via OS schedulers such as crontab, SAP has provided scripts which we can use for scheduling the backups
- 1651055 - Scheduling SAP HANA Database Backups in Linux
- 1950261 - SAP HANA Database Backup Policy Recommendations and Regular Backup Script
7.1.3 Using HANA Cockpit
HANA backups can also be scheduled using HANA Cockpit, SAP has provided step by step guide which can be used to schedule backups
- https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/1.0.12/en-US/6d385fa14ef64a6bab2c97a3d3...
- https://wiki.scn.sap.com/wiki/display/SAPHANA/How+to+Back+Up+and+Recover+the+SAP+HANA+Cockpit
- https://blogs.sap.com/2018/04/13/schedule-hana-2.0-backup-using-sap-hana-cockpit-2.0-sps-06/
7.1.4 Using HANA XS
HANA backups can also be scheduled using inbuild HANA XS. Both SAP HANA Extended Services classic model and advanced model provide job-scheduling features so that you can run routine administration tasks in the background at planned intervals.
- https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.04/en-US/445b9667c4aa4a7b9a17b9b45e...
- https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.04/en-US/791a171998384f029ae544f0b2...
7.2 Backup Retentions
7.2.1 Using HANA Cockpit
Beginning with SAP HANA 2 SPS 03, SAP HANA Cockpit can be used to set the retention policies for SAP HANA database backups. Based on the retention policies, SAP HANA Cockpit can automatically trigger jobs to delete old backups from catalogs, as well as the physical backups. This process also automatically deletes backup files stored in Amazon S3 buckets.
SAP has provided note 2950547 - How to delete old data backup and log backup automatically?
7.2.3 Using S3 Bucket Lifecycle
AWS S3 Bucket Lifecycle can be used to expire certain objects based on the prefix rules and same can be deleted after certain interval of time. Todo so need to follow the below procedure:
Goto Amazon S3 → select bucket → Management → Click Create Lifecycle Rule
AWS S3 Console
Specify the Rule name and prefix to expire the objects in the bucket
AWS S3 Console
Specify the Lifecycle rules and number of days after which lifecycle rule will apply
AWS S3 Console
Cross-check summary and click Save
AWS S3 Console
Create all the other rules which respect to different databases for log and data
AWS S3 Console
8. TROUBLESHOOTING LOGS
8.1 AWS Backint Agent Log
Below is the location where AWS Backing Agent Log can be found:
{INSTALLATION DIRECTORY}/aws-backint-agent/aws-backint-agent.logSSH Console
8.2 SystemDB Backup/Recovery Logs
Below is the location where Backup or Recovery Logs of the HANA System database can be found:
/usr/sap/<SID>/HDB<Instance No>/<hostname>/trace/backup.log
/usr/sap/<SID>/HDB<Instance No>/<hostname>/trace/backint.logSSH Console
8.3 TenantDB Backup/Recovery Logs
Below is the location where Backup or Recovery Logs of the HANA Tenant databases can be found:
/usr/sap/<SID>/HDB<Instance No>/<hostname>/trace/DB_<TENANT>/backup.log
/usr/sap/<SID>/HDB<Instance No>/<hostname>/trace/DB_<TENANT>/backint.logSSH Console
Now SAP HANA installed on AWS EC2 is getting backed up directly to Amazon S3 Bucket. BEST OF LUCK!!
9. CONCLUSION:
In this 3 parts of blog post, we have discussed how we can install, configure and manager Backup and Recovery of SAP HANA Database Directly to AWS S3 Bucket using AWS Backint Agent.
THANK YOU..
- SAP Managed Tags:
