Near-Zero Downtime Maintenance for SAP Business Suite Systems
Near-Zero Downtime Maintenance for SAP Business Suite Systems
Improvements of downtime minimization capabilities for Software Update Manager (SUM)
The downtime minimization feature nZDM for SUM has been general available for some months.
After a very successful period of controlled projects on customer site all over the world this feature is now included in the Software Update Manager 1.0 (SUM). Until it was shipped with SP7 it has been used on customer side in more than 150 executions, 40 productively. It is a main improvement to reduce the business downtime significantly.
The nZDM feature in SUM is still considered for continuous improvement activities.
What do you have to invest?
There are no separate license fees for using the downtime minimization capabilities of SUM. SUM is part of the SL Toolset and the newest version can be downloaded from the SAP Service Market Place.
This document describes the new available feature and the benefits you can expect regarding downtime optimization of SUM. It will be regularly updated to keep you up to date.
Benefits of nZDM for SUM
In SUM you have the option to use one of the following pre-configurations:
- “Single System” = no use of shadow instance (update) or shadow instance runs exclusively (upgrade)
- “Standard” = similar to EHPi or SAPup
- “Advanced” = intention to minimize the downtime
In SAP’s Software Update Manager (SUM) the advanced pre-configuration mode is used for extensive downtime optimization. Using the shadow instance for the main import (nZDM for SUM) is the main option in the “advanced” pre-configuration mode.
For further details about the shadow instance in SUM, see SCN blog http://scn.sap.com/community/it-management/alm/software-logistics/blog/2014/04/02/sum-introduction-to-shadow-system
Considering all downtime minimization capabilities which are offered in the advanced pre-configuration mode of SUM you can select the following capabilities:
- Intensive use of parallelization for Batch processing, DDL processes and R3TRANS
- SGEN in the shadow instance
- Customer transports import
With this feature, you consider the customer transports of your target release to the SUM procedure. The business downtime is reduced because repository related customer transports are imported during the uptime phase. In combination with nZDM the customer transports import can be used to execute z-table conversions in uptime.
The customer transports import is fully integrated with SPAU / SPDD, Transport Management System (TMS) and Change Request Management (ChARM). For further details please read SCN Blog: Import of customer transports for upgrades and customer releases available in SUM
When you use customer transports in SUM the screen to maintain SPAU and SPDD transport is not needed anymore. The procedure knows which of the transports are SPAU transports or SPDD transports. This also means that you can use as many transports for SPAU and SPDD as you want.
- nZDM for SUM
You can select which capabilities you need, the features are optional.
How to activate the offered downtime minimization capabilities?
See Settings to activate downtime minimization capabilities in SUM
nZDM for SUM can be used to apply Support Packages, applying enhancement packages (EHP) or to upgrade an ABAP-based SAP Business Suite or SAP NetWeaver system.
Which improvement can you expect when nZDM is used in SUM?
The downtime minimization benefit depends on the level of your update. An update of ECC 6.0 EHP1 to EHP7 is much more benefitial than an SPS implementation of SP9 to SPS10. Therefore the main focus of customer activities is on EHP implementations or upgrades.
Based on the already made experiences on customer site nZDM for SUM enables a minimum downtime of around 2:30h – 3:00h when implementing an EHP or running an upgrade.
The overall technical downtime (= minimum downtime + post-processing) is reduced significantly although the post-processing phase is extended due to the nZDM feature to around 0:45h – 1:00h
So in total we have an overall technical downtime of 3:15h – 4:00h
Just compare it with your key figures of your last EHP update.
Please find below 4 examples from real customer projects and a comparison between the overall technical downtime with SUM without nZDM or EHPi and SUM with nZDM:
As you see all the data we got are roughly 2 years old. At this time we offered nZDM of SUM on request and so we got a very good overview about working in different system environments and sizes.
After available on request we get feedback only based on the SUM upgrade info that are sent to us
Based on our experiences the technical downtime reduction seems to be stable and independent from database or data size as nZDM for SUM was tested by big customers as well.
The following measure shows an internal comparison between SUM for SAP ERP 6.0 with and without nZDM for SUM.
Also in this example the technical downtime is reduced > 55%!
Additionally you see that the overall project time is increased to approximately 20-30%. This additional time is used during the phases which are running in uptime. This has to be considered in the cutover plan.
Functionality
How does nZDM for SUM work?
The nZDM capability of SUM uses the shadow instance for the main import.
Quick facts about nZDM for SUM:
- Introduces the “Record & Replay” technique for business transactions based on database trigger technology
- Minimizes manual effort: all steps run automatically in background
- Minimal additional hardware requirements due to shadow-technique, only additional DB space needed (80 – 350 GB)
- The additional DB space is needed for the transferred tables to the shadow (30 – 150 GB) and for logging tables (50 – 200 GB).
- When you want to use nZDM in combination with customer transport requests import you have to consider z-tables for table conversions in addition
- Available for all ABAP-based Business Suite products
Record & Replay means that the database changes during uptime of the maintenance process create triggers, which records the changes. The recordings are only needed for tables which are used for the update / upgrade in the shadow. With the help of the recording the shadow tables will be updated after the upgrade/update phases in the uptime. The majority of the recording updates are run in the uptime. Only the delta (last 10%) has to run just before the switch in the downtime.
What are the consequences for CPU and memory consumption?
You maintain the amount of process which are used by the Record & Replay technique (CRR). The default value for the amount of processes is three. This is the value which is recommended from our side based on the experiences we made with our customers. I am not aware of an impact on pereformance for the enduser.
nZDM can be used for common used databases with the following minimal versions:
- IBM DB2 LUW 9.7 FP4
- Oracle 10g
- MS SQL 2005
- IBM DB2 for z/OS 10.1
- SAP Max DB 7.9
- SAP HANA 1.00.52
- SAP Adaptive Server Enterprise (ASE)
- DB2 for IBMi 7.1
SUM allows using the nZDM capabilities for the following standard software maintenance activities:
- Applying a Support Package on any ABAP-System with a target release based on SAP NetWeaver 7.0 EHP 2, SP8 or higher
- Installing an EHP on top of any ABAP system with a target release based on SAP NetWeaver 7.0 EHP 2, SP8 or higher
- Upgrading your system to any ABAP system with a target release based on SAP NetWeaver 7.0 EHP 3 or higher
For further operational details please see SAP Note https://css.wdf.sap.corp/sap/support/notes/1678564.
nZDM is described in the official user guide of SUM as well.
When you are looking for additional support please also check the frequent asked question document:
http://scn.sap.com/docs/DOC-40355
What is new with SUM 1.0 SP14 – SP16?
In the past you probably got issues with nZDM that for example the replication rate showed 95% but the SUM didn’t accept this.Or you got issues due to very big tables to be copied.
With the latest SUM SPs we optimized nZDM:
- excluding big tables (sapup parameter)
- copying improvements to optimize the runtime for “copy to shadow” phase
- optimizing of CRR handlingfor transaction CRR_CONTROL
- optimizing of nZDM runtime
When you need to copy big tables (SAPor custom tables) to the shadow instance you should be aware of a parameter explained in the SAP note 1678564.
The default exclusion value is 50 GB.
If you want to mainly run conversions you probably didn’t need this feature. You disable the the maximum table size feature using the SAPuppar parameter set_max_tab_size = OFF.
In older SUM versions it sometimes happened that the refresh of the CRR_CONTROL was very time consuming due to high load of CR entries. The threshold was not accurate. This has been changed with the new SUM 1.0 SP. For this a “Cut-Off” Limit (“Maximum Logtable Row Count”) is introduced. This threshold ensures that the calculation is not such time consuming.
With the next SUM SP17 we plan to improve the start / stop procedure to avoid conflicts.
Thanks for the details Martin. I see that nZDM only support ABAP , any plans for Java and dual stack systems in the future ? When you say trigger based technology , is it using the same technology that ICNV uses ?
Thanks again for these details
Prince Kudilil
Hi Prince,
thanks for your feedback.
regarding dual stack it is planned to have a nZDM support available this year. Regarding SPs and Patches we already have downtime minimization tools for PI dual stack available. (see http://scn.sap.com/docs/DOC-33260).
Regarding Java we are currently analyzing how to minimize. The proof of concepts are planned until Q3 this year.
Your question regarding database trigger for nZDM and ICNV. The nZDM database trigger is not used for structure changes. It is used to update the tables in the shadow based on direct database triggers when entries are deleted/changed/created. So as I understand it is not the same technnology but the communication to some databases have to be enhanced even.
Hi Martin,
while we used nZDM for EHP deployment on our ABAP platform, can certainly see the benefits but "Post Processing" phase taking longer then anticipated. So looking for your thoughts on this and share details on the next steps.
Regards
Sekhar
Hi Sekhar,
may I ask how long it took?
My experience is in general around 1 hour.
And as the post-processing phase is not released for customer activities officially. It has to be considered as a part of the technical downtime.
We had discussion with other customers due to this.
We plan to improve this part of the SUM procedure and the improvement is palnned for the next SUM SP.
It is intended to move the cleaning activities to the very end and the customer can run their activities in parallel. Before the cleaning step a dialog is planned to inform that the system is ready for use. This will reduce the postprocessing significantly.
When this is out as planned this year I will update this article and inform about in more detail.
Best regards,
Martin
Hi Martin,
follow-up on 'Post-processing' enhancements, I presume its improved by now and however want to check if you have any documentation that talks about this enhancement with time saving details.
Regards
Sekhar
Hi Martin,
follow-up on 'Post-processing' enhancements, I presume its improved by now and however want to check if you have any documentation that talks about this enhancement with time saving details.
Regards
Sekhar
Hello Sekhar,
many thanks for this questions. Yes, you are right, this is imporved with the curent SUM 1.0 SP12.
In the post-processing you have now a screen (MAIN_POSTPROC/REQUPFINAL) which informs you that the system is now available for all users without any restrictions.
This is also mentioned in the docu of the SUM.
In the past I saw postprocessing of 45min to 1,5h in average. But since nZDM is GA I see the upgrade data of customer projects only from time to time. SO I don't have a good feeling about the savings. It would be helpful when you share your expereicnes with me. Besides of this we are still improving the nZDM in SUM, so post-processing will be reduced also because of other smaller improvements.
And finally you have now a clearly defined start point for your activities.
Best regards,
Martin
Hi
good news, nZDM available for Java:
2274085 - Central SAP Note for near-Zero Downtime Maintenance 1.0 SP17 Java
Regards
Antonio
Hi Martin,
Thanks for providing light on the benefits of nZDM.
My questions are basically in terms of setting it up.At present SAP note 1678564 is under maintainence, so couldn't go through it.
What are the pre requisites in terms of CPU and memory ?
Last time when I attended workshop on nZDM there was concept of having additional server similar to productionn server and doing upgrade there and then switching it over during downtime.After advent of SUM 1.0 sp 08, how we can set nZDM functionilty after choosing advanced configuration for upgrade.
Thanks,
Ashraf
Hi Ashraf,
I have to apologize. The note is currently in update mode. A DB restriction will be taken out but we are waiting for the patch on the Marketplace. However I have addressed it to the colleagues to release the note as soon as possible again.
The downtime capabilities of SUM (incl. nZDM) are not based on a clone. Probably you had a session regarding nZDT! This is a service which is offered mainly in combination with a DB migration or unicode conversion. nZDM uses a shadow instance which only need some GB of the database and some workprocesses.
How to activate nZDM?
In the advanced mode in SUM, a checkbox enables to activate nZDM in the configuration. Main activity of nZDM is to synch the shadow instance and the productive instance for tables which are needed for the update. For this you have to maintain the amount of processes on the same screen.
When you need further info just tell me
Best regards,
Martin
Hi Martin,
Really appreciate your reply. You are correct , got confused with nZDM and NZDT.Thought this as another name for near zero downtime.
Actually I have recently performed SCM upgrade choosing standard configuration in SUM 1.0 sp08.May be next time, I will choose advanced configuration to have complete picture.
Thanks,
Ashraf
Hi Martin,
Thank you for the useful nDZM information, at the moment it is hard to find documentation on such a new topic.
I am currently using this new nZDM feature with SUM8 during an EHP7 upgrade on an ERP6.0 system and I would like to find out more information about fast data copy process, what is it's purpose. I am now at FDCS segmentation stage and this is running (logs are being updated and processes running) for more than one day (no memory errors or short dumps until now). Is this normal and is there a method in order to speed up the process? I am using parallel background processing and I also changed the parameters on the shadow as on note 1387739.
Thanks,
Dalmy
Hi Manu,
thanks for your feedback. I am interested in the Update data you get for your EHP7 update. So it would be great when I get the UPGANA. I will not publish it directly, it is just for our database.
Regarding your issue:
Fast Data Copy (partly) replaced the original copy to shadow and is based on SQL statement. However, it is not normal and it looks like an issue on your side.
Please create a message in the CSS, component: BC-UPG-DTM-TLA
Best regards,
Martin
Hello Martin.
I'm very interested on the nZDM procedure included in the SUM for an upgrade to ERP6 EHP7.
Concerning the SAP note 1678564 I have a question regarding point 4.1:
Is there a predefined set of tables, wich has to be included or excluded into the recording procedure? Where can I find this information ?
Thanks, Peter.
Hi Peter,
we discussed it yesterday but I would like to answer it also on this platform because this is a valid question and probably also of interest for other readers.
The SUM identifies the tables which are needed on his own. So no further activities are needed ffrom your side.
However in the past we often had problems with the tables STXH, STXL. They can be very big and therefore the copy to shadow became very long in the past. A valid workaround was to take these tables out. We should have fixed this issue with the two tables so far.
Best regards,
Martin
We had a conference call with SAP personnel to discuss maintenance downtime minimization and the slide deck from that meeting mentions that the start release of the system must be EHP7 SP4. I don't see that restriction mentioned above or in the OSS note that is cited above. Does that apply to this nZDM capability? We are currenly on ECC 6.0 EHP5 with Basis 7.02 SPS 11.
Hello George,
it is great that you heard already about Zero Downtime maintenance (ZDM) which is planned to be available on project basis at the end of this year (this is the current status and can be changed at anytime).
But this article explains nZDM (near Zero Downtime Maintenance) and this is a feature in the SUM advanced mode.
The nZDM feature in SUM advanced mode is general available since last year and successfully used by various customers to reduce the technical downtime.
The Zero Downtime Maintenance will be a new technology and procedure. For ZDM we currently eplain interested customers the constraints and the project setup for the first pre-release of ZDM. And I guess this is the info you got. When you need further info about nZDM or ZDM please let me know
Best regards,
Martin
Hello Martin,
Thanks for your sharing.
Since the nZDM contains some conversion activities, is there any relationship between ICNV and nZDM?
Best Regards,
Tong Ning
Hello Tong,
please find the answer on top of the comments (23rd April, 2013).
It is not a direct relationship. The incremental table conversion follows rules and not all conversions are possible. However with help of the nZDM functionality it is possible to execute all conversions in the shadow (uptime). ICNV is usedalso in SUm standard mode,
But regarding the conversion abilities of nZDM I am planning to publish a blog. And the relationship to should be mentioned.
Best regards,
Martin
Hello Martin,
Thanks a lot for this important information. I am waiting for your blog about this relationship between ICNV and nZDM. Now as far as I know, the ICNV function is limited (only for some candidate tables), the nZDM can almost do all conversions?
And also if I select both nZDM and ICNV option in the upgrade process, then they can cooperation with each other? (for example, ICNV convert the candidates and nZDM convert the all others)
Thanks & Best Regards,
Tong Ning
Hello Tong,
just to precise:
- there is no releationship between ICNV and nZDM feature in advanced mode.
- ICNV is not used when nZDM is active
Best regards,
Martin
Hello Martin,
If using nZDM is ICNV switched off completely ? In this case in which step is the table conversion being handled ? I presume downtime only.
Thank you!
the table conversion is managed in the shadow. When you use customer transports and nZDM is active you will also convert most of the custom tables as well
Best regards,
Martin
Thank you!
Hi Martin,
We have used nZDM for EHP7 deployment on ECC system and it went all fine. But we noticed there a table called 'CRRTASKHIST' remain in the system with significant size. So wondering why do we require it post EHP7 deployment and why this is not getting cleared/dropped as part of post-processing clean-up tasks.
Please advise if we can drop/truncate this table permanently or is there any use of keeping it for future Deployments?
Regards
Sekhar
Hello Martin,
Thankyou so much for sharing this blog..
Regards
S.BASIS
Hi
I have some questions about NZDM option in SUM
1) In classic shadow instance only repository is copied into shadow tables. They do not contain any data. As I understand Record & Replay technique requires full copy of database table into shadow tables . Is it correct?
2) Customer transport. Is this option suitable for workbench transports only ( SPAU etc) or for customizing requests also?
3) General SUM question - is it possible in easy way to run shadow system on dedicated application server?
Thank you
Regards
Vladimir Kogan
Hello Vladimir,
regarding the extended shadow for nZDM you copy technical tables incl. content. So your assumption is correct.
Regarding customer transports import, please read the SCN article
http://scn.sap.com/community/it-management/alm/software-logistics/blog/2013/07/04/considering-of-customer-transports-to-minimize-the-downtime-for-sap-business-suite-updates-and-upgrades
It explains the functionality in more detail. And yes, all transport objects are possible, customizing as well.
Regarding the shadow system (instance), please also check the SCN blog http://scn.sap.com/community/it-management/alm/software-logistics/blog/2014/04/02/sum-introduction-to-shadow-system
Best regards,
Martin
Thank you a lot !
As i have also understood , there are some activities, that in classic scenario runs during system downtime and are application depended and cannot run an classic shadow instance. Where can I find a details about this activities, that now switched to shadow in NZDM scenario?
Regards
Vladimir
Hi Vladimir,
I have not published such details. The activities for table adjustments, some dedicated After Import Methods (AIMs) and major parts of the main import phase are moved to the extended shadow operation.
This means:
XPRAs upg
tabim upg
parconv upg
xncv-phases
More details you might see in your UPGANA.
Best regards,
Martin
Hi Martin.
According to OSS note it is possible to upgrade from SAP R/3 4.6C to SAP ERP 6.0 Enhancement Package 6.
I guess that is a difference between data dictionary and business process between so difference releases.
what happens if the same business process in 46C and in ECC 6 EHP 6 updates different set of tables and fields?
Thank you
Regards
Vladimir Kogan
Hi Vladimir,
sorry for late reply, the first week after my vacation was full of activities and so I am in delay.
I hope I got you right.
The SUM uses an update procedure which is independent from business process and only considers tables. When some table conversions (not manageabke by DB) are needed the table is moved to the shadow and the nZDM functionality has to manage it.
For example, you run a transaction Z which uses a table "nice-functionality" and this table "nice-functionality" has been changed in the new target release, e.g. change data type from CHAR to NUMC. The table "nice functionality" will be copied with the new strructure in the shadow and conversion is running in ABAP based on the record and replication functionality.In downtime the shadow table replaces the old "nice functionality" table.
Best regards,
Martin
Hi Martin
I have performed test NZDT upgrade.
DB size is 320GB
Update from ECC EHP 5 to EHP 7
Application server 16 CPU 32 GB RAM
Database server 16 CPU 32 GB RAM
Total downtime was 6 hours.
Longest phase were:
MAIN_NEWBAS/TABIM_UPG about 2 hours
MAIN_NEWBAS/XPRAS_AIMMRG about 3 hours
The customer expectation were not more 3 hours.
It not looks really as NZ…
Regards
Vladimir
Hello Vladimir,
thanks for your feedback. But at first I would like to clarify some words and names upfront. NZDT is a service offered by SAP to minimize the downtime e.g. for unicode conversions or DB migrations in parallel to zpdates. It is based on a clone. This is not the nZDM feature in SUM (advanced mode). According to the experiences we made with the customers so far we are able to reduce the overall technical downtime to app. 50%. The technical downtime of SUM wis considered with app. 3-4 hours. So I don't see the expected technical downtime of 2h mentioned in the document. But as i see you have a problem even in the minimum downtime part of the SUM and this is not expected indeed.
Can you please send me the upgana file to my email adress? I would like to discuss this in the development team.
Best regards,
Martin
Hi Martin,
I still see the roadmap TABIM_UPG is running in execution roadmap even though the nZDM is selected.
So the main import should include what activities in uptime within nZDM.
Regards,
Ning
Hi Martin,
We been using nzdt for our system since last 3 years. We originally used it for EHP5 to EHP6 upgrade & it was fanstastic in terms of downtime.
We are using it again for our EHP7 upgrade for our ECC system. Since last year We can see that now we can decide how many processes we want to use for SGEN. Can you please confirm what is the maximum amount of SGEN Processes which we can use & is it dependent on any of values like ABAP Processes (Uptime) , SQL Processes, R3trans Proceses, R3load Processes, Parallel Phase?
So for example can we use 8-10 SGEN processes what do i need to do ?
Regards,
Ajay Sehgal