SLD CR Content Repair

You know the drill: every so often you need to download and import the latest CIM model and CR content into your SLD to keep things up-to-date. You meticulously follow the directions from Note 669669 (Update of SAP System Component Repository in SLD) every time, and you never have problems…

… until the day you do.

“Dingoes ate my SLD!” or, “How to recover when the unthinkable* happens”

(* “Unthinkable,” in this context, means “I downloaded and tried to import something that was apparently corrupt!”)

Yeah, this just happened to me with the import of CR Delta 11.0, aka “SP13 for SAP CR Content up to 2014” (this is the one that changes you from 10.x to 11.0 and preps you for the “Content up to 2015” deltas). At the end of the import I saw “3 errors occurred during import.” Catastrophe! All three errors were identical, something to do with Business Planning and Consolidation (a component we don’t even use) and a violation of how many of one type of instance can be associated with another type of instance, etc etc etc. I was pretty sure it was something bad in the delivered content delta zip file, and who knows, it might just have been fixed by the next delta I was about to apply, but you know the protocol: don’t apply any more imports until you know your SLD CR content is consistent.

But is it really corrupt?

Step one: find out if we really have a problem. So, you refer to your handy-dandy Note 1939864 (Check if SAP CR Content is corrupt in LMDB or SLD). You check to see if you have a high-enough Support Package in your Solution Manager system (sp11 or higher for SolMan 7.1), and if not, then apply the Note.

Once certain the Note or Support Package is applied, go to transaction SA38 and run program RLMDB_CR_CONTENT_HASHER. It’s time to hash some stuff up. Anyway, the program is pretty straightforward: check the radio button for Use remote SLD, enter the RFC Destination for the SLD in Source, leave the Namespace field alone (unless your suspected corruption is in a non-default namespace), and copy/paste the Expected Hash from the table in Note 1939864 for the CR content version you are checking (as reported by your SLD). In the example below, the hash represents content version 11.0.

The program kicks off a background job called RLMDB_CR_CONTENT_HASH. It will take a while to run (in my case, checking a remote standalone SLD, it took a little more than an hour). When the job is finished, check the job log. If you see something like:

Hash 864497b8-171c-7981-4c7f-16ffc5fccb90 does not match expected hash 123cb9c8-5886-4cf3-f01f-7ab9239981ec

well, then you just received the bad news. Your SLD content is corrupt, and you need to repair it.

Yes, it’s really corrupt. Now what?

All is not lost! This is not a Robert Redford movie! But, hopefully the support package level on your SLD meets at least the minimums specified in Note 1093168 (Repair of SLD CR content). If not, it’s not the end of the world, but some parts of new content may not be taken into account. However, the basic repair should still work.

Your next step is to create a reference namespace in your SLD that contains only the SAP-delivered content (not your landscape data), and then compare the reference namespace with your active namespace. You can do this online, but you do need to take into account a few restrictions mentioned in the Note, namely don’t try to import any CR content updates while executing the repair (duh, the content’s corrupt, and we know better than to try to import a new delta over a corrupt one, right?).

This is a good time to ensure your SLD database has plenty of free space. About 50% free should do it.

Create a reference namespace

Logon to your SLD (http(s)://<host>:<port>/sld) and select Administration -> Namespaces. Click on Add and give your new namespace a name (e.g. “sld/reference”).

Select the reference namespace as shown above, and select Administration -> Import. On the next page, make sure that the Target Namespace is the reference namespace and not your active one, then select the radio button for From Server and click Import:

You’ll notice that you’ll be able to import the “default” model for the release of SLD you have (in my example, this is on a NetWeaver 7.4 SR2 SLD). On the next screen, click Continue Import. The model import should go fairly quickly. When it’s done, if you return to Administration -> Details -> Data you should see something like this:

Now go back to Administration -> Import like before, again ensuring your target namespace is still reference, and again import from server:

In my case, this imported content version 9.0 and took just under half an hour.

Update reference to same model and content as active namespace

Now that your reference namespace is at model version 1.6.38 and content version 9.0, it’s time to get it up to model 1.6.46 and content 11.0 (or whatever is currently in your corrupt active namespace). This procedure is just the same as any SLD content update. You’ve already downloaded the new model update and the appropriate latest content update (because, well, you used them on your active namespace, didn’t you?), so all you need is any required intermediate content updates.

Check Note 669669 to find out what’s required to get from the current content version to the updated content version. In my case, to get from 9.0 to 11.0, I’ll need to download the intermediate update that gets us from 9.x to 10.0, which we find under Support Packages and Patches -> SAP Technology Components -> SAP CR CONTENT -> SAP CR CONTENT UP TO 2013 -> CRDelta91615_0-20010858.zip.

Back to Administration -> Import and this time check the radio button for From Browser and select the model update (the file that starts with cimsap…).

Repeat the import with the intermediate CR Delta to get from 9.x to 10.0 (or as appropriate for your situation), and the “final” delta to go from 10.x to 11.0 (or, again, as appropriate for you). This, by the way, is where you had errors before, right? So if this update fails again, then it’s a pretty safe bet there’s something wrong with the file you downloaded, and it could be time to submit a Customer Incident to SAP to let them know. If it doesn’t fail (and we hope it doesn’t), then the error was something else (cosmic rays, perhaps?), and we’ll continue with the repair process.

When you’re finished, you’ll have details something like this:

Don’t worry that the number of Instances and Associations aren’t the same between the active and reference namespaces. Remember, your active namespace has all your landscape data in it, whereas the reference namespace does not.

Start the repair!

Ok, that’s a lot of time spent creating a reference namespace that doesn’t do much. Now it’s time to make it useful. Be sure to get a backup of your active namespace before continuing (select the active namespace, then Administration -> Export -> Back Up -> All Instances).

Select Administration -> Maintenance -> Repair Software Catalog (or, if you’re on an older SLD, such as that included with SolMan 7.1, you might need to enter it directly as a URL like http(s)://<host>:<port>/sld/admin/CRRepair.jsp). Make sure your active namespace is in the Productive namespace field and your reference namespace in, well, obviously, the Reference namespace field, then click Start Repair.

The repair process will touch almost 1.6 million objects, so your first thought will be “wow, that’s going to take a long time.” However, it goes faster than you’d expect (faster, apparently, than an import of a fraction of the number of objects); mine completed in a little over five minutes. Afterwards, you’ll get some interesting statistics:

Check the result

After the repair is finished, go back to your Solution Manager and re-run RLMDB_CR_CONTENT_HASHER as you did at the start of this process, again using the expected hash for your content version from Note 1939864. This time you should get a match.

Once finished, you no longer need the reference namespace, so you can delete it if you wish. However, if space in your database is not an issue, you may choose to keep it… just in case.

Further reading

Software Catalog Data aka CR CONTENT in SLD and LMDB

System Landscape Directory (SLD) | SCN

Performing Initial Data Import – Configuring, Working with and Administering System Landscape Directory – SAP Library

Note 669669: Update of SAP System Component Repository in SLD

Note 1939864: Check if SAP CR Content is corrupt in LMDB or SLD

Note 1093168: Repair of SLD CR content

Note 1891566: Repair of LMDB CR Content

Note 1751175: SAP CR content anomalies in SLD or LMDB

Postscript

It is likely, after you are done, that you will need to repair the content in your LMDB as well. Wait until all synchronizations between SLD and LMDB are complete (you can check in LMDB Administration, or in SM37 in the logs for the most recent job SAP_LMDB_LDB_*. Then check the LMDB by once again running RLMDB_CR_CONTENT_HASHER, but this time choose the option Use local LMDB. The runtime will likely be about half what it was to check the SLD.

If you see the message in the job log afterwards about mismatched hash values, then your next step is to run RLMDB_SAP_CR_REPAIR_LMDB in background, as described in Note 1891566. As before, ensure there are no CR content updates made to the SLD during the repair.