Skip to Content
Technical Articles

Archiving – How it is designed, how does it work

Hello all,

In this blog I want to explain how we have designed Archiving in C4C (available from 2011, see this Product Information), how it works (not immediately after you active it in the Business Configuration 🙂 ), and why we have done it this way (to prevent the fear of data loss on your side).
Finally I will give you a foresight how it will get more simply in future release.

Let’s start with the activation:

As it is already stated in the Product Information you first need to choose in the Business Configuration for which of the Business Objects you want to switch Archiving on.
We will come later to these retention periods.

Now this has the following consequence:

  • Early morning each day 3 jobs are triggered for Archiving.
  • Each job is working on all activated Business Objects consecutively
  • The first job Move2Archive looks for potential candidates and move the verified ones to the archive, but let the data still persist in the tenant
  • The second job DeleteLocal checks if the residence period if over and removes the local data
  • The last job DeleteFromArchive finally deletes the data also from the archive

This means if you activate Archiving in the afternoon you have to wait until the next morning to see a potential result.

But what are the results?

Now we need to discuss the Retention Periods (they are pretty well explained in the blog mentioned above, but let me repeat):

  • Retention Period Before Archiving: This defines the “age” of the record. How long it was not changed anymore (for the techies: the LastChangeDate 🙂 ), giving a good estimate that this record is not used any longer.
    Nevertheless, this is no guarantee that this record can be archived. Think of a long running warranty. It will not be changed over years, but still be in effect.
  • Residence Period: During this time the data exists in both worlds, in the local database as well as in the archive. This provides confidence to you that the data can be restored easily (no fear of data loss).
    It starts from the point in time the data was moved to the archive.
  • Retention Period Before Deletion: Normally defined by legal requirements to keep the data for verification reasons.
    This period starts when the data was deleted locally.

So, the Move2Archive job collects all “old” records. These candidates are handed over to some business checks individually defined and implemented by each Business Object. In most cases they check at least for status values like Completed, Closed, …. Of course they can be even more sophisticated.
They are explained in more detail in this separate blog.

The business check returns not only the verified instances, but add also potential archiving relevant dependent objects (RDO) like pricing documents for quotes. Theses RDO can not exist by their own. They need the verified instance as a kind of anchor. Therefore we will collect not only the data of the anchor object but also of the RDO and place them together in the archive file. They will also be deleted together.

Finally we will update the Archiving status of the objects to ArchivingInProcess, so you & we know when the Archiving process has been started.

You will be able to search, find, and display these instances in the regular UI only.

Before the job DeleteLocal is executed for a given instance a simple change of the data (which is still available in the UI) is sufficient to reset the Archiving status and deleted the data from the archive. Thus, the object is “back again” (it was never away 😉 )
After the execution of the report the data is deleted from the local database and the Search Archive will be the only place where you can find the archived data.
This means from now on you will find the data only in the Search Archive (Work Center Administrator –> View Archiving).
Some details :

  • Keep in mind that you need any search criteria (even a asterisk) to trigger the search.
  • You may also filter by object type to narrow down your result list.
  • By clicking on the link the Quick View will open.
    From here you can even navigate to the Thing Inspector.

In future this will be the place from where you can trigger a restore. The data would be transformed back to the local database, the Archiving status would be reset and the data in the archive would be deleted.
In the current release we do not have this as the applications are really busy. You need to raise an incident, so we can trigger this manually.
So your result list in the normal UI is no longer cluttered with this old data.
If you feeling save about this you may also reduce the Residence Period so this effects happens earlier.

The DeleteFromArchive will delete the last remains of the record. After this job no restore would be possible. The data is gone.

But, why do we have three retention periods?

Two’s a company, three is a crowd. You need the first one to determine, what is “old” data, whereas the last one is often determined by legal terms.
But what about this Residence Period? It is introduced to give you the possibility to check what are the archived objects? To ensure you, that your data is not gone forever. A kind of smooth introduction to Archiving.

If you feel save about this (and we expect you to feel like that 🙂 ) you can reduce the Residence Period to zero. Thus the data would be sent to the archive and removed at the same day. You will notice the improvements (no cluttering of regular search, …) immediately.

Forecast for release 2108

And that is exactly the way we are planning to move ahead: If we get no big complaints now, this Residence Period will disappear in release 2108 and the jobs Move2Archive and DeleteLocal will be merged to one. It will make Archiving easier to understand and you will get the improvements in your tenant earlier.

Updates

Based on the comments below and other questions, let’s have a look at the impact for you as customers:

  • Visibility of the Archiving status on the UI
    There are several possibilities which may differ for each Business Object:

    • The field is added to the UI and visible
      You’re fine
    • The field is added to the UI marked as “pers. hidden”
      Here you need to switch to the Adaptation Mode to make it visible (maybe limited to a dedicated Business Role )
    • The field is not available
      Please request this field from the resp. application
  • What is the (visible) effect of the retention periods / executed jobs?
    • Move2Archive
      • As long as the field is not on the UI: No visible effect 🙁
    • DeleteLocal
      • The data appears only in the Search Archive
    • DeleteFromArchive
      • The data cannot be found anymore
  • Is there any information about the result of the jobs?
    • You can (re)use the following UI to detect, which object types are deleted:
      • Work Center “Administrator”
      • Work Center View “General Settings”
      • Area “Data Management, View deleted data”

In the resp. OWL you can restrict the search on the objects which you have been activated for Archiving as well as the date “Deleted On”.
Important: The column “Deleted By” must be empty as Archiving is using a technical user which will not be shown.

    • Any explanation why an object has been archived or not would depend on the business checks from each application. There is nothing available for that.
    • As long as you don’t see the status ArchivingInProcess there is no way to determine when such an object will be deleted locally.
    • The “Changed On” date in the Search Archive result list is the only information which tells you how long an object is in the archive. From this date and the Retention Period Before Deletion you can determine when the object will be deleted from the archive.

If there are requirements from your side for Archiving (I am sure, there are some 😉 ) please report them in the Idea Management, so we can collect and prioritize them.

9 Comments
You must be Logged on to comment or reply to a post.
        • Hello Damodar,

          The access to archived documents will be logged (Read Access Log (RAL)).

          With the next release 2105 the archiving is also honoring the Business Partner Deletion Block. This means that a document which refers a Business Partner (e.g. as Sales-To-Party) will not be archived at all and any archived document will not be removed form the archive.

          HTH,
          .    Horst

  • Dear Horst,

    thanks for the information!

    We still have some questions with regards to the archiving:

    1. It talks about an archiving status --> can we see that status anywhere at the object itself or only by knowing it also shows up in the Archiving area?

    2. Currently the blog says that objects that are in the archive can only be recovered by SAP TIcket, but in the future via an 'Action'.  We haven't seen anything, yet.  Is that already within this release? How is the process of recovering objects out of the archive thought of?

    3. - Retention Period: if e.g. it is 2 years and my objects wasn't changed in the last 2 years, then this object should appear in the 'Archiving' section. Currently (and also in the future?) it is there in both worlds, in the normal search as well as in the Archiving section --> Is that the correct process also in the future?

    - Residence Period: Does this period start with the date that the object was sent to the archive first? So e.g my object is older than 2 years (retention) and then it is being put in the archive and left there for e.g. 30 more days (Residence period?) before it is actually deleted from the 'normal' search and only exists in the Archiving area? In the blog you talk about that this residence period will be removed in the future releases. What does this mean then? That the retention run will directly put the objects to the archive without having them in both worlds for a period of time? Is that correcdt?

    - Retention Period before deletion: Objects older than this timeframe (last changed) are going to be deleted completely from the normal search as well as the Archiving area with no possibility to retrieve them back, correct?

    4. Logging We can set the retention/residence and deletion periods. But we do not have any overview at all what happened at which run. (like we had e.g. in the SARA in OnPrem)

    - Will there be a logging functionality so that we at least can see e.g. the amount or list of IDs that were archived that day/run?

    - Will there be a logging functionality for the cases that could NOT be archived with an explanation why? Currently we have many cases that we would expect to be archived, but they are not. Without a logging we have no possibility to check why these are not being archived. (cases, where the last changed date is old enough, status in Completed and no document flow….is there more criteria in the background?)

    Many Thanks for any info on the details.

    Best Regards,

    Kristina

    • Hello Kristina,

      Based on the long text, I see that you took archiving seriously. Thanks for that. 🙂

      1. Archiving Status visible on the UI:
        This depends on each application if the element is visible in the UI or even as personal hidden. You need to check in the UI and - if required - ask the application to add that element.
      2. Recovery of archive objects:
        The framework itself already provides a generic functionality to read the data from the archive, transform it into the database format and inject this data into the resp. tables.
        This functionality must be triggered by the applications. The current design foresees an action which is personal hidden and can/shall be added by the customer to specific Business Role which can be assigned to dedicated users.
        As the realization of this design is with the applications it might be a good idea to push this via the Idea Management by you 😉
      3. Retention Periods
        1. Before Archiving:
          After 2 years your document is a potential candidate for archiving. Now the business checks described here in this blog determine if the candidate can really be archived.
          The archived document is visible on the regular search only
        2. Residence Period
          It starts with putting the document into the archive.
          If we have the Restore functionality available for all archived objects we do not need this period anymore and any document moved to the archive would be deleted from the local database in the same step.
          During this period the document is still visible on the regular search only.
        3. Before Deletion
          If a document is in the archive for more time that this period defines it will be deleted from the archive.
          During this period the document is visible only in the archive search.
          After this period the document can not be found any more.
      4. Logging:
        Currently nothing is planned here. 🙁
        Please describe you requirements in the Idea Management as I can read you have a detailed point-of-view here.

      HTH,
      .    Horst