Email Archiving for ERMS and Interaction Center in SAP CRM
** Update February 2017 **
SAP has finally rolled out support for email archiving for SAP Connect and ERMS emails. Please see SAP Note 2197465 for full details!
————————————————————————————————————————————-
You can never have too much of a good thing! Well, except for emails. You can definitely have too much email in your inbox. As a general rule, I try to keep my inbox to just one page so that all my emails are visible without having to scroll. But I’ve seen colleagues with hundreds of unread emails!. And that’s just for a single user. A company on the other hand might receive that many emails in a single day — every day, every week, every year. And all of those emails get stored in the CRM system. For companies using SAP CRM to handle emails with ERMS and/or the Interaction Center, that could amount to hundreds of thousands, if not millions, of emails. Obviously this can raise concerns about storage size and system performance. So what is a company to do?
The simplest solution of course would be to do use archiving. Unfortunately, things aren’t always as simple as we would like them to be. For example, while we talk about “emails” in SAP CRM, what we are really dealing with in the system are not actual emails but rather CRM workflow items, SAP business objects, and an assortment of database tables. Technically, when an email passes through the SAPconnect interface into the SAP CRM system, the email is converted into a CRM Workflow item. Later, when that workflow item is selected by an agent from the Agent Inbox in the Interaction Center, the workflow item is completed and converted into an SAP business object.
To further complicate things, “emails” and the corresponding workflow items SAP business objects, attachments, and replies aren’t stored in one place. Rather, various items are stored in various different tables such as: SOFM, SOOD, SOC3, SOFFCONT1, SOOS, SOST, SOES, SOFFLOIO, SOFFPHIO, SOFFPHFK, and so on!
There are of course options for going about archiving workflow items in CRM, for example. But to be honest, the workflow items aren’t really what most customers need to be worried about in terms of storage size and performance. Similarly, there are also options for archiving text documents (type RAW & SCR) using the report RSSOAPUT. But text documents aren’t of particular concern either. What companies should be most concerned about of course are email attachments — which can be quite large.
Email attachments are stored using SAP Knowledge Provider (KPRO). The good news is that customer who concerned about excessive use of database storage for attachments have the option to use an external content management server specifically for email attachments! In the standard settings, the KPRO stores the SAPoffice data in the database table SOFFCONT1. You can also select an external content server or an archive system as somewhere to store the documents. The storage location is determined using the content repository ‘SOFFDB’. You can check the storage location for the SAPoffice data in your system using Transaction ‘OAC0’. See SAP note 904711 for full details.
You may also want to check out this thread on SCN for additional discussion.
Follow me on my blog at http://scn.sap.com/people/john.burton/blog
Hi John
Thanks for this post. This makes me thinking that SAP at least acknowledges that there is a problem here.
As you've pointed out the email implementation is complex and consists of different disconnected data objects: SAPOffice documents, ERMS workflow and ERMS data (crmd_erms* tables) and there is no holistic approach to remove/ archive the data from the system. There's even no BAPI which would allow to clean the crmd_erms* tables!
We implemented ERMS push scenario 2 years ago and we're stuck with 10k of inbound emails received on a daily basis trying to come up with some custom approach, observing the tables exploding and seeing the entire system slowing down (think about searches in agent inbox/ email workbench).
Moving the email attachments to an external storage releases significant amount of space but what with the rest of the data? Also, I would not underestimate the workflows. The ERMS workflow containers are big as they contain the entire body of the email and as the amount of emails grows it's getting difficult to handle the workflow tables.
Is SAP planning to come up here with something?
Thanks. Mariusz
PS. Looking from a perspective of a global implementation the ERMS push solution itself seems to be half baked only. We've had (have) lots of issues in synching the status of the ERMS workflows with the status of the ERMS data and the CMS status (simple scenario: what to do if your push to CMS fails?). I guess I could write an essay on this...
Hello Mariusz,
Thank you for your candid comments, and for your honest assessment. I certainly understand your frustration. The challenge with email archiving is that emails are used by CRM in general and not just specific to ERMS. In order to make progress with email archiving for ERMS we will need to coordinate and engage with the NetWeaver Basis teams and some other groups. Probably the best way forward would be to utilize ASUG and the SAP Customer Connection program to drive this requirement and recruit support from other customers. Unfortunately we have just wrapped up the roll-in phase for the current CRM 2015 focus group, so we will need to wait for the next round to submit a new enhancement request for email archiving.
Warm regards,
John
Hello John,
I would like to strongly support both points that Mariusz highlighted. The first is obvious, we also have several customers using both pull and push method in ERMS, some of them have more than 3mio customers, using campaigns and therefor generating many inbound and outbound emails.
The second problem is becoming more and more painful - customers using pull method would like to keep better control of their SLA fulfillment (and cut or keep costs, of course) and therefore considering transition to push method. For such a transition one could easily create a business case.
However, as Mariusz wrote, the standard here is not ready to support real-life situations. I can add another situation - all of our ERMS customers would like to combine both pull and push method, yet there is missing status synchronization between ERMS and CMS.
We have solved this with a small enhancement, utilizing the ICI IciItem_End method on Inbox Reserve button - this should be universal, fully in accordance with ICI specification, yet again, it does not work with SAP Contact Center (formerly BCM) for forwarded items - because there is no standard ERMS process for this (official OSS incident response)... So we are getting into a deadlock...
You can check for example Shamsul's discussion to see that we are not the only one: http://scn.sap.com/thread/3569796 .
Please, is there any specific procedure how to support SAP's engagement by customers and partners? Except signing in and posting idea in the SAP Influence collaboration room?
Thanks for the blog, it has saved me a lot of time of putting information together.
Regards,
Dawood.
Hi Dawood
Thanks for your post.
It's funny to see you mentioning exactly the same points we've been struggling with. CMS forward of the action items was one of them. Supported by the ICI specification and supported by our CMS (Genesys) but not supported by CRM. Easy to implement. In one of the tickets to SAP I provided slides outlining exactly the changes which need to be done to enable it.
It's also interesting what you write about users operating in pull/ push mode. We have situation in which some of the teams need to work in push whereas the others in pull. The push teams are not meant to be able to pull the emails from the inbox to avoid cherry picking but they should see emails assigned to them (ERMS workflows reserved by them). These are the real use cases not supported by SAP.
The disconnected status between ERMS workflow and ERMS tables is obviously a nightmare. We've implemented exits for the ERMS workflow reacting on status change to update the ERMS data. Sometimes I wonder if all this had to be really that complex?
Also the basic functions are not really there. Our users have been crying to be able to add multiple attachments to an email in one go. Is it really that difficult to implement it?
I could continue like this. Email is still the key channel and must be nailed down by a CRM system. Has it been nailed down by SAP CRM? I have doubts.
Thanks. Mariusz
Hi Mariusz,
exactly - you are confirming that our requests are not customer specific but common - hope SAP will notice this 😉 .
By the way, did you check OSS note 2070206 - Multi fileUpload ? This should be way to enable multiple upload. We tried to change the rendering in runtime in debug mode and it seems that it works.
You will have to extend the UI component and switch on the new MULTIPLE attribute...
Regards,
Dawood.
Thanks Dawood for the reference to the note! Surely will check it out.
Thanks John. I appreciate the complexity and I understand it might take some time until we have something available in the standard supporting the archiving E2E. In the meantime it would be useful to get some recommendation on how a custom solution could be implemented. A good, detailed consulting note covering the subject in a holistic way could do the trick.
Thanks!
Good news! It looks like SAP is going to address this issue via the SAP Customer Connection program. Check out this enhancement request from the CRM 2015 focus topic. Yet another reason why it pays to be part of the Customer Connection program to make your voice heard and help influence which features/functions SAP delivers next!
https://influence.sap.com/D6774
Hi John, Dawood, Mariusz
I happened upon this thread as the planned rollout for the improvement request I proposed is pushed back to 2016, and we need to implement a solution this year for deletion of certain emails, due to potential legal ramifications regarding data submitted by children.
We have been manually removing emails ad hoc - eg. when customers submit credit card details on email, even though we ask them not to... this is very time consuming, following the related programs around RSBCS_REORG. There is a reasonably helpful KBA on this: SAP Note 1641830.
Regarding the synchronisation between CRM and CMS (we are also using SAP Contact Center) for cases of pulled email, we are also making our own solution - helpful hints from Dawood. This would have been a prime candidate for submission to the CRM 2015 influence program, but I thought it may have been out of scope. However, I do plan to submit it in this round - kickoff next week: CRM 2016
Hi John,
I am currently working with a client, that is ingesting emails into CRM using the process you describe. I believe the email attachments are going into SAP Office documents (and are archived into KPRO).
Is there a way to Archivelink these attachments, using OAC2/3 config. Maybe using a different Workflow ? Preferably this can negate having to do developments, and be a Standard functionality
Hi Richard,
Did you consider information from KBA 2197465?
Best Regards,
Sigrid
Hi Sigrid,
I'm now in processing of this KBA 2197465.
My question would be, how to do the step 4?:
"4. Step: Archiving and deletion of the ERMS based work item.
Archiving and deletion of ERMS based work items can be achieved via transaction SWW_SARA"
What archiving object should be selected for ERMS based work items? On "WORKITEM" there is no according records.
Thanks.
Kind Regards,
Sergey