B2B Monitoring & Archive for CPI
Storing and finding EDI files is an important part of B2B IT operations. For B2B operations people, It is a daily task to find an EDI message, type A from partner B that is sent C days ago, check the contents themselves or pass the file to business for further analysis.
Customers using legacy systems adapted their workflows and tools. Some even use grep on Linux which, I would say, is very capable. Giving fewer features and options than they are used to, will bring more workload to people in operations.
What are your alternatives now and in the future for SAP CPI?
You can use MPL(Message Processing Logs) to store files as attachments. It has a 1 GB logging limitation per day. They are deleted after 30 days by default. For message discovery, you should define clear flow names for integrations and you can store e.g. IDoc document number in SAP_ApplicationID. Your search options are using organized flows and using only one document number.
Edit(2020-09-14): To be able to search for more fields, you can also use Groovy to add CustomHeaderProperties and search them with APIs, there is no user interface yet.
2. Persist Message
Persist Message step doesn’t have the daily limitation but it also eats up space with runtime and MPL logs from the database. And CPI has a 32 GB database by default. If you use this step too much it can strain the database resources.
3. Future: Archiving on the roadmap
There is an archiving feature on the roadmap, and we are eagerly waiting for the feature to shape up.
Update: Archiving feature is released: https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/Cloud/en-US/bc71f883d41a44098e9a525e34d0a318.html
Related blog post: https://blogs.sap.com/2021/10/01/data-archiving-with-sap-cloud-integration/
4. SFTP server
You can use an SFTP server to store the files. You can define a good directory structure.
5. Our Prebuilt Solution (Early Beta)
Now you have another alternative! We have built an EDI archive and search solution for SAP CPI.
It is designed to be performant, flexible, fault-tolerant, and to have little-to-no effect on runtime. The data is stored on your cloud account and under your provision.
The archive storage is as cheap, as scalable, and as durable as AWS or Azure storage services.
We can add more search options based on message metadata and payload.
We are always looking for ways to improve your complete integration workflow. We are ready to discuss your needs and how we can build better solutions together.
If you are interested, you can always contact me.
Thanks for reading!
Thanks for the blog.
How and from where do we access this EDI archive?
It is a licensed system where we deploy it to customer's own cloud. Then the URL changes based on preference.
I often share free tools or open-source libraries but this one is a product right now.
Where can I get more details about "5. Our Prebuilt Solution". We are interested in it.
Also Road Map doesn't say much information about "3. Future: Archiving on the roadmap". Could you provide more details about this.
I followed your profile, you can follow me and we can communicate over https://messages.sap.com/ or you can add me on Linkedin.
For point 3, unfortunately, I also don't know many details. I'm waiting for the solution to take shape then I want to make the product more aligned with the standard solution, or remove it! 😉
For the prebuilt solution, we can discuss your need & use cases in detail. I'm also interested in hearing different perspectives.