Product Information
Intelligent Invoice Scanning
In this blog post, I will have a closer look at the ‘Intelligent Invoice Scanning’ feature, introduced in SAP Business ByDesign with release 2008. There are two key differentiators compared to the existing scanning solution:
- It is now sufficient to only upload PDF documents; a TXT document is not needed anymore.
- Scanning of the PDF is done using SAP’s Document Information Extraction Service (DOX), including an integrated OCR service as well as a machine-learning based model.
Supported languages are English, German, French, and Spanish. Please be aware, that the DOX service is currently not available in a Chinese data center, and therefore cannot be used in ByD tenants hosted in China.
In addition, I am pleased to mention that usage of the new scanning service is included in the normal SAP Business ByDesign licenses, no additional costs will be incurred!
This graphic shows the end-2-end process flow. It starts with uploading the PDF documents from your local file share using a new work center view ‘Invoice Scanning’. A mass upload of PDFs can also be done using SAP’s Collaboration Window, This can be downloaded from the Self-Service section in an SAP Business ByDesign system (Work Center ‘Home’).
Once uploaded, the PDFs are automatically sent to SAP’s Document Information Extraction Service, and invoice header and item information is sent back to SAP Business ByDesign. Since supplier master data can be uploaded to DOX, the supplier ID can directly be mapped to the scanned supplier information, like name, address, or other contact information.
The scanned information is first stored in a staging object, where the results can be reviewed, corrected, and once fine, the supplier invoice can be created.
Let’s have a look at the steps in more detail.
To activate the solution in your tenant, you must set the ‘Do you want to enter supplier invoices via PDF scanning?’ question in scope. If you want to automate the creation of the invoice in addition, you must also select ‘Do you want automatic creation of invoices from scanned documents?’. This means, that automatically a supplier invoice is created from the staging object.
No further setup needs to be done to establish the connection of the SAP Business ByDesign tenant to the DOX service.
Next, the new view ‘Invoice Scanning’ of the ‘Supplier Invoicing’ work center needs to be assigned to the relevant users.
I recommend to upload the supplier master data to DOX before the first invoices are scanned. This can be done in the ‘Invoice Scanning’ view in the work center ‘Supplier Invoicing’. Currently, whenever new scanning-relevant supplier master data have been created or changed, this upload should be repeated.
The action ‘Synchronize Master Data’ launches a screen, where the synchronization can be started.
Now, the system is prepared to start the scanning process.
As mentioned above, a mass upload of PDF documents can best be done using SAP’s Collaboration Window. Please do the following configuration settings under Menu – Settings – Invoice Scanning:
Please flag both options and specify the ‘Observed’ folder, from where to upload the PDFs. Next, you need to copy your invoice PDF into the observed folder, from where they get automatically uploaded, if the Collaboration Window is open.
The company information is retrieved from the user logged in to the Collaboration Window.
Independent of the way you use for uploading the PDFs, they then show up in the ‘Invoice Scanning’ view:
The status indicates, if scanning has been completed. You might need to refresh the list after a while. Once scanning is complete, you can take a look at the scanned values using the the action ‘Review’, that opens the staging object:
The scanned values are visible in the read-only fields on the left side of the UI. In case some values have not been scanned correctly, or have not been recognized at all, the reviewed values section allows you to manually override them. In my example, the supplier was not recognized. Once you are fine with the result, the invoice can directly be created from this UI, or from the ‘Invoice Scanning’ work list.
The invoice, that got created, can of course be reviewed once more in the ‘New Scanned Invoice’ UI, where also additional data like accounting assignment can be added.
With that, the invoice scanning and creation process is complete. Please also check out the video
YouTube link.
Blogs on newly added features:
Good update.
Will study this further in more depth for multiple processes.
Hi Stefan,
Very helpful blog.
Best Regards,
Arun
Hi Stefan,
this is great, thank you very much!
Best regards,
Daniel
Thanks Stefan!
Any possibility of configuring an email account, i.e. AP@company.com which suppliers can send all supplier invoices that will eliminate the need to manually upload PDF's?
Hi Andre,
currently this in not supported, but I would ask you to create a customer influencing request for the same, so that we can see how many partners and customers vote for it.
Best regards,
Stefan
Andre, this is a very good request. I opened a customer influence request. The ID is 252970. Let's vote on this request.
Name: Intelligent Invoice Scanning - corporate email account
Request ID: 252970, Category: Supplier Relationship Management
Hi Stefan!
We have run a few tests to check the capabilities and make some experience with this new feature in our test tenant.
The result is a mixed bag:
The classical process (with pdf and txt files) works much better - so far. But it needs the two files (prepared by OCR scanning manually or using an automated process).
But it would be good to learn some details about the AI features so that the functionality is fully understood, and expectations set accordingly. Things like: How was this tested? What model was used? Are there statistics of success vs manual handling etc.
Thanks
-Jan-
Hi Jan,
it would make sense to have a short meeting to discuss the next steps. What would be the best way to contact you?
Best regards,
Stefan
Hi Stefan!
Absolutely. A meeting would be great. I see others are interested in this too.
Works best for me in the coming week, Monday 24th or Friday 28th August.
Thanks
-Jan-
Hi Jan,
please get into contact with me via stefan.resag@sap.com.
Regards
Stefan
Hi Stefan
Exciting feature, but I am having the same issues as Jan Liedtke.
Can we also have a short meeting to take this forward.
Regards
David
Hi David,
sure. Please let me know how to best contact you.
Best regards
Stefan
Hello David, we are testing this feature and the system is working as defined in your blog. This is a great new future and will test it for a few weeks and then, if everything works well, we will deploy it in our company. Kudos to your team
Hi Stefan,
Thank you for so waiting-to-have feature !!!
In case of Errors Occured while fetching Scanned Invoice Data, shall I contact the SAP Support ? Also, if values are not well scanned ?
Kind regards,
Gosia
Hi Gosia,
This time, I have the same error as you. My invoices are scanned but no values are detected.
All fields are empty when reviewing the scanned invoice.
Perhaps OCR service not running on tenant / time to activate it ?
I also tried a manual upload and got the same error. And it took about 10 minutes before I could get the error message.
Kind regards,
François-Luc
Hi Fracois-Luc,
of course it should not happen that no values are detected. Please raise an incident. Of course, if you use a not-supported language then this might happen.
Kind regards
Stefan
Hi Gosia,
if you have the feeling that certain data are not recognized with an acceptable accuracy (e.g. 80%) then please create an incident. Please provide example invoices and the values that have not been recognized properly.
Kind regards
Stefan
Hi everyone, Stefan,
Has anyone tested the feature on partially invoiced POs ? (when only a part of a PO is invoiced)
Any positive tests on French or Spanish invoices for anyone so far ?
Regards,
Gosia
Hi Gosia,
I have made some test with discrepancies. DOX is scanning the pdf and show scanned values.
But once you create the invoice, you're back to the standard process flow and all figures (amounts and quantities) available in the PO are presented. I didn't find -up to now- a way to carry over the scanned values.
Stefan Resag, --> Correct me if I'm wrong, but I think you will have to retype the reviewed values.
Francois-Luc
Hi Francois-Luc,
if the scanning finds a PO then it will retrieve information from the PO, because the assumption is that this info is more accurate.
Best regards,
Stefan
Hi Stefan,
I wouldn't agree that values of the PO are more accurate than the ones of the scanned invoice. There are scanned (good) amounts shown in the review window nevertheless the created invoice retrieves the PO values instead of the scanned ones.
@François-Luc,
I logged the issue as an incident with SAP to see if they can do anything. I have also voted for your idea on this issue.
Regards,
Gosia
Hi Stefan,
As mentioned above, the company information is retrieved from the user logged into the Collaboration Window”.
If you have a tenant with different companies, the user logged into the SAP Collaboration Window will logically be employed by 1 company. But this is not always the company for which he/she wants to scan invoices.
Do you have to review all invoices ? A work around could be to find a colleague working for the right company to open the collaboration window and move the pdf’s...
Or have the possibility to define more than 1 e-mail as mentioned by @mmolinasap in his customer influence request.
Hi Francois-Luc,
can you please create an improvement request for the same so that we can see how many customers need an improved multi-company handling with the collaboration window.
Thanks and kind regards
Stefan
Hi,
Improvement request https://influence.sap.com/sap/ino/#/idea/253194 has been created. Don't hesitate to vote for it!
Hi Stefan,
Is it possible to influence the format of the scanned date?
The date is usually formatted DD-MM-YYYY (in Western Europe)
But 06/08/2020 is scanned as 8th of June.
Is it also linked to logged user in the SAP Collaboration Window ?
BR
Francois-Luc
Hi Stefan,
We have tested this new functionality and the biggest issue we've seen so far is that the supplier is not getting recognized on any of the invoices we tested.
Wonder if anyone else have the same issue.
Hi Jiawen,
I hope you have synchronized the supplier master data first, otherwise the ID would not get mapped. If you have synchronized the master data and still face the issue, you should raise an incident.
Kind regards
Stefan
Hi Stefan,
Synchronisation of master data always errors in sendning to API and it is the SAP Team which needs to unblock it every time (via incidents).
We are still in a phase of testing (as per known issues) but I am going to present the subject to my users in 10 days time.
Is it only us who are impacted by this error ? I cannot tell my users to use the scanning (even with current limitations) if every time I will need to contact SAP Support to unblock them with the synchronization.
What is your opinion on that ?
Regards,
Gosia
Hi Gosia,
I got informed by development that they plan to fix this. The fix is planned to be applied on the next weekend. Can you please check again on Monday.
Best regards,
Stefan
Hi Stefan,
Thank you for your update. Support confirmed to me that the fix should happen on the 15th but it is not a problem for me to wait.
As I have been conducting some more tests which failed in different areas, shall I contact the support and/or would you like me to share my results with you ?
Regards,
Gosia
Hi Gosia,
concerning the tests that failed. Yes, please contact support.
Best regards,
Stefan
Hi,
Is there in future possibility to teach the system to read the invoices. In many so called external services there is this possibility. Like where to find PO number, Suppliers invoice ID, Supplier name and how to read the line items. Some invoices goes thru quite ok in test the supplier field stays empty in all cases. Supplier data upload status successful.
br,
Heidi
Hi,
currently customer are not able to teach the system, but we are planning to provide a feature, that customers can send feedback. Based on the feedback we would then be able to train the model, which should over time result in improved accuracy. We plan to provide a pilot for 2011 that will allow us to test our internal process and also to see, how much the model can be influenced by that feedback.
Kind regards
Stefan
Hi Stefan,
Thanks for your feedback. Would it make sense to build this idea on so called "data bank" idea, where companies provide the invoice model to you. They would fill in the identification data could be like VAT and country where the company is located and then upload the invoice model in your engine. With analyze and modelling this would then be in use for all your customer. And then you could from customer upload pick the VAT as first indentification, might be that in some countries this field could be something else and it would be combined with upload model. The request for supplier to upload the invoice would be normal communication from us to our supplier as instructions how we handle their invoices.
Br,
Heidi
Hi Heidi,
what you described is one of the options the team is currently evaluating. But it will definitely take some time until we know if this approach will work and can also be operated by us with reasonable costs (knowing that the service is free of charge for our ByD customers).
Best regards
Stefan
Hi Stefan,
Is there a feedback channel on issues for this function or is it ok to give it here ? One issue occur her for me, how should you handle credit memos. Since the only function is to create invoice and we do scan credit memos too, will there be later a button create credit memo. Sometime in small companies you prefer to have all the attachments with the scanned invoice. It works quite ok when you really have the invoice in first pages but for example the problem is credit card so called statements you have lines that are debit and maybe the amount is in total credit some German companies mark that with H (haben) , even if they are as first page. If you have a receive behind where the word invoice is used ( like hotel invoice) the scanning pics this info. We are for moment working with non PO supplier invoices, it will get of course better when the PO process is in place later this year.
Solution is in good an review function is a good first step before creating the invoice but to thinking building automatic posting as own solution on this I think there is some work to be done.
Thanks for your support
br,
Heidi
Hi Heidi,
thanks a lot for your feedback. The best feedback channel is using the customer influence session for ByD ( hope this link works for you: https://influence.sap.com/sap/ino/#/campaign/886 ). By looking at the votes also from other customers and partners we can see the priority of this request.
Once votes are 20 or higher, we will review the requests.
Best regards
Stefan
Hi Stefan,
I have another subject related to amounts of PO which overwrite amounts of a scanned invoice: POs which are partially received. Ex. PO for 10j and 1j received per month.
Have you considered this use case ? In all our tests the link is not made. We also tried to match a receipt with a scanned invoice but it doesn't work either.
Shall I create an idea on that ?
Regards,
Gosia
Hi Gosia,
we plan to support a configuration (hopefully with 2102) that allows you to chose if PO values or scanned values shall be taken for invoice creation. There is already an idea you can vote for: https://influence.sap.com/sap/ino/#/idea/253792
Best regards,
Stefan
Great. Thank you for the news on 2102.
I added my comment to the idea however my use case narrows the proposition there as would like to have scanned invoices matched with goods/services receipts and not POs. Hopefully you can take it into your consideration.
Good morning Stefan,
Is our PO number a must to have in supplier's invoice for a match done during the scanning ?
It is because all my tests where it wasn't mentioned, are not linked with the PO.
Regards,
Gosia
Hi Gosia,
PO number is not a must during scanning. I am not clear about your statement 'It is because all my tests where it wasn’t mentioned, are not linked with the PO.' Of course, if the PO number is not detected, then the invoice is not linked to the PO automatically. You can always manually add the PO number, e.g. in the invoice creation UI.
Best regards,
Stefan
"Of course, if the PO number is not detected, then the invoice is not linked to the PO automatically."
It is the confirmation which I was looking for 🙂
I had been wondering if the intelligence behind is so clever that even if a PO number is not mentioned in a supplier's invoice, it will find a corresponding PO anyway (by an amount/supplier match).
Hence to have a PO detected automatically, a supplier needs to mention its number in their invoice.
Regards,
Gosia
Hi Gosia,
now I got your point. No, this is currently not supported.
Regards,
Stefan
2105 ? 😉
Hi Dennis,
the scanning solution is currently based on a pre-trained global model. Therefore a parnter-based localization is not foreseen. However, according to the official documentation the underlying DOX service already supports Dutch language. Please have a look here https://help.sap.com/viewer/5fa7265b9ff64d73bac7cec61ee55ae6/SHIP/en-US/5bf847f7d1a848dcb3513eff9ec70412.html
I would therefore ask you to try out a couple of Dutch invoices.
Best regards,
Stefan
Thanks Stefan for the great overview!
helpful blog for Intelligent Invoice scanning process
Invoice Scanning- Retirement of Collaboration Window
Hi Stefan
I understand the Collaboration Window is being retired soon.
This will impact the process you describe above as it is used to monitor a folder to import the pdf invoices.
What is the plan to replace this feature (if there is one)
Currently, customers and prospects are looking for more automation, not less. Scraping attachments from email is one common request (more like an expectation really). Is the RPA that I see in the roadmap for 21.08 intended to replace this process?
Thanks in advance
Dave
Hi David,
as per my understanding Microsoft Silverlight will be out of maintenance soon, but Collaboration Window can still be used. Nevertheless, we are currently working on a iRPA-based solution to use Bots for uploading PDFs from a file share for scanning. The current plan is to have the bot available with 2108.
Once this is more clear I will update the blog accordingly.
Regards,
Stefan
Many thanks, as always, for the prompt reply and information.
Dave
David Thorne you may like my little blog on scraping attachements
https://blogs.sap.com/2021/07/08/microsoft-power-automate-and-business-bydesign-supplier-invoice-scanning
Thanks John, something that is commonly asked for!
Dave
Hi Stefan Resag,
Thank you so much for the blog!!!.
Just have a question?
Will it support ODATA API to upload the invoices from external system?, I could see the option is available but when I try to consume the ODATA no entries are in the system.
Have raised the incident with SAP two weeks before, but no response yet.
Thanks,
Senthil
Hello Senthil,
yes ODATA is provided in standard and should work. We use the data ourselves to do the upload of PDFs for scanning via bot. Here you can find the ODATA service:
Work center 'Application and User Management' - 'Input and output management' - ODATA services:
Best regards,
Stefan
Hi Stefan Resag, ,
I agree with Jonas Bert , the OData is not working as expected, Can you share the sample payload for scanning via bot.
I've tried the below approach ended in error, also raised incident long back - 701747/2021
create staging scan invoice
process file
scanned invoice not processed or error
Best Regards,
Senthil
Hi Senthil,
this is the reply I got from dev. (...) they think you missed to send the document in the payload during the instance creation and trying to update it during processing of invoice which is incorrect. (...). I also got a postman collection which I added below. This worked on our side.
Best regards,
Stefan
Dear Stefan Resag,
We are currently investigating this option.
However when we try to upload an invoice via Odata, the invoice scanning remains in the status 'Scanning in Process'.
We are using a technical user which is assigned to the invoice scanning workcenter.
When we upload using user credentials, the scanning works fine.
Can you please advice on how to proceed?
Kind regards,
Jonas
Hi Stefan Resag, ,
Wonderful blog, thank you so much for this blog.
Our company was looking for a solution like this, can you please tell me whether uploading invoice image/ invoice pdf from external system via your api-services is available or not?
If so, then can you please redirect me to a documentation link ? I am searching for "Invoice Parsing in SAP" and among other words, but I am not being very successful in that.
Thank you.
Hello Sanjeeb,
we have an ODATA service 'scannedinvoicestagingodata' available in standard, which you can use to upload PDF documents for scanning. Once uploaded, the process follows the same pattern as if you would have uploaded the PDFs manually.
Best regards,
Stefan
Hi, one idea is to adapt the Invoice Upload template bot we released recently, which is design to picking the PDF from a folder and uploading it to IIS and customize it to pick the PDF from your other tool instead of a folder. That will minimize your implementation work. Note: IIS in ByDesing is only build to scan PDF invoices (no image format is allowed).
Is there a report of the changes that a user makes from the data identified in the scan to what is then posted?
Hi David,
there is no such report. What would be the use case? And as always, feel free to create an improvement request.
Best regards,
Stefan