New SCP Service (Beta) – SAP Data Quality Management, microservices for location data
NOTE: This offering has now exited beta and is now generally available to customers
You can find more information in the blog on SAP Data Quality Management, microservices General Availability.
What is “SAP Data Quality Management, microservices for location data”?
SAP Data Quality Management, microservices allows developers to embed data cleansing and enrichment services within any business process or application with a self-service, right-sized consumption model.
This is a new offering from SAP currently available as beta on the SAP Cloud Platform, developer edition.
This offering provides HTTP/JSON-based services running on SAP HANA Cloud Platform. Developers can simply integrate these services into their own applications to provide address cleansing/validation, geocoding, and reverse geocoding capabilities. Below is a summary of functionality provided in the beta.
Key Capabilities – 3 Services provided in the beta offering
- Address Cleansing
- Verify address per postal authority reference data as correct
- Standardize the way the address looks and is structured
- Correct components of an address if incorrect
- Enhance an address with missing attributes
- Assign diagnostic codes which describe why incorrect or what was corrected
- Geocoding
- Append latitude and longitude given an address
- Reverse Geocoding
- Provide address(es) given a latitude and longitude
We are hoping to get some early feedback on this service and if you have additional questions or feedback please post some comments here.
We are planning to provide pre-built integrations of these services in other applications from SAP which will provide a very simple way to consume this functionality. More to come on that in the future. If you have an application from SAP and you wish we provided these or other data quality type services within that application please let us know as well! These capabilities are just our starting point.
If you are an ASUG member can you find a recent recorded webinar and presentation on this offering: https://discuss.asug.com/docs/DOC-46194
What countries/languages do we support with this Beta offering?
This is available in the documentation provided for the service during beta which can be found here: Developer Guide
We are working on supporting additional countries/languages for our general release and beyond.
What are the benefits of this offering?
- Simple consumption model in the Cloud. HTTP/JSON-based services.
- Integration w/ many SAP Applications in the future
- Leading solution w/ 20+ years of experience in address cleansing and geocoding capabilities (technology acquired from Firstlogic/BusinessObjects)
- Strong partnerships with many worldwide postal authorities and data providers
- Complementary offering to our on-premise offerings in SAP Data Services and SAP HANA, smart data quality (SDQ)
How is this different than what’s offered in SAP Data Services or SAP HANA, smart data quality?
Some of you who are familiar with SAP Data Services or HANA SDQ may be asking how is this offering similar or different than those solutions.
- Leveraging the same core software engines in Data Services and HANA SDQ
- Similar functionality and behavior as our on-premise offerings
- Services in the cloud will provide the following benefits
- No downloads/updates of reference data (directories) – all managed by SAP in the Cloud
- Cloud pricing model(s) to provide more options and flexibility in consuming this functionality as needed (more to come after beta)
- Simple API
- No additional products/infrastructure needed
If you didn’t know that SAP offered these capabilities in other products here is more information on Data Services and HANA SDQ:
Data Services:
Data Quality – Enterprise Information Management – SCN Wiki
Geocoder Transform – Enterprise Information Management – SCN Wiki
Global Address Cleanse Transform – Enterprise Information Management – SCN Wiki
HANA SDQ:
SAP HANA smart data integration & smart data quality – YouTube
SAP HANA Academy – Smart Data Quality: Address Cleanse Transformation Basics [SPS09] – YouTube
SAP HANA Academy – Smart Data Quality: Geocode Transformation Basics [SPS09] – YouTube
SAP HANA Academy – Smart Data Quality: Reverse Geocode [SPS 11] – YouTube
How do I get access to use the beta?
1. Get an SAP Cloud Platform, developer edition account
Go here: Try SAP Cloud Platform
Ensure you check “Enable Beta Features” for your account once you have access to SAP Cloud Platform Cockpit.
First select the edit button for your account on SCP, developer edition. You can get to this screen below by selecting the first drop down at the top of the SCP Cockpit and selecting the “Europe(Trial)” data center which relates to the SCP, developer edition. Then select the pencil icon for your trial account as seen below.
Then select “Enable Beta Features” for that account and save:
2. Enable our service for your account
Browse to “services” in SAP Cloud Platform Cockpit.
Under the “Business Services” category you’ll see a tile for “Data Quality Microservices”
Click on the tile and on the next screen you will see an “Enable” button. Select that.
Once enabled you are ready to go.
3. Review the Documentation
You will see a link on the service page where you enabled our service to the documentation.
Or you can directly access here:
Examples of calling the Service(s)
Postman
We have recently posted some scripts which can be used with the Postman Service utility (www.getpostman.com).
You can find these sample scripts and how to configure them for use with your SCP, developer edition account and our microservices here at the public SAP Github repository:
https://github.com/SAP/cloud-dqm-sample-payloads
Java Example
There is also a sample JAVA application which you can find a blog post here:
https://blogs.sap.com/2016/07/11/test-drive-the-new-dqaas/
However, use this Github repository instead going forward vs. the one mentioned in this blog:
https://github.com/SAP/SAP-cloud-dqm-sample-java
SAP Data Services:
If you are an SAP Data Services customer we have recently introduced a built-in transformation which integrates these microservices. This is included in the recent release SAP Data Services 4.2 SP8.
Example Data Services Jobs (Blueprints) can be found here:
https://wiki.scn.sap.com/wiki/display/EIM/SAP+Data+Services+Blueprints
You can find more information in the official documentation here starting on page 754:
http://help.sap.com/businessobject/product_guides/sbods42/en/ds_42_reference_en.pdf
Limitations
This service is limited to 90 days of usage or 1000 transactions per country. Tracking starts with the first transaction you make to the service.
Can someone better clarify where is possibile to find the list of providers mentioned in the statement "Strong partnerships with many worldwide postal authorities and data providers"?
thanks
Massimo,
We don't document or generally communicate who all the partners are that we work with to provide these services. However, if you have a business reason for needing more insight on who we're working with we can communicate offline on this topic. If you want to follow-me on SCN I can send you a direct message for further follow-up.
Doh - and Australia isn't on the supported country list. Fortunately, I can still get most of the data I need from Google's services. Although I guess some people would have issues sharing location data with Google?
This is primarily due to it being a beta release right now. There are a number of countries that aren't on the supported list right now during the beta but will be when we release to customer and beyond. In our on-premise technologies we have a certified address cleansing capability with Australia Post (AMAS Certification). You can find SAP listed here: Address validation - Australia Post We also support the geocoding features for Australia today in our on-premsie technologies. The SAP on-premise technologies that also include these features are SAP Data Services and SAP HANA, smart data quality. So we will support Australia in the future.
Chris,
Today we released support for Australia and also the United Kingdom for our service to HCP developer edition. The documentation is still in process of being updated though. Please give it a try.
Ryan
Hi there,
I've started to create a simple consumption model & native JAVA API wrapper - it's very basic as-is, but I'll make it more fancy in the coming days/weeks.
https://github.com/SAP/cloud-service-dq-api
Cheers!
FYI We are also running a guided beta of this service from July 15-September 8th. The guided beta will include some additional capabilities and also include the ability to test an updated version of SAP Data Quality Management, version for SAP which is an ABAP Add-on for SAP ERP Central Component (ECC), CRM, and Master Data Governance which provides a built-in way to do address cleansing and geocoding for customers, suppliers,accounts, contacts, etc....
To register for the Guided Beta go here: https://influence.sap.com/BetaSAPDataQualityManagementmicroservices
Some more information about SAP Data Quality Management, version for SAP can be found here:
https://wiki.scn.sap.com/wiki/display/EIM/Data+Quality+Management+for+SAP
Data Quality Management, version for SAP, Product Tutorials
SAP Data Quality Management for SAP Solutions, 4.0 SP08 – SAP Help Portal Page
Hi Ryan,
Will this offering be limited to only the applications built on HCP platform or even Third Party systems can use this API ?
Regards - Santosh G.
These services will be available for application consumption outside of HCP for sure. That is part of the goal. We're using Hana Cloud Platform as our platform for this service. We are also planning to make this service available stand alone so would not require a purchase of Hana Cloud Platform itself.
These work really well and are easy to consume. Myself and a colleague (Jon Green) managed to invoke the services after a few minutes trial and error with a REST tool.
Michael
Unfortunately I can't find the the documentation the the endpoint URLs for geocoding and suggestion lists.
I was able to find endpoint URLs in http://help.sap.com/saphelpiis_dqmmicro1/dqm_micro_loc_1_dev_en/frameset.htm?9f7e7a5f9cb7482785ad8913e5141bf2.html and in http://help.sap.com/saphelpiis_dqmmicro1/dqm_micro_loc_1_dev_en/frameset.htm?96a794e1ebc440808fa4952fe257d103.html for Cleansing and Reverse Geocoding.
Geocoding is actually just part of the "addressCleanse" endpoint. You just request the additional outputs with LAT/LONG and if needed additional informational fields for geocoding. Reverse Geocoding is it's own endpoint. You can find more information here: https://uacp2.hana.ondemand.com/viewer/#/1e649b6da162465db21c818291f7ed3a/latest/en-US/c1ec4a16c1674a368859d90cacaaa3c8.html
We have recently made an update to the HCP Developer Edition for this service so please recheck the tile under services as the link for documentation has been updated and details about the service has changed. More to come here in the next few days.
Ryan
Hi,
I was looking into creating a PoC with this service, but unfortunately, the 90 day trial period had expired. I glossed over that point when I first tried it and was only looking into the capabilities of HCP. When I started looking into it seriously and was prepared to demo something with this service, the 90 day trial literally expired the weekend before the demo. Is there anyway to extend the trial?
Thanks,
Karen
Thanks Ryan for this great blog and for keeping us updated on the GA of the DQM microservice.
Has anyone try it on SAP Hybris C4C?
Cheers
Linh
Hi,
I got this up and running in Postman in a few minutes, and I'm no developer, so thanks. I can't however get it to recognise a UK address, even though I set the country as GB. Would you expect this to work for the UK at the moment? The documentation seems to suggest not?
https://help.sap.com/viewer/d95546360fea44988eb614718ff7e959/Cloud/en-US/1d9c9b02a6b1434c89f2929e3636e431.html
But the /refData/country service does return GB as a possible country.
Thanks,
Ian
Thanks,
Ian
Hello Ian,
UK is availible for use now. I am belive that when you tried thetrial last November (2017) the microservice was configured to work with the UK data but it was not officially supported.
Regards,
Michael