Technical Articles
CAP: Consume External Service – Part 2
| Updated on 9 November 2022 |
In my previous blog post Consume External Service – Part 1, I have tested the application using mock and real data. Just by configuring the NorthWind OData service URL in the credentials.url property, I was able to connect to the external service. This approach is only applicable for local development testing. If you try to deploy this into SAP BTP, you will encounter an error stating that this approach is not valid for production use.
The reason behind this is that external service consumption in this CDS framework is meant to use the Destination Service in Cloud Foundry. And this is the topic of this blog — deploying the Node.js project into SAP BTP Cloud Foundry using Destination Service.
Prerequisites
- SAP Business Application Studio / Visual Studio Code
- SAP BTP – Cloud Foundry Environment
- You’ve already gone through Consume External Service – Part 1
Blog Post Series
- Consume External Service – Part 1
- Consume External Service – Part 2
- Consume External Service – Part 3
Setup Destination Configuration
First of all, why do we need to set up a destination? A simple explanation is that it is best practice to avoid hard coding the external services that you use in your application inside your code. It is better for it to be configured separately from your application because as you deploy your application into different environments like DEV, QA, or PROD, you can have a different setup of the destination per environment. Also, it keeps sensitive information like credentials outside of your code.
See Step 8 of Deploy the application to BTP Cloud Foundry
Deploy the application to BTP Cloud Foundry
Now let’s go back to our Node.js project — if you were able to follow through my previous blog post, then your project will look exactly like the one I have below:
https://github.com/jcailan/cap-samples/tree/blog-es-part1
The next step is to prepare the application for deployment to BTP Cloud Foundry.
- 1. Generate the mta.yaml file using the command below:
> cds add mta
- 2. Generate the security descriptor file xs-security.json using the command below:
> cds add xsuaa
- 3. Update mta.yaml file with declaration of xsuaa and destination resources and then binding it to our node module cap-samples-srv.
---
_schema-version: "3.1"
ID: com.jcailan.cap-samples
version: 1.0.0
description: "A simple CAP project."
parameters:
enable-parallel-deployments: true
build-parameters:
before-all:
- builder: custom
commands:
- npx -p @sap/cds-dk cds build --production
modules:
- name: cap-samples-srv
type: nodejs
path: gen/srv
parameters:
buildpack: nodejs_buildpack
build-parameters:
builder: npm-ci
provides:
- name: srv-api # required by consumers of CAP services (e.g. approuter)
properties:
srv-url: ${default-url}
requires:
- name: cap-samples-uaa
- name: cap-samples-destination
resources:
- name: cap-samples-uaa
type: org.cloudfoundry.managed-service
parameters:
service: xsuaa
service-plan: application
path: ./xs-security.json
config:
xsappname: cap-samples-${org}-${space}
tenant-mode: dedicated
- name: cap-samples-destination
type: org.cloudfoundry.managed-service
parameters:
service: destination
service-plan: liteNote that in order for the destination service consumption to work, an xsuaa service is required to be bound to our application.
- 4. Update the package.json cds configurations to use the NorthWind destination (we will create the actual destination configuration later after deployment):
"cds": {
"requires": {
"NorthWind": {
"kind": "odata",
"model": "srv/external/NorthWind",
"[backend]": {
"credentials": {
"url": "https://services.odata.org/Experimental/OData/OData.svc"
}
},
"[production]": {
"credentials": {
"destination": "NorthWind"
}
}
},
"[production]": {
"auth": {
"kind": "xsuaa",
"restrict_all_services": false
}
}
}
}| NOTE: The restrict_all_services = false removes authentication to all the provided service endpoints and we are using this for this example just for the sake of simplicity and because the original service endpoint doesn’t have authentication in place. However, in normal cases, you would want to keep the authentication on especially when the app is used productively. |
- 5. Build the MTA Project by using the command:
> mbt buildMake sure you have saved all the file changes we did before doing the build.
An MTA archive file will be generated in the mta_archives folder.
- 6. Deploy the MTA archive file into BTP Cloud Foundry using the command:
> cf deploy mta_archives/com.jcailan.cap-samples_1.0.0.mtar- 7. Once the deployment has been completed, look out for the terminal logs which state the URL of your
cap-samples-srvmodule. In my case, here’s the generated URL:
s0017687913trial-dev-cap-samples-srv.cfapps.eu10.hana.ondemand.com- 8. But before proceeding to test the application, make sure to add the configuration for the
NorthWinddestination. Go to BTP Cockpit and look for the destination service instancecap-samples-destination— then add below configuration:
Test the deployed Node.js app service
- 1. Open the URL we got from the previous step using your favorite browser, the initial page will show, and now click on the Products entity. You should see the result below:
As you can see, it is very easy to deploy the app and use the destination service. We don’t really need to do any additional JavaScript coding, all that we did is just do a little bit of configuration.
Testing the Node.js app locally
Now that we have deployed our application into BTP Cloud Foundry, we can test the app locally while still using the destination service in BTP Cloud Foundry. In order to be able to test locally, we need to capture the environment variable assigned to our Node.js module in BTP.
- 1. Execute the command below to fetch the environment variables bounded to your deployed node module app:
> cf default-env cap-samples-srv
Environment variables for cap-samples-srv written to default-env.json| NOTE: You need to have the CF Plugin called DefaultEnv installed in your machine — see installation instructions here: https://github.com/saphanaacademy/DefaultEnv |
- 3. Next thing to do is start the app locally using the command:
> cds watch --profile production- 4. Test the app on a browser by using the URL below:
http://localhost:4004/catalog/Products
As you can see we now can test the app locally using real data while also using the destination service of BTP Cloud Foundry.
If by any chance you still want to revert to testing the app using our mock data, you can still do so by executing the command:
> cds watch
Closing
Now you know how easy it is to consume an external service using CAP Model and use Destination Service to manage the connection configuration. We also tested our application in three different ways:
- Test the service directly in BTP Cloud Foundry
- Test the service locally while still using BTP CF Destination Service
- Test the service locally using mock data
Take note that mocking the data for your service is essential for automated unit testing, but this is another topic better dealt with over another blog.
UPDATE:
Taking this project further into — Unit Testing using Mocha and Chai
~~~~~~~~~~~~~~~~
Appreciate it if you have any comments, suggestions, or questions. Cheers!~
Hi Jhodel, Im having troubles making the call to the destination it always prints a 500 erro with a 400 status code in the message. Also it says about not finding a proccess env called HTTPS_PROXY and then it fails the service run.
Don't know whats happening, and I followed your blog perfectly.
Thanks Jhodel Cailan
Hi Roc Antonio
It looks like the issue is in your configuration of default-env.json, make sure it has the right structure. Also, you need to make sure that the destination service and xsuaa services are setup properly.
Thanks and regards,
Jhodel
Hi Roc,
I has the same problem, have you found the way to resolve this?
I was missing the Connectivity Service on the Cap Instance
Hi Antonio,
I am also getting exact same error even after using full URL. How did you fixed that issue ?
I am getting this issue even for northwind odata.
Thanks,
Manu
Hi Jhodel,
Very useful post, I followed your steps and works fine, but now I want to replay this for a S/4HANA service, but I always get this error:
This is my mta.yaml
and my package.json
The destination is been using without problems by other apps, but with this method using external services in CDS is not working.
Thanks in advance
Hi Luis Guillermo León Guzmán
I saw your post in the questions section, and it is true that you didn't configure the destination with the full URL of the external service. This is currently a limitation in the CDS framework.
Cheers!
Jhodel
Thank´s Jhodel,
Now it works with the full URL.
Best regards
Hi Luis,
If I use the URL instead of destination it works fine, but when I use destination it don't work anymore, need to use destination because using URL is not a suitable solution for my.
Can you share how to solve using destination please?
Thanks
This is a great pair of posts, nice work Jhodel!
Thanks DJ Adams ! It's an honor to be noticed by you! 😀
Hi again Jhodel,
I noticed that you cannot do expands with a remote service like this although you can do filters.
It's just me or its the same on all people ?
Thanks for the great post 😉
Thanks Jose! Yes, it’s the framework issue ?. It is a limitation at the moment. If you really need this feature then you have to code it yourself — without the use of CDS fluent API.
Cheers! Jhodel
Hi Jhodel...
I am using on-premise s4h service. I want to read child entities by using $expand. without coding can we achieve it right now?
Thanks Jhodel! Very clear step-by-step instructions, super helpful!
Happy to know that my post has helped you Chunyang!
Thanks, Excellent Blog, worked perfectly.
I want to expose few more entities . I did below changes and it is not working. What should I code more ?
Thanks Karthiheyan !!
Is it not working because you got an error? or because it doesn't show any data? I checked your code and it looks fine. You should just create a file called NorthWind-ProductDetails.csv which has a similar data like below:
Hi,
It is working locally. But after deploy to CF , it is giving below error,
Request failed with status code 400 - error after clicking on Entityset in CF link.
Hi ,
It worked after specifying field names like ID,Name
Hi Thanks Jhodel Cailan
Thanks for the well-written article.
I tried the steps. It worked fine in local but when deployed it gave me 500 (internal server error).
Can you please suggest how can I troubleshoot?
Regards,
Lalit
Hi Lalit Goyal
I assume that you were able to deploy it but when you are testing the service it gave you an error 500 Internal Server Error? If this is a yes, then go back to your terminal and execute below command
This will connect you to your deployed service and will see logs in real-time. Trigger again the service that is causing errors. Then analyze the logs from your terminal and see if you can find any clue.
I am attaching the error here.
2020-07-09T16:34:50.49+0530 [APP/PROC/WEB/0] ERR [2020-07-09T11:04:50.497Z | ERROR | 1884455]: Failed to build
HTTP request for destination: failed to load destination!
2020-07-09T16:34:50.49+0530 [APP/PROC/WEB/0] ERR [2020-07-09T11:04:50.497Z | ERROR | 1884455]: Error stacktrace: Error: Failed to build HTTP request for destination: failed to load destination! at Object.errorWithCause (/home/vcap/app/node_modules/@sap-cloud-sdk/util/dist/error.js:6:20) at /home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/http-client/http-client.js:143:38 at process._tickCallback (internal/process/next_tick.js:68:7) Caused by: Error: Unable to get access token for "destination" service! No service instance of type "destination" found. at Object.resolveService (/home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/environment-accessor.js:190:19) at Object.<anonymous> (/home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/token-accessor.js:73:54) at step (/home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/token-accessor.js:45:23)
at Object.next (/home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/token-accessor.js:26:53) at /home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/token-accessor.js:20:71 at new Promise (<anonymous>)
at __awaiter (/home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/token-accessor.js:16:12) at Object.serviceToken (/home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/token-accessor.js:70:12) at /home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/destination-accessor.js:197:59 at step (/home/vcap/app/node_modules/@sap-cloud-sdk/core/dist/scp-cf/destination-accessor.js:45:23)
Also, I want to add that I am using a trial account.
Hi Lalit Goyal
I just got to know you have responded. Is your destination configuration correct? can you share your destination configuration?
Hi Jhodel,
I am attaching the destination configuration screen shot.
Regards,
Lalit
Destination Configuration
Hi Lalit Goyal
It looks correct to me, except that the name of your destination is NorthWind_CLONING. If you named your destination like this, then you need to make sure you use this exact name on your configuration when you want to consume this destination from your CAP Model project.
Hi Thanks Jhodel Cailan,
I found out the issue. Well, I didn't restart the application after binding to the destination.
After restarting the application, it worked. Maybe you can add that step too 😀
Thanks a lot for your help!
Regards,
Lalit
Hi Lalit Goyal
Glad to know that it is working fine now. There must be some steps that you did in between which requires a restart for your case. For my steps above, the destination was configured first, then build and deploy of MTA. The MTA will be the one who will bind to the service (auto-binding) and the app should be restarted afterward, hence, manual restart is not necessary.
Well, that's how we learn, which is by doing it! 🙂
I was facing the same issue, but what I can see is that the service instance demo-destination and demo-uaa are not auto binding to the CAP app. after I created the binding manually, the service works as expected.
Hello,
I also have a problem getting this setup. Im using business application studio.
The error I get looks like this:
Thats my MTA.yaml:
My Apps rootfolders package.json:
And the default-env.json:
The destination configuration:
When accessing the entity I get this error:
Are you testing the service locally? For local testing you should use the "url" as credentials. The configuration you have in package.json is good for SCP deployment testing.
Hello Jhodel
Yes, I want to test locally using "cds watch". I tried doing that, the code looks like this:
However now upon calling the entity I get this error:
Maybe the error was right? You don’t have a custom handler implemented? Did you read the first part of this blog post? In part 1, I have step there to implement the custom handler .js file.
I actually had, my cat-service-cds looks like this:
The cat-service.js like this:
What I did, I just changed:
to
Now this error pops up:
If I console.log the query, I get a HTML structure returned, having this paragraph:
It's still tough to see where the issue is. If you can share the actual project you have right now through GitHub (or any other Git versioning tools), that would be great.
Hi Christian Hanusch
I am also getting exact same issue after deploying the app to CF. Were you able to fix ?
I am getting this issue even for northwind odata.
Thanks,
Manu
Hi Christian Hanusch
Any good news ? I got same issues .
Hi Jhodel,
Thanks for such wonderful posts! I was able to follow and perform all the steps. Only part i am not able to make work was correct deploy to CF.
Though deploy is successful and when i open the link after deploy, i even see metadata. But when i click on 'Products', i get 500 internal server error. What could be the issue?
Gaurav
Thanks Gaurav Karkara !
500 internal server errors are usually brought by some problems with the implementation of the service. It's likely that you got some issues with your code, but I can't tell what is it based on the information provided.
What you can do is go to SCP Cockpit, navigate to the applications section and open the cap service details and extract logs there to view the logs information and try to make sense of what does the error logs tell you. Usually, it will become obvious where is the error from the logs, but if that's not the case, you can share it here so that we can analyze. Cheers!
Thanks Jhodel for replying.
I have looked at the logs and find the error:
'Unable to get access token for 'destination' service! No service instance of type 'destination' found.'
I have followed the exact steps as you have followed and my destination instance and corresponding entries in mta.yaml seems fine.
It doesn't look like it's because of code. It's something to do with destination instance access. Still trying to find..
Thanks for your support.
Gaurav
Worked. had to bind instances to applications. Thanks
Hi Gaurav Karkara
Good to know that you found the issue and it is now working! 😀
Awesome Blog and really appreciate it. But I am facing one problem I am trying to test it Locally but I am Facing this issue.
Can you please help me to resolve this issue?
This is my mta.yaml
Hi ,
Any lead on how to use service.tx(request).run(request.query) for a post call to S/4 odata.
Exact question
Regards,
Karthi
I'm afraid that you are going to code the logic for that on your own. According to the documentation below:
https://cap.cloud.sap/docs/guides/consuming-services#sending-requests
the fluent API can cater to do create (POST) request, however, from my experience, it didn't work when it starts to use the service via connectivity proxy. So what I did, is I have coded the solution my self using the axios node module -- it is the same node module that cap for node.js use.
Thanks Jhodel 🙂
Hi guys,
is this working on webide or I should deploy to make it work? because I'm trying to access oData service but I receive
{"message":"Unable to create \"SAP-Connectivity-Authentication\" header: no JWT found on the current request.\n Continuing without header. Connecting to on-premise systems may not be possible.","level":"warn","custom_fields":{"package":"core","messageContext":"connectivity-service"},"logger":"sap-cloud-sdk-logger","timestamp":"2020-08-05T18:49:38.260Z","msg":"Unable to create \"SAP-Connectivity-Authentication\" header: no JWT found on the current request.\n Continuing without header. Connecting to on-premise systems may not be possible.","written_ts":1596653378260,"written_at":"2020-08-05T18:49:38.260Z"}
{"message":"OnPrem destination proxy settings from connectivity service will be used.","level":"info","custom_fields":{"package":"core","messageContext":"proxy-util"},"logger":"sap-cloud-sdk-logger","timestamp":"2020-08-05T18:49:38.261Z","msg":"OnPrem destination proxy settings from connectivity service will be used.","written_ts":1596653378261,"written_at":"2020-08-05T18:49:38.261Z"}
Error: Failed to build HTTP request for destination: failed to build headers!
The steps outlined in this blog post is not meant to be executed in SAP WebIDE. Read the prerequisites section for more information.
Hi Thanks Jhodel Cailan,
thanks a lot for this very well article, I’m tring with it since a few days ago. I'm trying to use an external Hana Odata service.
I achieved to make it work locally both with URL and destination, but in the latter case, for it to work I had to delete the node_modules directory, another case if this directory exists I can't reach the Odata and got an 404 or 500 error. I don’t understand why that happens, I suspect that there is some incompatibility between the npm packages that are in that directory and the components in the CF, or not?
Is there an option so that when deploying it does not take the npm packages that exist in the node_modules directory and instead uses the ones that exist in the CF?
Hi
I'm not sure why this should be the root cause of your issue, but you can avoid deploying the node_modules folder to CF in two ways:
Regards
Hi Klaus,
thanks for your answer, you are right the command "cf push" does not generate the directory "node_modules", instead it installs the dependencies in the process, however the process fails due to the following errors:
My package.json file is:
Hi Fadel,
I just have a vague idea what happened. Could you check in the log output of npm which npm registry is contacted to download the cds package? If it is still npm.sap.com, then this is a general issue, because cds 4 is not there. cds 4 is only available on npmjs.org. AFAIK, npm.sap.com will not be maintained anymore ...
Regards
When i am using destination with basic authentication then it is working fine. But when i am using destination of type OAuth2SAMLBearerAssertion then system is not able to retrieve destination.
cds.connect.to(‘SF’)
can above statement automatically pas required JWT token. Please help is very urgent.
I am also getting below error:
Unable to match a specific XSUAA service instance to the given JWT. Is this common?
@Jhodel
Can you please help here.
System is able to load destination with basic authentication but destination with type OAuth2SAMLBearerAssertion it is not.
Failed to build HTTP request for destination: failed to load destination!
---new update
Looks like cds.run and related statements don't support to pass JWT token to destination. Changing all the code to axios.
Hi Thanks Jhodel Cailan,
I followed the steps mentioned in your blog to consume an external service in my project using the destinations service instance.
I was successfully able to test the application locally, but as soon as I deploy the code to CF environment and try to access the service, I get 502 - Bad gateway error. Any idea what could be the reason for this error?
The cds > requires section in package.json file is as follows:
Also the destination that was created is a shown below:
Regards,
Sangita Purkayastha
Hi Sangita Purkayastha
The error 502 is very generic. Based on the information you provided, I can suspect that there can be an issue in your connection to HANA database. I suggest to do some troubleshooting on the deployed app like catching the error in the section of your code and logging the error into the console to get more information about the 502 error.
Hi Jhodel @Thanks Jhodel Cailan
Thank you very much for sharing the knowledge on this topic. I had similar issue here. Could you please kindly provide some hint on how to solve this issue?
When I tried to "Deploy the application to SCP Cloud Foundry", Using the created URL, i could access the top screen as shown below, but I can not enter the "Products" detail screen" only got 502 Bad Gateway error.
top top could be shown properly
Products page could not be shown
Guys,
Problem lies in your destination in CF, Check the name, It must be NorthWind where W must be caps. I would request to use the attached destination to import directly.
#
#Mon Apr 26 15:59:02 UTC 2021
URL=https\://services.odata.org/Experimental/OData/OData.svc
Name=NorthWind
ProxyType=Internet
Type=HTTP
Authentication=NoAuthentication
Description=NorthWind
I am also facing the same issue..Please help me how you resolved it
Hi Sangita,
I am also facing the same problem when using odata destination which have Basic authentications, With local data its working fine as per Thanks Jhodel Cailan part1 of this series but when using destination its giving "Bad Gateway" error, Are you able to resolve it somehow?
Hi Thanks Jhodel Cailan
Thanks for this article, very helpful and works like a charm!
Is it possible to reuse the external service in the local CAP model once defined? I.e. incorporate the external service built using this guide into another service which is running in the local data model.
In essence I have this external service (business partners from my backend ERP)
Implemented like so:
Now I want to utilize that entity from another service, basically like this:
This compiles fine and publishes the services. But if I try to navigate to a BusinessPartner entity from within the "projectService" I get:
Can you shed some light on what the intended design pattern is here? I would like to avoid duplicating the code implementing the external service (and doing that causes other issues since the cds.connect.to('service') part only returns the actual service on the first shot).
Again, thanks for the guide and hoping for some input on this scenario.
Regards
//Carl
Hi Carl,
Did you solve this problem?
Hi,
No. I think it is doable but we also had issues with the external service being oData v2 and managing transposal between v2 and v4 became cumbersome. We decided to set up replication of the source objects into the CAP data model instead.
Regards
//Carl
Hi. When I'm deploying the file to BTP I get the error "In production mode it is required to set `options.destination`". How can I set it up? Thanks
Hello Fedor,
I am getting the same issue. Did you fix this?
Hi. If you create an app from template then do not forget to add flag node start - productive
Hi Fedor,
Where did you insert this flag? I am getting the same issue
in mta
hello,
I am getting the same issue. but i have the meta configuration with --production
any idea ?
Hi Jhodel Cailan,
Great Blog post!!
How can i consume non odata services into CAP ?
Thanks,
Rajdeep Bhuva
Awesome Blog! Appreciate it a lot, Just one problem on the final step when i am trying to test it locally, i am getting this error, Can anyone please suggest some solution?
Also I can clearly see that binding of XSUAA is already there
bindings
my mta looks like this(confused about build params if they are correct or something more required)