Skip to Content

The following steps will explain how to introduce resilience to your application using the SAP S/4HANA Cloud SDK. If you want to follow this tutorial, we highly recommend checking out the previous parts of this series. For a complete overview visit the SAP S/4HANA Cloud SDK Overview.

Goal of this Blog Post

This blog post covers the following steps:

  1. Explain what resilience is and why you should care about it
  2. Make the call to the OData service resilient by using Hystrix-based commands
  3. Write Tests for the new Hystrix-based command
  4. Deploy the application on SAP Cloud Platform Cloud Foundry

Resilience

Consider the following situation: you are developing a cloud application to provide a service to your customers. In order to keep your customers happy, you’re of course interested in achieving the highest possible availability of your application.

However, cloud applications, possibly spanned across multiple services, are inherently complex. So we can assume that something, somewhere will fail at some point in time. For example a call to your database might fail and cause one part of your application to fail. If other parts of your application rely on the part that has failed, these parts will fail as well. So a single failure might cascade through the whole system and break it. This is especially critical for multi-tenancy applications, where a single instance of your application serves multiple customers. A typical S/4HANA multi-tenancy application involves many downstream services, such as on-premise S/4HANA ERP systems.

Let’s look at a concrete example: Suppose you have 30 systems your cloud application is dependent on and each system has a “perfect” availability of 99.99%. This means each service is unavailable for 4.32 minutes each month (43200min * (1 – 0.9999) = 4.32min).

Now assume failures are cascading, so one service being unavailable means the whole application becomes unavailable. Given the equation used above, the situation now looks like this:

43200min * (1 – 0.9999^30) = 43200min * (1 – 0.997) = 129.6min

So your multi-tenancy application is unavailable for more than two hours every month for every customer!

In order to avoid such scenarios, we need to equip applications with the ability to deal with failure. If an application can deal with failures, we call it resilient. So resilience is the means by which we achieve availability.

Hystrix

The SAP S/4HANA Cloud SDK builds upon the Hystrix library in order to provide resilience for your cloud applications. Or, to put into the words of the creators of Hystrix: “Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.”

Hystrix comes with many interlocking mechanisms to protect your application from failures. The most important are timeouts, thread-pools and circuit-breakers.

  • Timeouts: Hystrix allows setting custom timeout durations for every remote service. If the response time of a remote service exceeds the specified timeout duration, the remote service call is considered as failure. This value should be adapted according to the mean response time of the remote service.
  • Thread-pools: By default, every command has a separate thread-pool from which it can requests threads to execute the remote service call in. This has multiple benefits: every command is isolated from your application, so whatever happens in these threads will not affect the performance of your application. Also, the usage of threads allows Hystrix to perform remote service calls asynchronously and concurrently. These threads are non-container-managed, so regardless of how many threads are used by your Hystrix commands, they do not interfere with your runtime container.
  • Circuit breaker: Hystrix uses the circuit breaker pattern to determine whether a remote service is currently available. Breakers are closed by default. If a remote service call fails too many times, Hystrix will open/trip the breaker. This means that any further calls that should be made to the same remote service, are automatically stopped. Hystrix will periodically check if the service is available again, and open the closed breaker again accordingly. For more information on the circuit breaker pattern, check this article by Martin Fowler.

Additionally, Hystrix enables you to simply provide a fallback solution. So if a call fails, for example because the thread-pool is depleted or the circuit breaker is open/tripped, Hystrix will check whether a fallback is implemented and call it automatically. So even if a service is unavailable we can still provide some useful result, e.g. by serving cached data.

If you want to gain a deeper understanding of the inner workings, checkout the Hystrix Wiki.

Make your OData call resilient

Now that we have covered why resilience is important and how Hystrix can help us achieve resilience, it’s finally time to introduce it into our application. In the last tutorial we created a simple servlet that uses the SDK’s Virtual Data Model (VDM) and other helpful abstractions to retrieve business partners from an ERP system. In order to make this call resilient, we have to wrap it in an ErpCommand. So first we will create the following class:

./application/src/main/java/com/sap/cloud/sdk/tutorial/GetBusinessPartnersCommand.java

package com.sap.cloud.sdk.tutorial;

import org.slf4j.Logger;

import java.util.Collections;
import java.util.List;

import com.sap.cloud.sdk.cloudplatform.logging.CloudLoggerFactory;

import com.sap.cloud.sdk.s4hana.connectivity.ErpCommand;
import com.sap.cloud.sdk.s4hana.datamodel.odata.helper.Order;
import com.sap.cloud.sdk.s4hana.datamodel.odata.namespaces.businesspartner.BusinessPartner;
import com.sap.cloud.sdk.s4hana.datamodel.odata.services.DefaultBusinessPartnerService;

public class GetBusinessPartnersCommand extends ErpCommand<List<BusinessPartner>> {
    private static final Logger logger = CloudLoggerFactory.getLogger(GetBusinessPartnersCommand.class);

    private static final String CATEGORY_PERSON = "1";

    protected GetBusinessPartnersCommand() {
        super(GetBusinessPartnersCommand.class);
    }

    @Override
    protected List<BusinessPartner> run()
            throws Exception {
        final List<BusinessPartner> businessPartners =
                new DefaultBusinessPartnerService()
                        .getAllBusinessPartner()
                        .select(BusinessPartner.BUSINESS_PARTNER,
                                BusinessPartner.LAST_NAME,
                                BusinessPartner.FIRST_NAME,
                                BusinessPartner.IS_MALE,
                                BusinessPartner.IS_FEMALE,
                                BusinessPartner.CREATION_DATE)
                        .filter(BusinessPartner.BUSINESS_PARTNER_CATEGORY.eq(CATEGORY_PERSON))
                        .orderBy(BusinessPartner.LAST_NAME, Order.ASC)
                        .top(10)
                        .execute();

        return businessPartners;
    }

    @Override
    protected List<BusinessPartner> getFallback() {
        logger.warn("Fallback called because of exception:", getExecutionException());
        return Collections.emptyList();
    }
}

The GetBusinessPartnersCommand class inherits from ErpCommand, which is the SDK’s abstraction to provide easy-to-use Hystrix commands. To implement a valid ErpCommand we need to do two things: first, we need to provide a constructor. Here we simply add the default constructor. Second, we need to override the run() method. As you might have noticed already, we can simply use the VDM-based code we used to call our OData service from the previous blog post, and put it into the run() method. No changes needed!

Additionally, by overriding the getFallback() method, we can provide a fallback if the remote service call should fail. In this case we simply return an empty list after logging the exception that led to the fallback being called. We could also serve static data or check whether we have already cached a response to this call.

Hystrix comes with a default configuration, so you don’t need to perform any configuration on your own. In most cases the default configuration will suffice. However, if you need to change the configuration, you can find more information on this topic in Hystrix wiki.

In the following example we set the thread-pool size to 20 and the timeout to 10000 milliseconds.

protected GetBusinessPartnersCommand() {
    super(HystrixUtil
            .getDefaultErpCommandSetter(
                    GetBusinessPartnersCommand.class,
                    HystrixUtil.getDefaultErpCommandProperties().withExecutionTimeoutInMilliseconds(10000))
            .andThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter().withCoreSize(20)));
}

Now that we have a working command, we need to adapt our BusinessPartnerServlet:

./application/src/main/java/com/sap/cloud/sdk/tutorial/BusinessPartnerServlet.java

package com.sap.cloud.sdk.tutorial;

import com.google.gson.Gson;
import org.slf4j.Logger;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.List;

import com.sap.cloud.sdk.cloudplatform.logging.CloudLoggerFactory;

import com.sap.cloud.sdk.s4hana.datamodel.odata.namespaces.businesspartner.BusinessPartner;

@WebServlet("/businesspartners")
public class BusinessPartnerServlet extends HttpServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger logger = CloudLoggerFactory.getLogger(BusinessPartnerServlet.class);

    @Override
    protected void doGet(final HttpServletRequest request, final HttpServletResponse response)
            throws ServletException, IOException {
        try {
            final List<BusinessPartner> businessPartners =
                    new GetBusinessPartnersCommand().execute();

            response.setContentType("application/json");
            response.getWriter().write(new Gson().toJson(businessPartners));

        } catch (final Exception e) {
            logger.error(e.getMessage(), e);
            response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
            response.getWriter().write(e.getMessage());
        }
    }
}

Thanks to our new GetBusinessPartnersCommand, we can now simply create a new command and execute it. As before, we get a list of business partners as result. But now we can be sure that our application will not stop working all-together if the OData service is temporarily unavailable for any tenant.

Write Tests for the Hystrix command

There are two things we need to address in order to properly test our code: we need to provide our tests with an ERP endpoint and a Hystrix request context.

Credentials for SAP S/4HANA System

We provide the ERP destination as explained in the previous blog post as a systems.json or systems.yml file. In addition, you may provide a credentials.yml file to reuse your SAP S/4HANA login configuration as described in the Appendix of the previous tutorial step.

To provide credentials in this manner, create the following credentials.yml file in a safe location (e.g., like storing your ssh keys in ~/.ssh), i.e., not in the source code repository.

/secure/local/path/credentials.yml

---
credentials:
- alias: "ERP_TEST_SYSTEM"
  username: "user"
  password: "pass"

Afterwards you can pass the credentials.yml when running tests. Make sure to pass the absolute path to the file:

mvn test -Dtest.credentials=/secure/local/path/credentials.yml

Hystrix request context

Since Hystrix commands run in non-container-managed threads, they cannot use ThreadLocal to access request specific information. Therefore Hystrix provides a solution in the form of request contexts. Usually these are created by a servlet filter whenever a request is being made. Since there are no requests in our tests, we need a different way to provide a Hystrix request context.

The MockUtil class of the SAP S/4HANA Cloud SDK, introduced in the previous blog post, is our friend once again. Using MockUtil’s requestContextExecutor() method we can wrap the execution of the GetBusinessPartnersCommand in a request context.

Now let’s have a look at the code, to be placed in a file integration-tests/src/test/java/com/sap/cloud/sdk/tutorial/GetBusinessPartnersCommandTest.java:

package com.sap.cloud.sdk.tutorial;

import org.junit.Before;
import org.junit.Test;

import java.net.URI;
import java.util.Collections;
import java.util.List;

import com.sap.cloud.sdk.cloudplatform.servlet.Executable;
import com.sap.cloud.sdk.testutil.MockUtil;

import com.sap.cloud.sdk.s4hana.datamodel.odata.namespaces.businesspartner.BusinessPartner;

import static org.assertj.core.api.Assertions.assertThat;

public class GetBusinessPartnersCommandTest {

    private MockUtil mockUtil;

    @Before
    public void beforeClass() {
        mockUtil = new MockUtil();
        mockUtil.mockDefaults();
    }

    private List<BusinessPartner> getBusinessPartners() {
        return new GetBusinessPartnersCommand().execute();
    }

    @Test
    public void testWithSuccess() throws Exception {
        mockUtil.mockErpDestination();
        new RequestContextExecutor().execute(new Executable() {
            @Override
            public void execute() throws Exception {
                assertThat(getBusinessPartners()).isNotEmpty();
            }
        });
    }

    @Test
    public void testWithFallback() throws Exception {
        mockUtil.mockDestination("ErpQueryEndpoint", new URI("http://localhost"));
        new RequestContextExecutor().execute(new Executable() {
            @Override
            public void execute() throws Exception {
                assertThat(getBusinessPartners()).isEqualTo(Collections.emptyList());
            }
        });
    }

}

We use JUnit’s @Before annotation to setup a fresh MockUtil for each test. Then in the tests (@Test) we do the following: first we create a new request context using mockUtil.requestContextExecutor and provide it with a new Executable. Then we override the Executable’s execute() method, where we finally put the code that we actually want to test together with the corresponding assertions.

For testWithSuccess(), we use the default ERP destination information using mockUtil.mockErpDestination so that the endpoint points to the SAP S/4HANA system configured in your systems.json file. For the sake of simplicity we simply assert that the response is not empty.

For testWithFallback(), we intentionally provide a destination (localhost) that does not provide the called OData service in order to make the command fail. Since we implemented a fallback for our command that returns an empty list, we assert that we actually receive an empty list as response.

Simply run mvn clean install as in the previous tutorials to test and build your application. Consider the following before deploying to Cloud Foundry.

Deploy the application on SAP Cloud Platform Cloud Foundry

The SDK integrates the circuit breakers and other resilience features of Hystrix with the SAP Cloud Platform, specifically with the tenant and user information. For example, circuit breakers are maintained per tenant to ensure that they are properly isolated. As of now, our application is not multitenant as far as user authentication is concerned – we will come to this in the next steps of the tutorial series (see Step 7 for Cloud Foundry and Step 8 on Neo).

When running such a non-multitenant application with caching, the required tenant and user information needs to be supplied separately, or be mocked. The (local) Neo environment  handles this out-of-the-box and you do not need to do anything when developing on Neo. On Cloud Foundry, the SDK provides a workaround for testing purposes, but you need to enable this workaround explicitly for security considerations. To do so, follow the step outlined in the following, but be aware of the security implications explained below.

Supply your Cloud Foundry application with an additional environment variable as follows:

cf set-env firstapp ALLOW_MOCKED_AUTH_HEADER true

When the variable ALLOW_MOCKED_AUTH_HEADER is explicitly set to true, the SDK will fall back to providing mock tenant and user information when no actual tenant information is available. This setting must never be enabled in productive environments. It is only meant to make testing easier if you do not yet implement the authentication mechanisms of Cloud Foundry. Delete the environment variable as soon as it is no longer required, for example, because you implemented Step 7 of this tutorial series (cf unset-env firstapp ALLOW_MOCKED_AUTH_HEADER).

Afterwards, simply run mvn clean install as in the previous tutorials to test and build your application and then run cf push to deploy your updated application to CloudFoundry!

If you want to run your application locally with mvn tomee:run (in folder application/), you need to similarly set the environment variable ALLOW_MOCKED_AUTH_HEADER=true on your local machine before starting the local server, in addition to supplying the destinations (as described in Step 4).

 

This wraps up the tutorial on making our sample application resilient using Hystrix and the SAP S/4HANA Cloud SDK. Continue with the next tutorial Step 6: Caching and also explore other tutorial posts on topics like security!

To report this post you need to login first.

13 Comments

You must be Logged on to comment or reply to a post.

  1. SIMMACO FERRIERO

    Hi Ekaterina,

    regarding the final part of your blog, I’ve tried with the latest 1.1.2 version of the archetype, but it seems that the Tomee plugin for local testing is not included. I had to manually add the plugin in the pom.xml file of the application.

    However, even after this, the mvn tomee:run command seems to be working fine, but when I do http://localhost:8080/hello I get an error: it says “This localhost page can’t be found”.

    Do you have any tips?

    Simmaco

     

    (0) 
    1. Sander Wozniak

      Hi Simmaco,

      I’ve just created a project as follows:

      mvn archetype:generate -DarchetypeGroupId=com.sap.cloud.s4hana.archetypes -DarchetypeArtifactId=scp-cf-tomee -DarchetypeVersion=1.1.2

      Then, I build and run the project with:

      cd application/
      mvn clean package
      mvn tomee:run

      This is the plugin configuration that comes with the latest archetype:

      <plugin>
          <groupId>org.apache.openejb.maven</groupId>
          <artifactId>tomee-maven-plugin</artifactId>
          <version>1.7.4</version>
          <configuration>
              <context>ROOT</context>
              <libs>
                  <lib>org.apache.tomcat:tomcat-catalina:8.5.20</lib>
              </libs>
          </configuration>
      </plugin>

      Can you please share more details on your configuration?

      Thanks!
      Sander

      (0) 
  2. Former Member

    Hi Ekaterina,

    Thank you for posting this blog. When I run the application with command mvn clean install, It is failing with the integration tests notifying with that couldn’t resolve the URISyntaxException. Can you please help in solving this issue?

     

    Thanks,

    Sankeerth

     

    (0) 
    1. Ekaterina Kruse Post author

      Hello Sankeerth,

      did you execute the Steps 1-4 and did they work properly? What S/4HANA system do you use for the evaluation (URL in systems.yml)?

      Best regards,

      Ekaterina

      (0) 
      1. Former Member
        Thanks for the response Ekaterina. The issue is resolved when I added the line below. It is working fine now.
        
        import java.net.URISyntaxException;

         

         

        (0) 
  3. Apoorva Savvasere Krishnarao

    Hi Ekaterina,

    Thanks for the tutorial. When I run mvn clean install, I get an error as it cannot recognize HystrixUtil and HystrixThreadPoolProperties. Please let me know if I missed anything.

    Regards,

    Apoorva

    (0) 
    1. Alexander Duemont

      Hello Apoorva,

      can you please attach an excerpt from the error log? Is it failing during compilation (mvn compile) oder testing (mvn test)? can you check whether “com.sap.cloud.s4hana.frameworks:hystrix:jar:1.x.x:compile” is showing up in your mvn dependency:tree ?

      Best regards

      Alexander

      (0) 
      1. Apoorva Savvasere Krishnarao

        Hi Alexander,

        Thanks for your reply. It is failing during compilation stage.

        I checked the dependency hierarchy and hystrix:1.5.0 [compile] is included under “Resolved Dependencies”.

        Also, I tried adding the dependency manually in pom.xml:

        Please let me know how to resolve this issue. Thanks.

         

        Regards,

        Apoorva

        (0) 
        1. Alexander Duemont

          Hello Apoorva,

          Can you please check whether the following lines are part of the class imports of GetBusinessPartnersCommand.java ?

          import com.netflix.hystrix.HystrixThreadPoolProperties;
          import com.sap.cloud.sdk.frameworks.hystrix.HystrixUtil;

          Best regards

          Alexander

          (0) 
          1. Apoorva Savvasere Krishnarao

            Hi Alexander,

             

            Thank you. This worked. However, there is still a problem with json schema validation.

            It executes fine without any error and I can see “BUILD SUCCESS” only if I do not use Hystrix. It is creating a problem when I try to change the threadpool size and timeout using Hystrix.

            Regards,

            Apoorva

             

            (0) 
            1. Alexander Duemont

              Hello Apoorva,

              the timeout limit is reached while waiting for an ERP response. Either increase the previously configured 10 seconds to (e.g.) 30 seconds or preferably add the “.top(Numeric)” modifier to the query builder before executing. The latter will reduce the runtime significantly.

              final List<BusinessPartner> businessPartners =
                new DefaultBusinessPartnerService()
                  .getAllBusinessPartner()
                  .select(BusinessPartner.BUSINESS_PARTNER,
                    BusinessPartner.LAST_NAME,
                    BusinessPartner.FIRST_NAME,
                    BusinessPartner.IS_MALE,
                    BusinessPartner.IS_FEMALE,
                    BusinessPartner.CREATION_DATE)
                  .filter(BusinessPartner.BUSINESS_PARTNER_CATEGORY.eq(CATEGORY_PERSON))
                  .orderBy(BusinessPartner.LAST_NAME, Order.ASC)
                        
                  .top(10) // <----- only take 10 items as result
                        
                  .execute();

              Best regards
              Alexander

              (0) 

Leave a Reply