SAP Commerce Cloud (previously known as SAP Hybris Commerce) is often used as headless service to be used with SAP Spartacus or third-party storefront solutions. The communication between the storefront and the platform happens via the SAP Commerce REST interface called Omni-Channel Connect (OCC). Additionally, SAP Commerce Cloud provides a set of APIs for integration (Integration APIs), content and product management (CMS and Product Management APIs).

All these APIs are highly customizable, and even smallest changes in configuration, data model or Java code may result in performance degradation. Performance testing is especially important in cases where concurrency may be the source of the problem. To recognize and fix such issues timely, it is recommended to perform regular performance testing, ideally integrated into the build and deployment pipelines.

However, SAP Commerce Cloud intensively uses caching and the test plans may not reflect the true situation if not designed properly. Almost all database queries are cached inside the platform so that the thousands of data fetching operations (per a webservice call)  don’t even reach the database. If a cache was disabled, an average single call would take minutes. Fortunately, the cache is always on, but depending on many factors it may or may not work efficiently.

The test scenarios should be designed in such a way that they mimic the real-time user action happening in the production. If your product catalog has thousands of products, and you expect a normal distribution, it will be a bad idea to test the system against just a small bunch of test products. You may face the risk that Commerce Cloud demonstrates exceptional performance with this small bunch of products, but completely different results with the actual traffic.

In order to mimic user behavior, API sample calls can be either generated or collected from the logs. For the APIs, this dataset has a combination of when and what: a timestamp, a list of URLs, an HTTP method, a set of HTTP headers, GET and POST payloads. It is often easier to simplify the model by firing the calls in some pattern reflecting the actual traffic in its average approximations.

In my project, I needed to generate a list of URLs and JSON payloads to get them close to what system will experience after go-live. In my test plan, I needed to read the POST payload from a file and send it as a body for a POST request to the REST API, parse the response, and log the response time along with a response code and webservice response body. I also needed to control the pattern of the load and visualize the results.

For example, you have 10000 products, and you need to load test the product details API (customer facing) and product stock update API (used by the internal systems). For both, you need to inject a product code to the URL (in case of product details API) or in the JSON (for the stock update).

What tool are you using to create such a load for regular checking the performance? I worked with three:

• Apache JMeter (Java)
• Tsung (Erlang)
• Custom-made (Python, asyncio+matplotlib).

In this article, I explain how to use JMeter for the SAP Commerce API and storefront testing and what challenges you may come across along this way. 

This article is part of a series on performance testing:

• Part 1. SAP Commerce Cloud Performance Testing Tools: Apache JMeter
• Part 2. Performance Testing Tools: Tsung
• Part 3. Performance Scripting: Writing Load Generators from Scratch

 

Apache JMeter

Apache JMeter is one of the most popular and oldest open source tools for performance and load testing. It is normally used in the performance testing, but I believe this tools should be used by the developers too to catch the issues at the earliest and deliver better code. 

JMeter can be configured to send different combinations of unique URLs with the unique POST payloads per URL. In the standard JMeter GUI, you can only specify how many users you want to use during the test and how fast they should come as well as specify the URL (with a payload) for generating a load. However, functions and variables allow you to implement complex scenarios.

In order to inject dynamic content to the HTTP Request sampler, you should use the variables:

${__FileToString(${__eval(${JSONFILE})},,)} extracts the contents of the file which name is in the JSONFILE variable and use it as a POST payload.

JSONFILE is a variable set by CSV Data Set Config Element configured as follows:

If you need to use more than one parameter, you can list them in the LIST_OF_JSONS.txt file specified in the Filename section.

For the test above, this file contains a list of full paths to the JSONs.

If we need to add a new parameter to use it in the URL together with the JSONs, just add a new column:

/Users/raaliev/perf/dev_out2/10.json,10
/Users/raaliev/perf/dev_out2/100.json,20
/Users/raaliev/perf/dev_out2/101.json,30
/Users/raaliev/perf/dev_out2/102.json,40

Then add the mapping into the CSV Data Set Config’s Variable Names:

JSONFILE,NO

And use the “NO” variable in the HTTP Request sampler:

Path: /${NO}/post

In this example, we used a __FileToString function. This function allows you to read the file contents from the specifed file which is a variable in our example.

Another useful function is __StringFromFile which is basically the same as CSV Data Set Config, but the CSV Data Set Config element is limited to one file only. Using the function you can read multiple files. If the end of the file is reached, the file is rotated.

For testing Commerce Cloud APIs, you will need to add HTTP Header Manager:

and configure HTTP Authorization Manager:

If username and password are supposed to be different for different calls, you can use __StringFromFile to extract the login and password from the CSV file.

For the OAuth-protected endpoints, we need to extract the authorization token and pass it into the next request.  

First of all, we need to add one more “HTTP Request” sampler and configure it accordingly:

The response is a JSON structure with access_token in it. Then we need to extract the access token. We need to add Regular Expression Extractor as under the token HTTP request element:

The access_token variable is used for the subsequent calls:

The similar approach can be used for the storefront pages accessible only for the authenticated users. You need to

• create a call to the login form, extract CSRF token (HTTP Request + Header Manager + Regular Expression Extractor)

• create a call to Spring’s /<context>/j_spring_security_check with j_username, j_password and a CSRF token extracted from the previous call and extract the JSESSIONID cookie from the response. A Content-Type header should be set to application/x-www-form-urlencoded.
• create a call to a target page using JSESSIONID cookie

For checking Add To Cart for an anonymous user you need only CSRF extraction phase and populate the CSRF parameter of the add to cart form with the extracted value.  

For API testing, you may find it useful to have multiple thread groups. The first one is responsible for a setup, authentication and creating a user shopping cart. The last one is responsible for retriving the results. In between, you can use a loop where hundreds of products are placed in the cart with the specified rate.

The variables are designed to be local within a thread group. To pass them between different thread groups (such as setup and add-to-cart), you can use BeanShell Assertion to save the variable as a global property.

The sample scenario for testing the shopping cart functionality may look as follows:

If you need to inject a random ID to bypass caching, you can use a __UUID or __RandomString function.

To use JMeter for testing of storefront pages, you need to keep a session up during requests. To implement that, you need to include HTTP Cookie Manager config element. If you need to simulate user login, you need to include HTTP Request pointing at the spring security URL /j_spring_security_check). For the API testing, you need to extract a token to use it in the subsequent queries. 

For the SAP Commerce Cloud Storefront Accelerator, the plans are provided by SAP:

bin/ext-template/yacceleratortest/resources/jmeter/B2BAcceleratorTestPlan.jmx
bin/ext-template/yacceleratortest/resources/jmeter/MarketplaceAcceleratorTestPlan.jmx
bin/ext-template/yacceleratortest/resources/jmeter/AcceleratorTestPlan.jmx

In this article, I explained how the original task can be solved with JMeter.

However, JMeter is far not as fast as Tsung and far not as flexible as Python. I am going to share the details about these two very soon. 

Stay tuned!