SAP Commerce Cloud, previously known as SAP Hybris Commerce, is often used as a headless service with SAP Spartacus or third-party storefront solutions. Communication between the storefront and the platform happens via the SAP Commerce REST interface called Omni-Channel Connect (OCC). Additionally, SAP Commerce Cloud provides a set of APIs for integration (Integration APIs), content management, and product management (CMS and Product Management APIs).

All these APIs are highly customizable, and even the smallest changes in configuration, data model, or Java code may result in performance degradation. Performance testing is especially important in cases where concurrency may be the source of the problem. To recognize and fix such issues in time, it is recommended to perform regular performance testing, ideally integrated into the build and deployment pipelines.

However, SAP Commerce Cloud uses caching intensively, and test plans may not reflect the true situation if they are not designed properly. Almost all database queries are cached inside the platform, so thousands of data-fetching operations per web service call do not even reach the database. If the cache were disabled, an average single call would take minutes. Fortunately, the cache is always on, but depending on many factors, it may or may not work efficiently.

Test scenarios should be designed in such a way that they mimic real-time user actions happening in production. If your product catalog has thousands of products and you expect a normal distribution, it is a bad idea to test the system against just a small bunch of test products. You may face the risk that Commerce Cloud demonstrates exceptional performance with this small bunch of products, but completely different results with actual traffic.

To mimic user behavior, API sample calls can be either generated or collected from the logs. For APIs, this dataset has a combination of when and what: a timestamp, a list of URLs, an HTTP method, a set of HTTP headers, and GET and POST payloads. It is often easier to simplify the model by firing the calls in a pattern reflecting actual traffic in its average approximations.

In my project, I needed to generate a list of URLs and JSON payloads to get them close to what the system would experience after go-live. In my test plan, I needed to read the POST payload from a file and send it as a body for a POST request to the REST API, parse the response, and log the response time along with a response code and web service response body. I also needed to control the load pattern and visualize the results.

For example, you have 10,000 products, and you need to load-test the product details API, which is customer-facing, and the product stock update API, which is used by internal systems. For both, you need to inject a product code into the URL, in the case of the product details API, or into the JSON, for the stock update.

What tool are you using to create such a load for regular performance checks? I worked with three:

• Apache JMeter (Java)
• Tsung (Erlang)
• Custom-made tools (Python, asyncio + matplotlib)

In this article, I explain how to use JMeter for SAP Commerce API and storefront testing, and what challenges you may come across along the way.

This article is part of a series on performance testing:

• Part 1. SAP Commerce Cloud Performance Testing Tools: Apache JMeter
• Part 2. Performance Testing Tools: Tsung
• Part 3. Performance Scripting: Writing Load Generators from Scratch

Apache JMeter

Apache JMeter is one of the most popular and oldest open-source tools for performance and load testing. It is normally used for performance testing, but I believe this tool should be used by developers too, to catch issues as early as possible and deliver better code.

JMeter can be configured to send different combinations of unique URLs with unique POST payloads per URL. In the standard JMeter GUI, you can only specify how many users you want to use during the test, how fast they should come, and the URL with a payload for generating the load. However, functions and variables allow you to implement complex scenarios.

To inject dynamic content into the HTTP Request sampler, you should use variables:

${__FileToString(${__eval(${JSONFILE})},,)} extracts the contents of the file whose name is in the JSONFILE variable and uses it as a POST payload.

JSONFILE is a variable set by the CSV Data Set Config element, configured as follows:

If you need to use more than one parameter, you can list them in the LIST_OF_JSONS.txt file specified in the Filename section.

For the test above, this file contains a list of full paths to the JSON files.

If we need to add a new parameter to use it in the URL together with the JSON files, just add a new column:

/Users/raaliev/perf/dev_out2/10.json,10
/Users/raaliev/perf/dev_out2/100.json,20
/Users/raaliev/perf/dev_out2/101.json,30
/Users/raaliev/perf/dev_out2/102.json,40

Then add the mapping into the CSV Data Set Config’s Variable Names:

JSONFILE,NO

And use the NO variable in the HTTP Request sampler:

Path: /${NO}/post

In this example, we used the __FileToString function. This function allows you to read the file contents from the specified file, which is a variable in our example.

Another useful function is __StringFromFile, which is basically the same as CSV Data Set Config, but the CSV Data Set Config element is limited to one file only. Using the function, you can read multiple files. If the end of the file is reached, the file is rotated.

For testing Commerce Cloud APIs, you will need to add an HTTP Header Manager:

and configure the HTTP Authorization Manager:

If the username and password are supposed to be different for different calls, you can use __StringFromFile to extract the login and password from the CSV file.

For OAuth-protected endpoints, we need to extract the authorization token and pass it into the next request.

First of all, we need to add one more “HTTP Request” sampler and configure it accordingly:

The response is a JSON structure with access_token in it. Then we need to extract the access token. We need to add Regular Expression Extractor under the token HTTP request element:

The access_token variable is used for the subsequent calls:

A similar approach can be used for storefront pages accessible only to authenticated users. You need to:

• create a call to the login form and extract the CSRF token (HTTP Request + Header Manager + Regular Expression Extractor)

• create a call to Spring’s /<context>/j_spring_security_check with j_username, j_password, and a CSRF token extracted from the previous call, and extract the JSESSIONID cookie from the response. A Content-Type header should be set to application/x-www-form-urlencoded.
• create a call to a target page using the JSESSIONID cookie.

For checking Add To Cart for an anonymous user, you need only the CSRF extraction phase and to populate the CSRF parameter of the add-to-cart form with the extracted value.

For API testing, you may find it useful to have multiple thread groups. The first one is responsible for setup, authentication, and creating a user shopping cart. The last one is responsible for retrieving the results. In between, you can use a loop where hundreds of products are placed in the cart at the specified rate.

Variables are designed to be local within a thread group. To pass them between different thread groups, such as setup and add-to-cart, you can use BeanShell Assertion to save the variable as a global property.

The sample scenario for testing shopping cart functionality may look as follows:

If you need to inject a random ID to bypass caching, you can use the __UUID or __RandomString function.

To use JMeter for testing storefront pages, you need to keep a session alive during requests. To implement that, you need to include the HTTP Cookie Manager config element. If you need to simulate user login, you need to include an HTTP Request pointing at the Spring Security URL /j_spring_security_check. For API testing, you need to extract a token to use it in subsequent queries.

For the SAP Commerce Cloud Storefront Accelerator, the plans are provided by SAP:

bin/ext-template/yacceleratortest/resources/jmeter/B2BAcceleratorTestPlan.jmx
bin/ext-template/yacceleratortest/resources/jmeter/MarketplaceAcceleratorTestPlan.jmx
bin/ext-template/yacceleratortest/resources/jmeter/AcceleratorTestPlan.jmx

In this article, I explained how the original task can be solved with JMeter.

However, JMeter is not nearly as fast as Tsung and not nearly as flexible as Python. I am going to share the details about these two very soon.

Stay tuned!