Shopping Cart in SAP Commerce: Performance, Architecture, Pitfalls, Testing

A shopping cart is the centerpiece of almost every e-commerce website.

In fact, this component is the first thing that makes an e-commerce store different from just an online catalog. Working with issues and challenges related to the cart is something you’ll never forget.

The Russian novelist Leo Tolstoy, at the opening of his novel Anna Karenina, writes, “All happy families resemble one another, but each unhappy family is unhappy in its own way.” You know, the diversity of cart-related issues I have observed in SAP Commerce projects made me think that Tolstoy meant developers as well. The diversity of issues and challenges in this component is so vast that every case is different. However, when it comes to building or optimizing shopping cart processes, understanding the limitations, pitfalls, and peculiarities is a huge plus.

Let’s have a look at how a shopping cart works in SAP Commerce under the hood and how to avoid making mistakes in shopping cart customization.

From the customer’s perspective, there are just four use cases:

• View shopping cart items, such as products or services
• Add an item to a cart
• Change the quantity of an item
  • Remove an item from a cart, which is basically a special case of changing quantity

For the sake of simplicity, I am not touching extended cart functionality such as product recommendations, wishlists, B2B quotes, bulk add-to-cart, etc. Why complicate things if even the basic functionality presents a lot of challenges? Let’s start with the basic operations and have a deeper look at them.

Add To Cart

An anonymous or registered user chooses a product they wish to buy and clicks the “Add to cart” button to send the product to the shopping cart. After that, if a user decides to open the shopping cart page, the items they have chosen to buy will be displayed.

Implementation Details

What out-of-the-box SAP Commerce Accelerator does under the hood when a user attempts to add a product to a cart:

• The controller (AddToCartController.addToCart) receives a product code and a quantity
• The controller delegates processing to the facade layer (CartFacade.addToCart)
• The facade layer delegates processing to the Commerce Cart Service (CommerceCartService.addToCart)
• Commerce Cart Services delegates processing to the Commerce Add To Cart Strategy (DefaultCommerceAddToCartStrategy.addToCart) or some other strategies coming from optional modules — see the diagram below

The DefaultCommerceAddToCartStrategy is the default implementation.

A Selective Cart feature, also known as “save for later,” extends the default implementation. It allows customers to select which items they wish to purchase and leave other items in the cart for future consideration. This improves the shopping experience and increases the conversion rate.

All the magic happens inside the strategy, which performs the following steps:

• Hooks: beforeAddToCart. Executes Commerce AddToCartMethod Hooks one by one (beforeAddToCart)

The default configuration contains five hooks. Remove whitespaces to get the class names:

• Configurable Product Add To Cart Method Hook*
• If you use the Configurable Bundle module
  • Bundle Add To Cart Method Hook
  • Bundle Selection Criteria Add To Cart Method Hook
• If you use the Subscription Services module
  • Subscription Add To Cart Method Hook
• etc.

It is important to remember that, unlike Cart Calculation, a default add-to-cart strategy is non-transactional. So, any changes made by before- and after-add-to-cart hooks won’t be automatically reverted if their code, or any code following them, throws an exception.

Each of these hooks provides a beforeAddToCart implementation, which may or may not be empty in the default configuration. The hooks can be disabled via configuration (commerceservices.commerceaddtocartmethodhook.enabled=false).

The order in which the hooks are executed is not strictly defined. If your logic depends on the order of hooks, you risk facing hard-to-detect and non-reproducible errors.

If a hook throws an exception, all further hooks won’t be executed. Also, if that happens in the before*Hook, its counterpart, an after*Hook, won’t be triggered.

[!] Avoid changing the cart state within a before*Hook when those changes are supposed to be reverted in the after*Hook. These changes may not be reverted properly.

[!] Check whether your cart-related hook classes throw exceptions and whether you need them executed in a predefined order.

[!] Check what hooks are used in your configuration. You may be surprised to find that your list contains some non-relevant hooks. Some SAP Commerce extensions inject hooks automatically.

• Adjusts product quantity. If a new quantity, together with the expected amount, exceeds what is in stock, the system will attempt to perform a partial addToCart.

[!] This functionality indirectly exposes product stock levels to a customer. A user can attempt to add, say, ten million items just to challenge your system, and the number of items that will actually be added will reflect or correlate with the number of items available in stock. You probably don’t want to expose this information. If so, you need to add validation to the quantity parameter to compare what comes from the frontend against the maximum allowed quantity for your store. Alternatively, you can use the product max order quantity to set the maximum allowed to be added per product code.

• Validating parameters. A quantity must be a positive non-zero value. A product you are attempting to add should be a variant product, not a base product. Additionally, you can define your own validators. Look for addToCartValidators.
• Creating or updating a cart entry. A new cart entry is created or, if the specified product code is already in the cart, the quantity is updated.
• Merging same-code entries. It is used to handle adding products that already exist in the cart.

The cart is set to a “not calculated” state (isCalculated=false). This is a kind of message to the cart calculation strategy to recalculate the cart with the updated items.

• Calculate Cart via Commerce Cart Calculation Strategy. Recalculates a cart when isCalculated is set to false, which happens if a cart is changed with AddToCart.

[!] Please remember that for the JSP frontend, out of the box, this call to calculate cart may be a second call per page load. The first one may be triggered even before the page controller starts working, in Spring MVC filters.

Creating a shopping cart object

A cart should be created only with the first valid add-to-cart. This is the default implementation.

A common mistake is creating a cart with the first page load.

Corner cases to consider for Add To Cart

• Product is not in the catalog at all, for a selected country, for example
• Product is not available
• Product does not have a price
• Product is already in the cart
• Type of product is not purchasable
• Different variation of a product is already in the cart
• Specified product quantity may be too big for a single add-to-cart
• A product is attempted to be added to a non-existent cart, removed by the cleanup job

Test Cases

• Customer can add a product to a shopping cart.
• Customer can add the same product multiple times.
• Customer can change their quantity directly in the cart.
• Customer can add different variations, such as color, size, etc., of the same product.
• Shopping cart totals are updated when a user adds a product to the cart.
• Shopping cart totals are updated when a user changes quantity or removes a product from the cart.
• Customer can’t add a product not in stock, not available, or with negative stock.
• Customer can’t add a product not in the database.
• Customer can’t add a product with a zero price.
• Customer can’t add a product with a negative price.
• Customer can’t add a product without a price row.
• Customer can’t specify too large a quantity.
• Customer can’t specify a zero quantity.
• Customer can’t add products to other customers’ carts by cart ID.
• Customer can’t add a product to a cart recently removed by the cleanup job.

Cart Calculation

Cart Calculation is performed when a cart is in the “not calculated” state.

This component is rather complex under the hood. It is used by many components, and any customizations may impact other functionalities. For example, storeSessionService.setCurrentCurrency includes cart recalculation as part of the currency change logic.

Cart Calculation is heavily used by other components:

For example, in the CommerceDeliveryModeValidationStrategy, calculateCart is called up to three times per page load.

Additionally, cart calculation may be triggered from the Cart Restoration Filter even before the controller starts working. This may add a fourth call per page load.

[!] Check your Dynatrace reports — you will see that cart-related SQL queries are among the slowest, often together with media-related queries, stock level queries, and price rows, and cart-related modification SQL statements are very likely ranked as the slowest.

So you can see that cart calculation, as well as recalculation, is very important from a performance perspective, and all changes that may affect this component should be carefully planned and tested.

There are two implementations for CommerceCartCalculationStrategy available out of the box:

• DefaultCommerceCartCalculationStrategy — a default strategy to calculate the cart when not in checkout status. This implementation supports rollback.
  • If cart requires calculation
    • Rollback-if-not-successful (beforeCalculate -> calculationService.calculate -> updating promos -> afterCalculate) -> calculate external taxes
• NonTransactionalCommerceCartCalculationStrategy — a non-transactional strategy to calculate the cart when not in checkout status:
  • If cart requires calculation
    • beforeCalculate -> calculationService.calculate -> updating promos -> afterCalculate -> calculate external taxes

Transactions help revert the changes made by Before/After Calculate Hooks, Calculation Service, and Update Promotions. If something goes wrong and one of these ends with an error, the cart won’t be affected.

Internally, it runs the hooks, calculates the totals, and applies promotions and discounts.

• Before-Calculation Hooks
  • Commerce Cart Calculation Hooks
  • If calculation is required (isCalculated == false?)
• Calculating cart entries
• Calculating cart totals
  • Calculating discounts
  • Calculating taxes, built in, not an external service
• Updating promotions
  • Not thread-safe; uses per-session locks and synchronized methods
  • Uses Drools for calculating promotions; initializes Drools session -> calculates -> applies -> destroys Drools session
• Calculating the taxes via an external service (if configured)
• After-Calculation Hooks
  • Commerce Cart Calculation Hooks
    • commerceQuoteCartCalculationMethodHook is used here even if you don’t use quotes in your shopping cart

Make sure that your before- and after-calculation hooks are not supposed to be run in a predefined order. Also make sure that before-calculation hooks don’t make any changes that are supposed to be reverted in after-calculation hooks, especially if those changes may be in conflict with any modules outside the code between before- and after-calculation hooks.

[!] If you use an external tax calculation provider, use caching to reduce the number of service calls.

[!] If you use an external tax calculation provider, implement a circuit breaker pattern. The effect of an extremely slow response from a tax service or a timeout error is an increase in the number of concurrent active requests in the system.

[!] If you use an external tax calculation provider, you have a critical dependency. Implement a feature switch to disable all add-to-cart buttons if something bad happens with the external tax calculation service.

[!] Any customizations to the promotion engine, as well as any customizations related to promotion rules, may negatively affect cart calculation. Design and test thoroughly!

[!] Restrict the number of cart entries.

[!] Check interceptors, both type and controllers, before-view and before-controller handlers, and Spring AOP definitions related to cart services. These are often time bombs that heavily contribute to performance issues.

For example, Selective Cart Split List Addon introduces a before-controller handler, CartPageBeforeControllerHandler. It updates the wishlists and carts before the add-to-cart controller starts working.

Normally, SAP Commerce OOTB handles all the corner cases well, but your customizations may affect the standard behavior.

Updating Promotions

Updating Promotions is performed by PromotionEngineService, which replaced the legacy PromotionService many years ago.

It is important to know that promotion calculation is not thread-safe, so at any moment in time, only one calculation is performed per node in the cluster.

Also, it is important to know that in SAP Commerce, promotions are calculated each time the cart is calculated. See the diagram above showing how many consumers of cart calculation SAP Commerce has. It happens not only when a cart is requested but even after a product is added to the cart.

[!] The promotion engine is not thread-safe; the system can’t calculate promotions for more than one cart or order at a time per node. It has been sort of “running on empty” in many scenarios, so any customizations or added complexity in promotions may result in slowdowns and performance issues.

Promotion calculation is not cacheable in SAP Commerce. Partly, that is because the promotion mechanics may involve almost everything from the current time to the customer location. The cache key is hard to make consistent. If you calculate promotions three times per call, the whole process will be repeated three times. For example, when I worked with hybris Travel Accelerator in 2017, I found out that its code performed tens of promotion engine calls per customer web request, which took about 30 times longer than the out-of-the-box electronics demo store normally took for cart calculation.

The Drools facts, populated via RAO objects, are formed by the RAO providers. There are order-level and product-level RAO providers. Each provider contributes to the set of Drools facts. For example, CartRAOProvider performs expansion defined by cartRAOProviderValidOptions:

For example, EXPAND_CATEGORIES means that all product categories associated with all products in the cart will be added to the Drools fact list. So, if you have 100 products in the cart, and each product has 20 categories, you may end up with 100*20=2000 items in the facts registry — only for categories.

Unfortunately, the number of facts in the registry is often not controlled by the development team. Technically, Drools is capable of working with thousands and tens of thousands of facts, but the more facts you have, the slower the process will be.

• In SAP Commerce, Drools is not thread-safe, so hybris can perform only one promotion evaluation process at a time.
• In SAP Commerce, Drools is used in stateful mode, and initialization and destruction of the data structures and services are performed every calculation.
• In SAP Commerce, promotions are calculated often, even when not required.

Corner Cases

Corner cases for cart calculation:

• The external tax service is not reachable.
• The product availability dropped since the last operation — not below the ordered quantity.
• The product availability dropped since the last operation below the ordered quantity.
• The price changes since the last operation.
• Too many unique products are in the cart: hundreds or thousands — a large number of cart entries.
• Too many non-unique products are in the cart: hundreds or thousands — a large total number of items.
• Same-name different-SKU products.
• Cart or Cart Entries were removed by the cleanup job but requested by a customer.

Also check for:

• Unnecessary calculations and recalculations are not performed.
• Unnecessary external tax calculation calls are not performed.

Look at the following edge cases and typical solutions for them:

• Product availability dropped since the last time a user interacted with the cart — from X to Y, where Y is below the ordered quantity and Y is not zero.
  • We need to inform a user that we can’t deliver the ordered amount. Just reducing the quantity silently and automatically, as implemented in SAP Commerce Accelerator, is probably not a good idea for businesses where the quantity may matter.
• Product availability dropped since the last time a user interacted with the cart — from X to zero.
  • Just removing a product, as implemented in SAP Commerce Accelerator, may not be a good option. Marking the option as unavailable and recommending the closest substitute product seems much better.

Test Cases

• Products are displayed correctly: names, images, prices, discounts, etc.
• The product titles are clickable.
• The links lead to corresponding pages with the product info. Promotions, vouchers, and discounts are automatically accounted for in totals.
• If a product becomes unavailable, it should be automatically removed from the shopping cart. A customer should be notified about the change and the reason for the change.

Corner cases for “Quantity is changed” and “A cart entry is removed”:

• If all products are removed, manually or automatically, the shopping cart totals should be zero.
• If there is no entry with the given number, the system should return an error.
• User can’t change quantity in other users’ carts.
• If a product becomes unavailable, it should be automatically removed from the shopping cart. A customer should be notified about the change and the reason for the change.

Cart Restoration Strategies

From the customer’s perspective, having extra items in the cart after logging in may be confusing and undesired, especially if it happens during the checkout process and especially if the shopping carts are large and complex.

There are five possible solutions, not all supported by SAP Commerce OOTB:

• Non-interactive
  • “Session cart priority.” If a customer has an anonymous cart, the account-linked shopping cart will be ignored after login. If an anonymous cart is empty, the shopping cart contains the items from the account-linked cart.
  • “Cart Archive”: Replacing the old cart with the latest cart; moving the contents of the old cart to the wish list or archived carts list and informing the user about the actions done.
  • “Merge.” Combine the shopping carts together so the user still has a single shopping cart, the merged cart replaces the old account-linked cart, and the user is informed about the changes.
  • “Multiple carts”: Merging carts by default, but letting the customer undo the operation and restore one of two saved carts, “anonymous/guest” and “old account-linked.”
• Interactive:
  • “Interactive” Explaining the situation to the customer and letting them decide what they want to do, namely which cart to keep or whether they want to merge them:
    • You have items in your present cart. But you also have items stored in a previous cart. Would you like to:
      • with your present items
      • your saved items into your present purchase

In SAP Commerce, the default strategy is “restoring an account-linked cart only if the anonymous cart is empty,” which is titled as “Merge” above. SAP Commerce ignores an account-linked cart if a session cart is present.

The alternative SAP Commerce out-of-the-box strategy is about merging the account-linked and anonymous carts so that the resulting account-linked cart contains all their unique items combined together.

See /2019/02/24/merging-carts-when-a-customer-logs-in-problems-solutions-and-recommendations/

Shopping Cart via Polyglot Persistence

In v1905, SAP introduced Polyglot Persistence, a feature that was supposed to help relieve the load on the main database or provide dedicated storage for some resource-intensive data, such as shopping carts.

To make it possible, developers need to optimize the data structure and its related types as a single composed structure, or document.

Polyglot Persistence offers a default implementation for the Cart type, the ydocumentcart extension template. Currently, ydocumentcart supports Azure SQL Server, HSQL, and MySQL.

Additionally, there are transaction management and caching subsystems, which allow you to cache all modifications performed on a single item and flush them to persistent storage when the main operation ends. Reading is realized through a query language similar to FlexibleSearch.

I am not aware of any implementations where Polyglot Persistence would be leveraged in any way, particularly for Carts. Experimenting with it is on my bucket list. Stay tuned!

Monitoring and Alerting

In order to keep shopping cart performance under control, I recommend:

• Collecting the following metrics and having a dashboard to enable near real-time visual tracking.
• Installing and configuring an alerting system to get notified if any of the metrics fall outside the defined range.

The following metrics are useful and easy to collect:

• Number of add-to-carts per time unit.
• Number of sessions where at least one item is added to the shopping cart / total number of sessions — per time unit.
• Average latency per time unit — add-to-cart.
• Average latency per time unit — view-cart.
• Average response time for external tax calculation, if present, per time unit.
• Number of errors related to external tax calculation, if present, per time unit.

Cart Clean-up Processes

A large number of carts not being used by customers may make your system sluggish and bloated. Since the carts and cartentries tables are fast-growing, it is important to clean them up regularly.

There is a cart data retention cronjob available in SAP Commerce called OldCartRemovalCronJob. It comes with the commercewebservices extension, which exposes part of Commerce Facades as REST-based web services. Make sure that this extension is included in your configuration if you need it.

[!] Make sure that an OldCartRemovalCronJob is included in your configuration and enabled in your instance.

The job removes carts, together with their cart entries, older than the specified time in seconds provided in the configuration. The default values are 14 days for anonymous carts and 1 month for anonymous carts.