Shopping Cart in SAP Commerce: Performance, Architecture, Pitfalls, Testing

A shopping cart is the centerpiece of almost every e-commerce website.

In fact, this component is the first thing that makes an e-commerce store different from just an online catalog. Working with issues and challenges related to the cart is something you’ll never forget. 

The Russian novelist Leo Tolstoy, at the opening of his novel Anna Karenina, writes ‘All Happy families resemble one another, but each unhappy family is unhappy in its own way.’ You know, the diversity of cart-related issues I have been observing in the SAP Commerce projects, made me think that Tolstoy meant the developers as well. The diversity of the issues and challenges in this component is so vast that every case is different. However, when it comes to building or optimizing the shopping cart processes, understanding the limitations,  pitfalls, and peculiarities is a huge plus. 

Let’s have a look at how a shopping cart works in SAP Commerce under the hood and how to avoid making mistakes in shopping cart customization.

From the customer perspective there are just four use cases:

• View Shopping Cart items (products or services)
• Add am item to a cart
• Change quantity of an item
  • Remove an item from a cart (which is basically a special case of Change Quantity)

For the sake of simplicity, I am not touching extended cart functionality such as product recommendations, wishlists, B2B quotes, bulk add-to-cart, etc. Why complicate things if even the basic functionality gives a lot of challenges! Let’s start with the basic operations and have a deeper look at them. 

Add To Cart

An anonymous or registered user chooses a product they wish to buy and clicks the ‘Add to cart’ button to send their product to the shopping cart. After that, if a user decides to open a shopping cart page, the items they have chosen to buy will be displayed.

Implementation Details

What out-of-the-box SAP Commerce Accelerator does under the hood when a user attempts to add a product to a cart:

• The controller (AddToCartController.addToCart) receives a product code and a quantity
• The controller delegates processing to the facade layer (CartFacade.addToCart)
• The facade layer delegates processing to the Commerce Cart Service (CommerceCartService.addToCart)
• Commerce Cart Services delegates processing to the Commerce Add To Cart Strategy (DefaultCommerceAddToCartStrategy.addToCart) (or some other strategies coming from optional modules – see the diagram below)

 

The DefaultCommerceAddToCartStrategy is a default implementation. 

A Selective Cart feature (also known as “save for later”) extends the default implementation. It allows customers to select which items they wish to purchase and to leave other items in the cart for future consideration. This improves the shopping experience and increases the conversion rate.

All magic happens inside the strategy which performs the following steps:

• Hooks: beforeAddToCart. Executes Commerce AddToCartMethod Hooks one by one (“beforeAddToCart”)

 

The default configuration contains five hooks (remove whitespaces to get the class names):

• Configurable Product Add To Cart Method Hook*
• If you use Configurable Bundle module 
  • Bundle Add To Cart Method Hook
  • Bundle Selection Criteria Add To Cart Method Hook
• If you use Subscription Services module
  • Subscription Add To Cart Method Hook
• etc.

 

It is important to remember that, unlike Cart Calculation, a default add to cart strategy comes non-transactional. So, any changes made by before- and after add-to-cart hooks won’t be automatically reverted if their code or any code following them throws an exception. 

Each of these hooks provides beforeAddToCart implementation which may or may not be empty in the default configuration. The hooks can be disabled via configuration (commerceservices.commerceaddtocartmethodhook.enabled=false).

The order in which the hooks are executed is not strictly defined. If your logic depends on the order of hooks, you risk facing hard-to-detect and non-reproducible errors. 

If a hook throws an exception, all further hooks won’t be executed. Also, if that happens in the before*Hook, its counterpart, an after*Hook won’t be triggered.  

[!] Avoid changing a cart state within before*Hook which is supposed to be revered in the after*Hook. These changes may not be reverted properly.    

[!] Check whether your cart-related hook classes throw exceptions and whether you need them executed in some predefined order.  

[!] Check what hooks are used in your configuration. You may be surprised to know that your list contains some non-relevant hooks. Some of SAP Commerce extensions inject the hooks automatically. 

• Adjusts product quantity. If a new quantity (together with the expected amount) exceeds what is in stock, the system will attempt to perform a partial addToCart.

[!] This functionality indirectly exposes product stock levels to a customer. A user can attempt to add, say, ten million items, just to challenge your system, and the number of items that will be actually added will reflect or correlate with the number of items available in stock. Probably you don’t want to expose this information. If so, you need to add validation to the quantity parameter which would compare what comes from the frontend against the maximum allowed quantity for your store. Alternatively, you can use product max order quantity to set the maximum allowed to be added per product code.

• Validating parameters. A quantity must be a positive non-zero value. A product you are attempting to add should be a variant product, not a base product. Additionally, you can define your own validators (look for addToCartValidators). 
• Creating or updating a cart entry. A new cart entry is created or, if the specified product code is already in the cart, the quantity is updated. 
• Merging same-code entries. It is used to handle adding the products already existing in the cart. 

The cart is set to “not calculated” state (isCalculated=false). This is a kind of a message to the cart calculation strategy to perform re-calculating the cart with the updated items. 

• Calculate Cart via Commerce Cart Calculation Strategy. Recalculates a cart when isCalculated is set to False (what happens if a cart is changed with AddToCart).

[!] Please remember that for the JSP frontend (out of the box), this call to calculate cart may be a second call per page load. The first one may be triggered even before the page controller starts working, in Spring MVC Filters. 

 

Creating a shopping cart object

A cart should be created only with the first valid add-to-cart. This is the default implementation. 

A common mistake is creating a cart with the first page load. 

 

Corner cases to consider for Add To Cart

• Product is not in the catalog at all (for a selected country, for example)
• Product is not available
• Product does not have a price
• Product is already in the cart
• Type of product is not purchasable
• Different variation of a product is already in the cart
• Specified product quantity may be too big (single add-to-cart)
• A product is attempted to be added to a non-existent cart (removed by the clean up job)

Test Cases

• Customer can add a product to a shopping cart.
• Customer can add the same product multiple times
• Customer can change their quantity directly in the cart 
• Customer can add different variations (color, size, etc.) of the same product
• Shopping cart totals are updated when a user adds a product to the cart
• Shopping cart totals are updated when a user changes quantity or removes a product from the cart
• Customer can’t add a product not in stock (not available) or with the negative stock
• Customer can’t add a product not in the database
• Customer can’t add a product with a zero price
• Customer can’t add a product with a negative price
• Customer can’t add a product without a price row
• Customer can’t specify too big quantity
• Customer can’t specify a zero quantity
• Customer can’t add products to other customers’ carts (by cart ID)
• Customer can’t add a product to a cart recently removed by the clean-up job

 

Cart Calculation 

Cart Calculation is performed when a cart is in the “not calculated” state. 

So, this component is rather complex under the hood. It is used by many components and any customizations may impact other functionalities. For example, `storeSessionService.setCurrentCurrency` includes cart recalculation as part of the currency change logic. 

Cart Calculation is heavily used by other components:

For example, in the CommerceDeliveryModeValidationStrategy, calculateCart is called up to three times per page load.

Additionally, cart calculation may be triggered from the Cart Restoration Filter even before the controller starts working. This may even add a fourth call per page load. 

[!] Check your Dynatrace reports — you will see that cart-related SQL queries are among the slowest (often together with the media-related, stock level, and price rows), and cart-related modification SQL statements are very likely ranked as the slowest. 

So you can see that cart calculation (as well as re-calculation) is very important from the performance perspective, and all changes that may affect this component should be carefully planned and tested.  

 

There are two implementations for CommerceCartCalculationStrategy available out of the box:

• DefaultCommerceCartCalculationStrategy — a default strategy to calculate the cart when not in checkout status. This implementation supports a rollback
  • If cart requires calculation
    • Rollback-if-not-successful (beforeCalculate -> calculationService.calculate -> updating promos -> afterCalculate) -> calculate external taxes
• NonTransactionalCommerceCartCalculationStrategy — a non-transactional strategy to calculate the cart when not in checkout status:
  • If cart requires calculation, 
    • beforeCalculate -> calculationService.calculate -> updating promos -> afterCalculate -> calculate external taxes

 

Transactions help to revert the changes made by Before/After Calculate-Hooks, Calculation Service, and Update promotions. If something goes wrong and one of these ends with the error, the cart won’t be affected. 

Internally, it runs the hooks, calculates the totals, and applies promotions and discounts. 

• Before-Calculation Hooks
  • • Commerce Cart Calculation Hooks
  • If calculation is required (isCalculated == false?)

• Calculating cart entries
• Calculating cart totals
  • Calculating discounts
  • Calculating taxes (built-in, not an external service)
• Updating promotions 
  • • Not thread-safe; uses per-session locks and synchronized methods
    • Uses Drools for calculating promotions; initializes drools session -> calculates -> applies -> destroys drools session
• Calculating the taxes via an external service (if configured)
• After-Calculation Hooks
  • Commerce Cart Calculation Hooks
    • commerceQuoteCartCalculationMethodHook is used here even if you don’t use quotes in your shopping cart

 

Make sure that your before- and after-calculation hooks are not supposed to be run in some predefined order; Also make sure that before-calculation hooks don’t make any changes that are supposed to be reverted in after-calculation hooks (especially if those changes may be in conflict with any modules outside the code in between before- and after-calculation hooks).  

[!] If you use an external tax calculation provider, use caching to reduce the number of service calls. 

[!] If you use an external tax calculation provider, implement a circuit breaker pattern. The effect of an extremely slow response from a tax service or a timeout error is an increase in the number of concurrent active requests in the system. 

[!] If you use an external tax calculation provider, you have a critical dependency. Implement a feature switcher to disable all add-to-cart buttons if something bad happens with the tax calculation external service. 

[!] Any customizations to the promotion engine as well as any customizations related to the promotion rules may negatively affect cart calculation. Design and test thoroughly! 

[!] Restrict the number of cart entries.

[!] Check interceptors, both type and controllers, before-view and before-controller handlers, Spring AOP definitions related to cart services. These are often time bombs heavily contributing to performance issues. 

For example, Selective Cart Split List Addon introduces a before-controller handler (CartPageBeforeControllerHandler). It updates the wishlists and carts before add to cart controller starts working. 

Normally, SAP Commerce OOTB handles all the corner cases well, but your customizations may affect the standard behavior. 

Updating Promotions

Updating Promotions is performed by PromotionEngineService which replaced the legacy PromotionService many years ago. 

It is important to know that promotion calculation is not thread-safe, so at any moment of time, only one calculation is performed per node in the cluster. 

Also, it is important to know that in SAP Commerce the promotions are calculated each time the cart is calculated. See the diagram above showing how many consumers of cart calculation SAP Commerce has. It happens not only when a cart is requested but even after a product is added to the cart. 

[!] The promotion engine is not thread-safe; the system can’t calculate promotions more for one cart or order at a time per node; it has been sort of “running on empty” in many scenarios, so any customizations or adding complexity to promotions may result in slowdowns and performance issues. 

Promotion calculation is not cacheable in SAP Commerce. Partly that is because the promotion mechanics may involve almost everything from the current time to the customer location. The cache key is hard to make consistent. If you calculate promotions three times per call, the whole process will be repeated three times. For example, when I worked with hybris Travel Accelerator in 2017, I found out that its code performed tens of the promotion engine calls per customer (web) request, which took ~30 times longer than out-of-the-box electronics demo store normally took for cart calculation.

The Drools Facts  (populated via RAO objects) are formed by the RAO Providers. There are order-level and product-level RAO providers. Each provider contributes to the set of Drools facts. For example, CartRAOProvider performs expansion defined by cartRAOProviderValidOptions:

For example, EXPAND_CATEGORIES means that all product categories associated with all products in the cart will be added to the Drools fact list. So, if you have 100 products in the cart, and each product has 20 categories, you may end up with 100*20=2000 items in the facts registry – only for categories. 

Unfortunately, often the number of facts in the registry is not controlled by the development team. Technically, Drools is capable to work with thousands and tens of thousands, but the more facts you have, the slower the process will be.

• In SAP Commerce, Drools is not thread-safe, so hybris can perform only one promotion evaluation process at a time. 
• In SAP Commerce, Drools is used in STATEFUL mode, and initialization and destroying of the data structures and services are performed every calculation
• In SAP Commerce, promotions are calculated often, even when not required

Corner Cases

Corner cases for cart calculation:

• The external tax service is not reachable
• The product availability dropped since last op – not below the ordered quantity
• The product availability dropped since last op below the ordered quantity
• The price changes since last op
• Too many unique products are in the cart (hundreds, thousands) — a big number of cart entries
• Too many non-unique products are in the cart (hundreds, thousands) — a big total number of items
• Same-name different-SKU products
• Cart or Cart Entries were removed by the clean up job but requested by a customer

Also check for

• Unnecessary calculations and recalculations are not performed
• Unnecessary external tax calculation calls are not performed

Look at the following edge cases and typical solutions for them:

• Product availability dropped since the last time a user interacted with the cart – from X to Y, where Y is below the ordered quantity and Y is not zero
  • We need to inform a user that we can’t deliver the ordered amount. Just reducing the quantity silently and automatically, as implemented in SAP Commerce Accelerator, is probably not a good idea for the businesses where the quantity may matter.
• Product availability dropped since the last time a user interacted with the cart – from X to zero
  •  Just removing a product (as implemented in SAP Commerce Accelerator) may not be a good option. Marking the option as unavailable and recommending the closest substitution product seems to be much better. 

Test Cases

• Products are displayed correctly (names, images, and prices, discounts etc.) 
• The product titles are clickable.
• The links lead to corresponding pages with the product info. Promotions, vouchers, discounts are automatically accounted into totals
• If a product becomes unavailable, it should be automatically removed from the shopping cart. A customer should be notified about the change and the reason for the change.

Corner cases for “Quantity is changed” and “A cart entry is removed”:

• If all products are removed (manually or automatically), the shopping cart totals should be zero
• If there is no entry with the given number, the system should return an error
• User can’t change quantity in other users’ carts
• If a product becomes unavailable, it should be automatically removed from the shopping cart. A customer should be notified about the change and the reason for the change.

Cart Restoration Strategies

From the customer’s perspective, having extra items in the cart after logging in may be confusing and undesired, especially if done during the checkout process and especially if the shopping carts are large and complex.   

There are five possible solutions (not all are supported by SAP Commerce OOTB):

• Non-interactive
  • “Session cart priority”. If a customer has an anonymous cart, the account-linked shopping cart will be ignored after the login. If an anonymous cart is empty, the shopping cart contains the items from the account-linked cart.
  • “Cart Archive”: Replacing the old cart with the latest cart; moving the contents of the old cart to the wish list or archived carts list and informing the user about the actions done,
  • “Merge”. Combine the shopping carts together so the user still has a single shopping cart, and the merged cart replaces the old account-linked cart, and inform the user about the changes,
  • “Multiple carts”: Merging carts by default, but letting the customer undo the operation and restore one of two saved carts, “anonymous/guest” and “old account-linked”.
• Interactive:
  • “Interactive” Explaining the situation to the customer and letting him decide what he wants to do, namely, which cart to keep or if he wants to merge them:
    • You have items both in your present cart. But you also have items stored in a previous cart. Would you like to:
      • with your present items
      • your saved items into your present purchase

In SAP Commerce, the default strategy is “Restoring an account-linked cart only if the anonymous cart is empty”, which is titled as “Merge” above. SAP Commerce ignores an account-linked cart if a session cart is present. 

The alternative SAP Commerce’s out-of-the-box strategy is about merging the account-linked and anonymous carts so that the resulting account-linked cart contains all their unique items combined together. 

See https://hybrismart.com/2019/02/24/merging-carts-when-a-customer-logs-in-problems-solutions-and-recommendations/ 

Shopping Cart via Polyglot Persistence

In  v.1905, SAP introduced a Polyglot Persistence, a feature that was supposed to help to relieve the load of the main database or to provide dedicated storage for some resource-intensive data, such as shopping carts.  

To make it possible, the developers need to optimize the data structure and its related types as a single composed structure, or a document. 

Polyglot persistence offers a default implementation for the Cart type (ydocumentcart extension template). Currently, ydocumentcart supports Azure SQL Server, HSQL, MySQL. 

Additionally, there are transaction management and caching subsystem, which allows you to cache all modifications performed on a single item and flush them to the persistent storage when the main operation ends. Reading is realized through a query language similar to FlexibleSearch.

I am not aware of any implementations where polyglot persistence would be leveraged in any way and for Carts in particular. Experimenting with it is on my bucket list. Stay tuned!

Monitoring and Alerting

In order to keep shopping cart performance under control, I recommend 

• Collecting the following metrics and having a dashboard to enable near real-time visual tracking
• Installing and configuring an alerting system to get notified if any of the metrics fell outside the defined range

The following metrics are useful and easy to collect:

• Number of Add-to-carts per time unit
• Number of sessions where at least one item is added to shopping cart / total number of sessions – per time unit
• Average latency per time unit – add-to-cart
• Average latency per time unit – view-cart
• Average response time for external tax calculation (if present) per time unit
• Number of errors related to external tax calculation (if present) per time unit

Cart Clean-up Processes

A big number of carts not being used by customers may make your system sluggish and bloated. Since the tables carts and cartentries are fast-growing, it is important to regularly clean up.

There is a cart data retention cronjob available in SAP Commerce called OldCartRemovalCronJob. It comes with the commercewebservices extension which exposes part of the Commerce Facades as REST-based web services. Make sure that this extension is included in your configuration if you need it. 

[!] Make sure that a OldCartRemovalCronJob is included in your configuration and enabled in your instance. 

The job removes carts (together with their cart entries) older than specified time in seconds provided in the configuration. The default values are 14 days for anonymous carts and 1 month for anonymous carts.