As you know, all interactions with the database in SAP Commerce Cloud are performed via the ORM layer called the Persistence Layer. This layer has an API for developers so that they can perform create, read, update, and delete operations with objects using a database-agnostic language, and the Persistence Layer translates them into database queries.
This layer can detect and store changes in objects in a smart way, so objects are updated only if they have unsaved changes. This component also allows cascading updates, which help you store changes in several objects linked to each other with a single save command.
For example, if you create a new product and a new category for the product, then link the category to the product, and then save the product, the category will be created in the database automatically along with the product.
However, this mechanism has a flaw: when updates are performed in multiple threads or in a clustered environment with multiple processes, it is not concurrency-safe with the out-of-the-box setup.
The SAP Commerce Cloud platform provides optimistic locking through a version property (hjmpTS) on your persistent items. This property is automatically managed by the platform. It is incremented each time you update the item and there are no conflicts.
Conflicts happen if some other process or thread tries to update the item as well. The first attempt to submit an update wins; all the rest are rejected. This is how optimistic locking works in general, and in SAP Commerce Cloud in particular.
However, in many business scenarios, the second attempt should not be rejected automatically just because it changes the same item. This second, third, fourth, and so on attempt may bring the latest changes, which should not be ignored. The normal way is to queue these changes so that they are applied in the original order, but sequentially.
For single-node use, there is a synchronized keyword in Java. However, for a multi-server setup, it is not enough. For clustered environments, the nodes may use shared storage to keep information about which objects are locked in order to synchronize these update operations. A database or NoSQL storage may also be used for this. There are mechanisms for item update locking in MySQL and Oracle.
However, the Commerce Cloud ORM is “too smart” and performs database writes that are not directly related to the objects we change. These writes are classified as “updates,” “removes,” and “inserts.”
“Updates” are the most problematic operations. Besides the fact that we often can’t ignore concurrent updates, there are also cascaded updates that we can’t skip.
What is even more problematic is cascade updates involving relations. For example, you save a product item with changed supercategories.
Let’s say you have one category and one thousand products. All products are linked to the same category. For this, SAP Commerce Cloud creates one thousand records in the Cat2ProdRel table.
If you modify an existing product, and the attribute you modify is not a category, the system won’t touch the category object, so you can take care of product-related concurrency. However, if you create a new product, let’s say Product 1001, the system inserts a new item into Cat2ProdRel. This is safe because it is an insert operation, which is concurrency-safe.
However, SAP Commerce Cloud updates the last modification time for objects involved in a relation, namely Category in this example. This is because Category is a shared object. If you do 1,000 product inserts per minute, the system should also make 1,000 category updates per minute, and all these updates are basically unnecessary.
Another example is CategoryModel.setProducts(products), which triggers an update statement for each product in the set to update its last modified time.
When used in a multi-threaded or multi-process environment, it results in optimistic locking failure exception messages in the log.
Root Cause
For many-to-many relations, such as Product-to-Category, SAP Commerce Cloud updates the last modification time of both objects involved when the relation is updated. The class responsible for this is de.hybris.platform.persistence.links.jdbc.dml.relation.UnorderedRelation, or OrderedRelation for ordered relations.
Solution
The solution is simple: disable updating the last modified time for the relation. For the Product-to-Category relation, this configuration parameter is:
relation.CategoryProductRelation.markmodified=falsePossible Side Effects
Some system components, such as catalog synchronization or Solr indexing, may require the last modified time to know which items to process.