Troubleshooting Cronjobs Issues (HMC, Triggers etc)

This quick post is about how to troubleshoot issues with Cronjob, specifically on the situation when a cronjob is not started.

* * *

Many of you still use HMC. Yes, it was removed from SAP Commerce Cloud, but there are still hundreds of installations where HMC is in use.

I found an interesting issue today. The issue is related to cronjobs and HMC.

 

What I was observing today is one cronjob not started when its schedule says. It was enabled, but the system didn’t execute it on time.

HMC shows settings were correct. What’s wrong? First thing I checked if the cronjob has a node group or node id configured. If these settings are not correct, the cronjob won’t be executed at all. These were set properly.

The next step is making a snapshot of all relevant data. For the cronjobs, it is CronJob and all its subtypes, and Trigger. 

For exporting all data objects, I used the script from my article about exporting data. The script generates the type tree, you can pick the relevant types from the tree, then you can generate Impex export scripts for the types with all attributes.

I compared data extracted from the broken instance with the same set from the working instance and found that a Trigger object is not populated correctly. 

How it is possible? I know for sure that nobody would change these settings on purpose. Zeros means “a first day” while “-1” means “every day”. 

The reason is a bug in HMC. 

The screenshots below is from the out-of-the-box configuration SAP Commerce Cloud: 

Interval Range select boxes are populated incorrectly because there is no such value as “-1” on the list of available items. If one tries to save this object, the zeros will be saved, and the trigger will be re-configured silently. And, of course, it will stop working as expected. 

In order words, the UI says there is a zero value for days, seconds, hours, days, months, years , but in the database each of the listed item has “-1”. 

Checking data before pushing a “Save” button:

select {CronJob.code}, {day},{hour},{month},{second},{weekInterval} 
from {Trigger}, {CronJob} 
where {Trigger.cronjob} = {CronJob.pk} and {CronJob.code} = 'update-index-cronJob'

Then we push a “Save” button:

Checking data after saving the trigger:

select {CronJob.code}, {day},{hour},{month},{second},{weekInterval} 
from {Trigger}, {CronJob} 
where {Trigger.cronjob} = {CronJob.pk} and {CronJob.code} = 'update-index-cronJob'

There is no such issue in the backoffice because select boxes were replaced with input fields:

In order to fix the issue, you need to apply the following impex code (the example for ‘every 5 min` case):

INSERT_UPDATE Trigger;cronjob(code)[unique=true];day;hour;month;second;year;weekInterval
;update-index-cronJob;-1;-1;-1;-1;-1;; 

But This Issue Wasn’t the Only One

The reason was the update task was marked as failed because of a series of unsuccessful attempts and Trigger Task was marked as failed.

In Commerce Cloud, there are two objects named similarly, Trigger and TriggerTask. For each Trigger item, there must be a corresponding TriggerTask item. The TriggerTask items are created together with Trigger items (see CronJobManager.afterItemCreation). Sometimes, these objects become out of sync, and there are triggers without TriggerTasks. In previous versions, the OOTB code removes the TriggerTask in some situations, and cronjob stops working. But that wasn’t my case.

In my case, the TriggerTask item was marked as failed that was a reason why the corresponding Trigger was blocked and was not fired by the system on time. 

The Trigger Task item is not reachable from HMC/Backoffice from the Trigger panel. 

Conclusion

If you experience a similar issue, remember that there is a TriggerTask object corresponding to Trigger, and check its configuration as well.