SAP Commerce Cloud has been one of predominant commerce platforms for medium and large enterprise clients across the globe for at least 2-3 years now.
(This is probably why you are reading this article now.) Yet, commerce solutions, frameworks, programming languages, and even some programming patterns change over the course of time to accommodate changes in business or technology demand. Right now, we see that solutions are generally leaning towards
microservices &
Serverless vs. “old-school” monolith and on-prem deployments. SAP’s Commerce Cloud team works hard to accommodate the latest trends with innovations like
Kyma,
Context-Driven Services (formerly known as YaaS), a new Single Page App storefront –
Spartacus, constant upgrades of backend and frontend library versions, and major framework switches (e.g. home-grown promotions framework to
Drools, or old cockpits to backoffice via newer
ZKoss). At a higher level, SAP has also reinvented their ERP solution with HANA as a database and S/4 HANA as a platform foundation. So, clearly there will be no chance for us (engineers) to sit and enjoy a moment, without surviving some transformation and knowledge shift.
While the general course of action is understood and clearly appreciated within the SAP Hybris engineering community, there are always few things which fall out of the sight. Sometimes, important things. I am working with a considerably large community of Hybris engineers and know firsthand which areas of the platform are the most painful to them. Every new Java engineer who starts to study Hybris via
123 (formerly known as Trails) or
EPAM’s home-grown extension of it, mentions:
- “Ughhh… Apache Ant. Is this from 90s? You told me that it would be Maven!” (Don’t blame me.)
- “Why XML definition of POJOs? There are so many modern frameworks to perform ORM!” Some older folks further ask if this is an alternative of Hibernate 2.
I have answers for both, but essentially answer is one – it’s because Hybris was engineered in 1990s. The amazing thing is these two concepts are the only two which have been left unattended. Otherwise, the system design, overall architecture, underlying JDK version, framework versions would make me think it is a few-year-old thoroughly-design app. Kudos to Hybris team!
Let’s get back to Ant and
Type System though. The latter is a core platform feature, and with a respect to a number of SAP Hybris versions, clients and implementations, I cannot imagine how it can be changed or improved. Once, I heard a story of “company X” that implemented and hosted an airline ticket booking solution on a
mainframe and which is still on mainframe because of a number of their clients (time zones) and request flow intensity. That is, they cannot find any window within their client’s SLAs to introduce a downtime. That could be a story of Hybris’ Type System, unless SAP will release a version of Commerce that is not backwards compatible – i.e. has absolutely new ORM.
On the other side, the build system can change. A switch from Ant is not trivial but completely doable from an engineering standpoint – you will just need some time and persistence. The main concept is that a new build system must co-exist with Ant to keep the platform backwards compatible with older versions within next couple of years.
Why Maven?
First,
Apache Maven pom.xml files appeared in Hybris v5. I remember that at time, we dreamed that the next Hybris version would include Maven instead of Ant. A few months before that, in Q1 2013 we started our first attempts at EPAM to “mavenize” Hybris. We were to convert 96 extensions for Hybris v4.7.6 vs. 500+ extensions in modern SAP Commerce Cloud 1808. According to sources that I still have now – we did 70% of a job and then abandoned it because of “urgent delivery priorities”.
 |
| Picture 1 – I still love this concise look of 4.7.6 |
Today, Maven is not the only choice for a new build system.
Gradle fits even better. Some people may also argue in favor of more progressive build systems such as
Bazel or
Buck. Ant+Ivy is an option too. Anyway, large enterprise is always few years behind start up technology stacks, so let us assume that main players are Maven and Gradle. I won’t consider Ant+Ivy for the sake of doing real “transformation”, not just “optimization”.
Gradle looks shorter and works better with script injections, as it is written in
Groovy. Maven on the other hand is proven world-class standard for enterprise solutions for many years now. I can go into details about these two as much as I can, but you may still argue every argument… There is no best tool, there is a “right tool” with a high influence of the designer’s personal taste.
Migration Process
There were soooo many things when we started, that we decided to follow a bottom-up approach. That is, we factored out common things as-we-go and revisited decisions only when they were proven to be failing in practice.
First steps
Having a previous migration experience with Hybris v4.7.6, we knew that process should begin from a module(s) that has fewer dependencies on other modules. platform/ext modules were the right thing to start with. We immediately identified a problem with pre-compiled extensions with bin/*server.jar file and without source code. All *server.jar files, as well as all 3
rd party jars that we can’t find in Maven Central we put into “custom dependencies” in our local Nexus mirror.
(Download the XLSX file with the table above)
Once we completed our work with
platform/ext, we moved on to ext-commerce extensions and the parent
/bin pom.xml. There we spent numerous hours in discussion about how to properly organize modules structure and where to put common code. As you might have guessed,
/bin pom.xml became a host of all common dependencies, plugins and configurations.
Afterwards, we estimated the scope of work and it became clear that we needed to invent a tool for automatic pom generation. This was built it, and I can confirm that all
pom.xml files since its creation were generated via
java -jar pomGenerator-1.0.jar {extension_full_path} command. We just had to make small adjustments afterwards to accommodate unique extension features, dependencies or flows.
When we stabilized the design of extensions, we proceeded to tackle our remaining challenges:
- Generate sources: GeneratedJalo, Jalo, Model, Data
- Properties aggregation and use during config generation
- Hybris intricacies of clean and build phases
- Tomcat wrapper config generation
- Build callbacks
Below I will elaborate on some of these topics.
Types of POMs
There are two types of POMs: parent modules and extension artifacts. The idea is that each extension like commerceservices represents a single Maven project which can be packaged and used as an artifact. Parent POMs, on the other hand, gather extensions in modules. (e.g. ext-commerce aggregates commerceservices and commercefacades extensions)
Parent POMs also have packaging type of POM which allows to use it as aggregators for other artifacts:
<packaging>pom</packaging>
The base (bin) POM aggregates extensions as modules and includes platform with all ext-* artifacts:
<module>platform</module>
<module>ext-accelerator</module>
<module>ext-commerce</module>
Each of which is ready to be compiled and deployed and is packaged to jar.
<packaging>jar</packaging>
 |
| Picture 2 – A very basic view of Maven project structure |
Central vs. Modular Execution Approach
The main migration challenge in terms of architecture is Ant’s centralized build approach. The build executes from a single place with one root and environment: platform/build.xml. Maven, on the other hand, is modular, meaning that each artefact (a.k.a. extension) is built on its own and can be built independently of others. In addition to the above, an Ant build is imperative and is controlled by the script, meaning it can contain complex logic, whereas Maven is declarative and more straightforward.
It is inherently easier for Ant to work with application properties since it runs only once and scans every .properties it requires. Modular execution of Maven made it less straightforward because the build of some extensions may rely on properties in the platform and/or config directories. Reading these properties for every extension (over 150) would be an unwanted overhead. Therefore, we came up with a solution – load properties the first time they are required and store them in the top parent MavenProject context, so that all child modules can retrieve combined project properties.
Custom plugins
Ant and Maven are really different. At a certain point, we understood that some stages of the build process couldn’t be done with typical Maven flow or plugins from central. Of course, we did our best to use existing tools where possible. For example, we managed to map “ant clean” with “mvn clean” via usage of pre-configured
maven-clean-plugin. There are several clean patterns for different extension types (backoffice, hmc, with or without web module, etc.). The following snippet shows configuration for common extension:
Features that required complex logic like model generation and wrapper configuration were implemented using custom Mojo.
- BuildMojo – performs build logic in the same way ant does
- CopyAddonMojo – copies addon sources and resources to corresponding extension
- DeployMojo – sets up configurations for server
- GenJaloMojo – generates Jalo classes
- GenModelsAndBeansMojo – generates models and webservices beans
- InitialDeployMojo – special mojo used to deploy hybris-specific jars to custom Nexus mirror
- UpdateModelsMojo – ad-hoc re-generates models.jar from any extension. Usage: mvn compile hybris:models
Build callbacks
The build callbacks system was another point of interest since they are also implemented in Ant build script. Our decision was to use Groovy because of its interoperability with Java and scripting nature. Also, Groovy does not require a steep learning curve as it is already incorporated into many parts of existing platform code.
Every extension requiring build callbacks must have its own buildcallbacks.groovy file next to buildcallbacks.xml.
With the help of
gmavenplus-plugin we placed a groovy script in the main (bin) POM which is responsible for running callbacks in those buildcallbacks.groovy files. The build logic would lookup the .groovy file and, if it is there, the script would be evaluated as a part of build phase.
Common blocks of build callbacks were factored out as utility callbacks into utilsbuildcallbacks.groovy in bin directory and saved in MavenProject’s context which allows a quick retrieval of it by other callbacks throughout extensions.
SASS/LESS
Backoffice provides support for Syntactically Awesome Style Sheets (SASS) scripting language, a CSS preprocessor that gives better experience in styling your application.
Backoffice copies SCSS files from backoffice extension, Backoffice Framework JAR files and the registered backoffice extensions to the resources/backoffice/generated directory and then compiles them. Registered backoffice extensions are located at ${HYBRIS_HOME}\bin\mvn_sassextensions.
New location for a SCSS processing logic is
${HYBRIS_HOME}\bin\ext-backoffice\backoffice\buildcallbacks.groovy
and it is backwards compatible with Ant’s version of callback as it does the following:
- Copies and compiles .scss files from extensions and jar files
- Keeps track of changed files via a “touch” file
${HYBRIS_HOME}\bin\resources\backoffice\generated\.sasscompiled
- Unzip sass gem, collecting all SCSS files and compiles them using Ruby script
- Copies .scss files from CockpitNG jar, backoffice, registered backoffice extensions and additional resources (indicated in local.properties)
- Replaces all placeholders (@EXTENSION_VARIABLES_PLACEHOLDER extension@) with ‘@import extension-variables.scss’; or removes placeholder in case when extension-variables.scss doesn’t exist.
- In case when extension-variables.scss doesn’t exist. Macro takes into account cockpitng-globals.scss, backoffice-variables.scss, file indicated in local.properties (backoffice.sass.preferred.variables.file) and variables files from all registered extensions
- Copies .scss files from source directory to target directory and replaces certain tokens:
For those who ever tried to customize backoffice styles, logic to do so remains the same, except you must call mvn compile instead of ant all in step 2:
- Change any file *.scss. For example,
{HYBRIS_HOME}\bin\ext-backoffice\backoffice\resources\cockpitng\widgets\workflowsearch and change workflowsearch.scss:
- Then need call mvn compile from {HYBRIS_HOME}\bin or {HYBRIS_HOME}\bin\ext-backoffice\backoffice (for only backoffice extension)
- After this you can go to {HYBRIS_HOME}\bin\ext-backoffice\backoffice\resources\backoffice\generated\cockpitng\widgets\workflowsearch and open workflowsearch.css and you can see changes from workflowsearch.scss
The compiled CSS files are in the same directory as copied SCSS files.
Summary
Finally, after almost a year of work we completed our PoC and are now in a process of reviewing the code with the SAP Hybris Platform team. Fingers crossed, we all hope to see Hybris Mavenized in 2019!
Wondering what is the real benefit?
Generic
- Created directory-based pom hierarchy, which defines great level of abstraction and reuse for bottom-level poms.
- Developer-intuitive mapping of build phases from Ant to Maven.
- Re-thought and re-worked build process tasks – Central vs. Modular architecture
- 150+ extensions migrated to maven.
- Developed 8 custom Mojos
Build Process
- Reduced build time. Maven in a multi-threaded mode performs faster than Ant.
- Groovy callbacks – so much better! And developer-friendly too!
Utility
- POM Generator CLI can generate new pom.xml based on platform path. Significantly speeds up migration process from Ant to Maven for extensions that are not yet Mavenized
- Custom Artifacts CLI uploads custom and *server.jar artifacts to Nexus
Standardization
- Maven aligns SAP Hybris Commerce Platform build process to Java enterprise-class standards
- Easier switch from Maven to Gradle
- No more libraries in source code – reduced platform size
- No more .project files in code
- Declarative dependency management
- One-click platform installation in modern IDEs
- Native IDE support for library sources and documentation download
Future View
Possible other benefits that this new build system may bring to overall platform transformation in the future:
- Ability to deliver platform updates via incremental modular updates – platform binaries
- Platform upgrade process might become so much easier
- Widen Hybris technical community – younger engineering generation is not proud to mention Ant in their CVs
- Get rid of legacy, deprecated Ant code and custom tasks
- Standardize platform startup. Run tomcat from Maven, just as Spring Boot App.
