This article is intended for you, dear clients, both current and future. Whether you’re looking to build a new application, enhance an existing system, or integrate complex technologies, crafting an RFP that developers will love involves more than just listing technical specifications. It’s about creating a document that not only conveys your project’s vision but also fosters collaboration, innovation, and a mutual understanding of expectations.
In this guide, I will explore key strategies to help you write an RFP that attracts high-quality proposals and sets the stage for a successful partnership with your development team.
This article will help you understand the RFP process in the realm of software development from the developers’ perspective. By gaining insights into how developers interpret and respond to RFPs, you will be better equipped to communicate effectively and align expectations, leading to more successful collaborations and project outcomes. Read More »
When working with APIs, dealing with raw JSON for input and output can be cumbersome, especially when there’s a convenient UI designed specifically for that API. However, the UI doesn’t always keep up with changes in the service, and sometimes it’s useful to emulate changes in the service along with the UI to finalize decisions on what and how to modify.
Such a solution could allow you to replace actual API responses with tailored fake data, eliminating the need for altering real data or making code changes. It would accelerate slow requests by recording and replaying them, significantly reducing wait times for UI tweaks. Developers could simulate edge cases, such as handling 4XX or 5XX status codes or testing long strings, by modifying response bodies, status codes, or headers, or add a debug/verbose mode automatically to all calls to an API. Additionally, it would enable frontend development against unimplemented APIs by mocking responses, facilitating progress even when backend endpoints are incomplete. For integration with third-party platforms that only function on production domains, such a solution could reroute requests, allowing safe testing on production without risk. Furthermore, it would support debugging by adding artificial delays and simulating network errors, ensuring robust handling of edge cases. In essence, mocking servers on production in your browser would streamline development, enhance testing, and ensure a more resilient application.
As the “victim,” I chose the SAP Commerce demo store (to whose code I don’t have access to), specifically its search functionality. The goal was to inject my custom code between its backend and frontend, and as a demonstration of capabilities, automatically apply certain facets for specific queries. As a result, for the query “cheap cameras,” I will actually see cheap cameras and not tripods, for example.
Read More »In one of my projects, it was crucial for me to monitor data volumes and some specific e-commerce system parameters and alert the technical team if there were any abrupt or unexpected changes in them. These fluctuations occasionally (though not always) signaled potential existing issues or an increased likelihood of future problems.
I am focusing on operational parameters (orders, shopping carts, customers, logins, product data quality), not technical (page load time, timeouts).
Due to the lack of built-in solutions in SAP Commerce for this function, I had to design and implement a custom solution from the ground up. This article delves into the specifics of that development process and details of the architecture.
Read More »In one of my recent personal projects, I ventured into the vast archives of The Saturday Evening Post, a magazine that has captured American life since 1821. What started as a nostalgic dive into the past tu,/rned into a technical quest to make over a century of magazine issues searchable and accessible online.
Utilizing a blend of OCR with Tesseract and search mechanisms based on Apache Solr, OpenAI embeddings, FAISS, ML Re-ranking, I managed to create a searchable database of the magazine’s rich content from 1900 till our days. From Norman Rockwell’s iconic illustrations to pivotal historical articles, every page was digitized and ready to be explored.
This project engagingly occupied several weekends and further deepened my expertise in search technology. Dive into the article to see how vector search and traditional keyword search can be used to bring the past into the digital age.
Read More »Last night, a spark of inspiration on my evening drive turned into a monumental achievement by the early hours: I successfully migrated 6,000 posts from Facebook to my brand-new Wordpress blog and later translated them to English and published at RaufAliev.com. So now I have two blogs, in Russian and English, having the same content as my Facebook (I don’t post anything on Facebook that isn’t meant for others to see, and I don’t write or say anything that I wouldn’t say publicly, so from a privacy perspective, everything is okay. Also, I don’t transfer comments to the posts, only the posts themselves.) This isn’t just about launching another blog; it’s about preserving years of shared memories and insights in a more accessible and enduring format.
I tackled the challenge of exporting and translating a vast archive of content, making it available not only in the original Russian on BeingInAmerica.com but now also in English for a global audience. This project was driven by my frustration with social media’s fleeting nature and the limitations in searching tagged posts.
Curious about how I managed to automate the complex process of syncing thousands of posts across languages and platforms? I’ll be sharing the behind-the-scenes story of the tools and technologies that made this possible.
Read More »Navigating the vast seas of product searches in B2B and B2C marketplaces can be quite a headache, especially with loads of data from various suppliers. It’s tough for marketplaces to tweak supplier info without spending a fortune, yet when searches go awry, customers tend to blame the marketplace, not the suppliers. My new article suggests a slick new way to cut through the clutter—think product families, not just standalone products. This isn’t just about tidying up; it’s about making the search experience smoother and more intuitive for everyone involved.
Let’s dive into why the old way of doing things—splitting catalogs into neat categories and individual products—just doesn’t cut it for big, diverse inventories. Take electronics and clothes: sorting shirts by color and size is straightforward, but try categorizing gadgets that way and you’re in for a real challenge. The game-changer here could be switching to a product families model, which groups related products together under one big family umbrella. This approach keeps things organized and makes shopping easier, which is exactly what we need for a smoother, more customer-friendly search process in our digital storefronts.
The Part I of this topic is about Product Families as an extension of the Variant Products concept.
Read More »An incremental load is a form of ETL (Extract, Transform, Load) process that involves transferring only the data that has been altered since the last loading session. This approach is both more efficient and quicker than performing a full load because it minimizes the volume of data that needs to be moved and processed.
However, implementing an incremental load necessitates mechanisms for recognizing and monitoring modifications in the source data, using methods like timestamps, flags, or versioning. Additionally, it must effectively manage issues like conflicts and duplications, for instance, when a single record undergoes multiple updates.
In this article I am focusing on my experience in implementing the optimized data loader integrated into the target system which was SAP Commerce. Simply put, it handles duplicates and minimizes the number of unnecessary database writes.
Read More »Imagine a world where an AI system not only listens but observes, constantly collecting data unless explicitly told not to. This “exoskeleton for the brain” is not just a tool but a transformative extension of human capability, enhancing efficiency and decision-making in everyday tasks. For some, this raises alarms about privacy and the potential for misuse in a society already wary of data overreach. Yet, the real question lingers: how will such capabilities shape the way we live, work, and interact not just with technology but with each other?
As we stand on the brink of making these integrations a reality, it’s important to consider not only the current implications but also the generational shift that might come with it. The youth of today, growing up with AI as a norm rather than an addition, might view this integration as indispensable—melding their cognitive functions with artificial intelligence to an extent previously unimagined. This narrative isn’t just about the technology we create but also about the human experiences and societal structures that will evolve with it. Dive into the full discussion to explore the profound shifts looming on the horizon.
Read More »In this article, I want to share an interesting experience. It has nothing to do with SAP Commerce and might even be unrelated to eCommerce, but it could be useful for Solution Architects when tackling a task similar to mine. And the task was interesting.
It was a huge Excel file for generating estimates for B2B customers. A manager entered a bunch of parameters according to customers’ needs and get a printable quote. This Excel spreadsheet was really old, already about ten years old, and all these ten years, different people have been adjusting it, correcting it, adding formulas and constants. Basically, almost nobody knew how it worked in all details. All this evolved into a dozen tabs, each filled with numerous formulas. And these formulas refer to similar formulas from the adjacent tabs. It has almost a thousand formulas, mostly complex. How to untangle this knot?
There was no documentation, but even if there had been, where is the guarantee that the documentation fully matches the Excel file? After all, if it doesn’t match, finding the root causes of the discrepancies will be very difficult. Since the Excel file is the source of truth, it basically is the documentation.
These are the kinds of problems I love. In the article, you’ll find the solution and, believe me, it’s very non-trivial.
Read More »In this article, I want to talk about my experience with how to make a website or online store search more responsive to business requirements and user expectations.
It’s common to give too much attention to technical issues, especially now, with the advent of machine learning/AI in this field. But more often than not, the deepest problems are on the surface and their solution is much simpler than a functional refinement.
One of the main challenges in managing e-сommerce websites is ensuring the product search feature works effectively. Imagine this: you notice an issue with how products are being found through search, and you decide to fix it by tweaking some settings. However, altering these settings could unexpectedly impact your business. When we talk about something being ‘broken’, it’s not just about error messages popping up. Rather, it could mean that customers are having a harder time finding what they want, which might lead to fewer sales over time. This is a subtle effect that can spread across your entire platform. But how do you pinpoint the exact cause of such a decline? Testing and monitoring changes in an ecommerce environment are complex because even small modifications can have wide-ranging effects on customer behavior and, consequently, sales performance.
To monitor changes, it’s essential to understand what to collect and how to extract data from the gathered information. Specifically, for e-commerce search, there are no good ready-made monitoring tools available, but there are various DIY kits that address some needs. Such tools suggest collecting and analyzing search results for reference queries and visualizing changes over time. However, this approach falls short in many respects. For instance, it only covers the most frequent queries.
When it comes to modernizing site search, there’s a big question about whether this is done from scratch or if it’s possible to use accumulated data from existing sessions. Developing from scratch is always a risk—because any assumptions you have about how users will search for content may be wrong. Therefore, regardless of how good the first version of the search turned out to be, it will need to be revised and reworked many times.
Today, I want to share the approach that I have used in several projects. I call it Search Analytics. Read More »
This mechanism represents a significant opportunity for those looking to push beyond the platform’s standard capabilities, offering a pathway to tailor DTO generation precisely. However, navigating this feature requires insight into SAP Commerce Cloud’s DTO generation mechanism, including custom post-processors and XML configuration parsing.
Our article serves as a guide for developers eager to explore this advanced customization layer, detailing the steps to effectively utilize the hints mechanism without delving into the technical complexities here. Discover how to harness the full power of undocumented hints for your DTO customization needs in SAP Commerce Cloud by exploring our comprehensive exploration.
Read More »A number of customers have been talking about the upgrade policies defined by SAP for SAP Commerce Cloud and the Composable Storefront. I mentioned the upgrades last year in the hybrismart.com article.
Historically, since I first met hybris 4 in 2009 I have seen clients who did not upgrade their versions of SAP Commerce for years. However, that all changes with latest policy SAP have with the CCV2 environment.
These policies require that the merchant needs to keep on top of the upgrades which are currently being issued each month and therefore factor those upgrades into their development, testing and release cycles. Each release of SAP Commerce and the Composable Storefront has a 6 month validity period after which the release cannot be deployed.
A key question being asked is how onerous is this upgrading process ? and how often ? There are usually two views: 1) the “little and often” approach, a view also recommended by SAP, that upgrades should be factored frequently; 2) the “if it isn’t broke don’t fix it” approach where we have seen examples SAP Commerce version they use has passed its long term support date.
In the end, it comes down to priorities and costs and we have experienced upgrade projects taking months because the version being upgraded from is quite old.
In order to get a tangible view of the current upgrading of both SAP Commerce and the Composable Storefront I decided to undertake some sample upgrades on a local version I run. For those who are interested, I use an Oracle VirtualBox VM Debian guest running in a Windows 11 host). I have traditionally built SAP Commerce using the various out-of-the-box recipes and accelerators using the instructions published by SAP. To an extent, what I am doing probably reflects what many project teams have undertaken, in that someone needs to undertake the “upgrade” user story and share with their team.
In this article you will find the detailed walk-through.
Read More »I was partly correct – but I had underestimated what would happen !. In fact as of end-November 2023, there have been 9 releases of the Composable Storefront since 5.0, now being released on an almost monthly basis, and there have been 12 update releases of SAP Commerce Cloud itself, now in the form of the “Continuous Innovation” releases 2211.x, and at time of writing to 2211 updated to release 2211.15 and also released on an almost monthly basis.
I suspect that between the time I write this, and the time you read that at least one more release of Composable storefront and Commerce Cloud will have been released. In this article, I will share some important updates on this topic. Read More »
Introducing ‘fsdumpviewer’: The Tool for File System ‘ls-lR’ Snapshot Comparison and Synchronization
In this article, we’re diving into the features that have emerged between versions 2023.1.1 and 2023.2.7 of the Plugin. No need for elaborate words, let’s get straight to the highlights:
- Project Import 2.0: An upgraded import process with automatic database integration and improved module management.
- Kotlin Integration: Now supporting Kotlin language seamlessly, making development smoother.
- CCv2 Enhancements: New icons and seamless module imports for CCv2. Improved HAC cluster support, OCC enhancements, and better Groovy script management.
- FlexibleSearch 2.0: A revamped FlexibleSearch language with new icons, folding, and improved formatting.
- Type System Evolution: Fresh icons for various types and relations, and a diagramming experience for Type System in Intellij IDEA Ultimate.
- Business Process Refinements: Better code completion, custom language injections, and enhanced scripting support for business processes.
- ImpEx Power-ups: Improved code completion, macro handling, and sld.enabled modifier support for ImpEx files.
- Bean System Mastery: New icons, code completion, and reference enhancements for beans, with conversion possibilities on the horizon.
- User Rights 2.0: Revamped User Rights support in ImpEx files, making management more intuitive.
The article highlights improvements such as:
- API Usage: How JetBrains’s latest batch of deprecated APIs impacts the plugin.
- SAP Commerce in the Public Cloud (CCv2): Dive into the new features simplifying developers’ lives, including the addition of code contributions based on JSON schema.
- SAP Data Hub on CCv2: Explore the complexities and solutions offered for projects utilizing SAP Data Hub.
- JavaScript Storefront: The essential tools and updates for developers working in the SAP Public Cloud.
- Type System: Discover the new usability improvements in preview mode and items.xml.
- ImpEx & FlexibleSearch: Understand the nuances and tweaks in action group displays and custom icons.
- Cockpit NG System: From commons to the business process, take a look at the new references added.
- Extension Info: Dive deep into the added extras for extensioninfo.xml.
…and a plethora of other essential updates, adjustments, and fixes to ensure smoother, more efficient operations for SAP Commerce developers.
Read More »In the previous part, we discussed the importance of redundancy and failover in a cluster setup and introduced JGroups as a crucial component of SAP Commerce Cloud for distributed messaging and eventing. In this part, we will delve deeper into troubleshooting common issues that arise with JGroups in SAP Commerce.
In this article, we will explore troubleshooting tips for common issues related to JGroups in SAP Commerce. We will discuss how to address data inconsistencies, handle dynamic IPs and membership discovery, and effectively monitor the cluster.
It is important to highlight that these recommendations were formulated and tested for the on-premise, self-hosted SAP Commerce. This setup is now rare, as many SAP Commerce projects have already been migrated to the cloud. In the cloud setup, such challenges may never be your concern, as they fall under the responsibility of SAP CCV2 support engineers. However, understanding the limitations and constraints will still help you build the solution correctly.
Read More »This article focuses on how SAP Commerce stores models in memory, which is a vast topic. To begin with, we will explore a relatively simple aspect, such as the unusual structure that appears in the debugger when trying to view the contents of any SAP Commerce model. We will also discuss how to access all the object properties contained within it.
Read More »- stopping support on the previous Spartcus versions (4.x);
- that Version 5 would be only available within an authenticated policy to SAP partners / customers; and
- Spartacus is renamed to Composable Commerce.
Read More »
Many merchants have 3 such environments typically defined as development, staging and production and as time passes by, the production environment receives multiple BAU changes to core data areas such as web content, products/categories/classifications undertaken by merchandising teams or product management and of course a lot of data is generated through customer registrations and orders placed.
However, the non-production environments are frequently not kept in line in terms of web content, product data or representation in terms of quantities of customers or orders. I’m not referring here to the code and core configuration related data that is included in builds.
Read More »
Taking advantage of back-in-stock notifications, customers are informed that a product they were interested in recently has become available again. Alternatively, a customer may want to get subscribed to the product price updates and get notified when the price drops on a product.
As a result, customers are more likely to return to make a purchase. These two features are often considered as must-have for almost any marketplace solution.
Their implementation and design are also considered to be quick and easy. In fact, it is not so — there are many points for consideration easy to miss in the design phase. In this article, I focus on the potential pitfalls and best practices in implementing “Back-in-stock notifications” and “Price drop alerts”. Read More »
There are dozens, even hundreds of test automation frameworks available, and we developed our own. Why? Why? Was it worth the effort?
This article describes our needs and our solution, E2E CLI: the architecture of the API testing tool which has been used by our team for about 2 years. Back in 2020, it was developed by me from scratch over the course of a weekend — In other words, it is a very simple piece based on very simple ideas. That first version was supporting 80% of the functionality available today in the last version (and described below). We don’t add features without good reason.
We use this tool to end-to-end test all our services and integration flows. Whenever we make changes to a software system, we need to ensure that they do not break what was already working.
As the architect and lead developer, I designed and developed it from the ground up, so, of course, I am very proud of my brainchild. But the field cannot be well seen from within the field. You know, reinventing wheels is not always bad. When you build something on your own, you are in full control over what is being built, what its purpose will be, and when it is finally conceived and executed as you wish. It has been two years since I have encountered any alternative to our solution.
To reach a wider audience, I found it beneficial to share key concepts hoping to get some feedback from the community and enhance the product.
Technically, the tool has nothing to do with SAP Commerce; it’s for testing. However, our entire team, which uses this tool daily, is focused mainly on SAP Commerce development. And the tool is tailored to be used with SAP Commerce.
While the client (for whose benefit the tool is being used) has authorized me to share high-level details, I cannot disclose names or code. In spite of this, the conceptual view should be useful on its own and might become a good foundation for your own solution.
Read More »A shopping cart is the centerpiece of almost every e-commerce website.
In fact, this component is the first thing that makes an e-commerce store different from just an online catalog. Working with issues and challenges related to the cart is something you’ll never forget.
The Russian novelist Leo Tolstoy, at the opening of his novel Anna Karenina, writes ‘All Happy families resemble one another, but each unhappy family is unhappy in its own way.’ You know, the diversity of cart-related issues I have been observing in the SAP Commerce projects, made me think that Tolstoy meant the developers as well. The diversity of the issues and challenges in this component is so vast that every case is different. However, when it comes to building or optimizing the shopping cart processes, understanding the limitations, pitfalls, and peculiarities is a huge plus.
Let’s have a look at how a shopping cart works in SAP Commerce under the hood and how to avoid making mistakes in shopping cart customization.
Read More »It all started with the idea of making the built-in flexible search console more convenient for a developer. It is common to find ourselves forgetting the name of an attribute or its type. Imagine my surprise when I found the traces of this feature in the HAC source code. SAP left them unfinished and probably abandoned. For example, for the Impex console, suggestions are basically implemented and almost ready, but for some reason, they have not been finished and not enabled.
In this article, I’m filling this gap and document how to enable autosuggestions in the Impex console. Additionally, I introduce my custom solution for Flexible Search suggestions and autocomplete. Read More »
The essentials were thoroughly explained in the first part of the series. Since the topic is controversial and should be used with caution, we highly recommend reading the introductory part before going further with this piece.
You need to be aware of the potential risks, limitations, pitfalls, and caveats of the ORM engine. In this work, we are explaining those details. We’d like to touch upon a few elements or aspects, you need to take into account when considering using native SQL in your SAP Commerce code. In this part you will learn more about the SAP Commerce L1 cache, multitenancy aspects, service layer direct, and performing faster bulk updates.
Come take a look into the v2005 release of SAP Commerce Cloud. I’ll cover cool and interesting new features, enhancements, and changes, and offer some commentary on what I like and what I like… less. Judge for yourself what SAP did well and what might warrant some extra polish.
Read More »Some say they believe they won’t be able to use the knowledge and experience anywhere else in their careers.
Some of them see this move as ‘boxing’ themselves in.
It is generally believed that focusing on the niche product may cause taking a lot of time in the future to get re-trained for a different product or technology if the current market conditions change and SAP Commerce will fade in relevance and popularity. Proprietary platforms come and go, and developers feel afraid to find themselves with useless knowledge and meaningless experience.
Even though I generally share the view and arguments above, I also believe that SAP Commerce is a special case. What I want to leave you, then, is with the explanation of why I think so.
Read More »Read More »
When the coronavirus outbreak happened in China, it boosted local e-commerce as many stuck at their homes forced to buy their goods online. Sales of fresh food on JD.com reportedly jumped 215% during the 10-day period and counting. The ability to serve a large number of customers is growing every year. The traffic may be doubled in a week. How to make sure your system is capable to serve high traffic demand? In today’s article I’m sharing my experience with Tsung, a high-performance testing framework.
Read More »The SAP Commerce Cloud APIs are highly customizable, and even smallest changes in configuration, data model or Java code may result in performance degradation. Performance testing is especially important in cases where concurrency may be the source of the problem. To recognize and fix such issues timely, it is recommended to perform regular performance testing, ideally integrated into the build and deployment pipelines. This article is devoted to Apache JMeter, one of the well-known tools for automated (and) performance testing.
Read More »Any SAP Commerce Cloud setup is based on a multi-server setup organized as web cluster. This product comes with clustering support out of the box since very early versions. However, many years in a row, it is still one of the most challenging topics.
JGroups is an essential component of SAP Commerce Cloud. It is widely used as an embedded solution for distributed messaging and eventing in the cluster. It is hard to say that it is underdocumented or it has some quirks that nobody knows how to encounter. However, almost every large project stumbles with JGroups. This article is aimed to shed some light on the internals of the SAP Commerce Cloud clustering and JGroups.
Read More »SAP Commerce developers everywhere face the problem speed of building/restarting hybris. Probably everyone knows this is a long time waiting for the project to build and waiting for the server to start. In cases when the developer makes frequent changes in the code and wants to see the result immediately, the work is significantly slowed down. Many developers use JRebel and similar tools, but unfortunately, they are not suitable for everyone and don’t work for a number of cases. This article explains how to make the compile time shorter.
Read More »Product data is a core of e-commerce and integral part of the customer experience and customer journey. The well-organized product data drives customer acquisition, sales and eventually results in the increasing lifetime value of a customer.
SAP has recently announced a new SaaS offering, SAP Product Content Hub, a cloud-based solution for product and information management (PIM). The new product relies on the SaaS model only.
In this article, I’m basically just going to give a little bit of an introduction to what SAP Product Content Hub is as well as shed some light on what is under the hood in terms of architecture and customizability. Read More »
What content management system you use is of a much higher importance today than it was years ago. Today’s CMS needs to contend with a multi-device world with a wide variety of tablets, mobiles, laptops, and be architecturally ready for new not-yet-existing channels in the consumer’s universe, such as AR/VR and the Internet of Things.
A CMS typically consists of a frontend part, or a «head», and backend. The headless CMS, as their name says, don’t stick to a single predefined “head”. Theoretically, you can develop your own if it is good for your case. Another name for headless CMS is API-first because this interaction is based on APIs.
For example, the old hybris CMS Cockpit had both parts built with ZK Framework, which orchestrates both the HTML/Javascript “head” and was the essential part of the CMS backend. These parts were tightly connected and intertwined into a single unit called CMS Cockpit. Later, SAP announced Smartedit, which has been a big bet and a quantum leap for Commerce Cloud’s content management. Spartacus, the open-source Javascript storefront from SAP, well complemented Smartedit. I published the comprehensive reviews of both products in the blog earlier.
There are a number of headless content management systems that are mature enough to efficiently replace the built-in Smartedit CMS. Among them, I would highlight Contentful, Prismic, DotCMS, Cloud CMS, Amplience, and Contentstack as well-established and innovative products with a clear roadmap and good integration capabilities.
One of my recent research projects was aimed at using Contentstack as a headless content management system for SAP Commerce Cloud. This article uncovers the key highlights of the research. As a result of that work, I also created a data migration tool for moving existing CMS content from Commerce Cloud to Contentstack.
What is Contentstack? Contentstack is a SaaS headless CMS with an API-first approach that offers the approach known as Content-as-a-Service. It focuses on managing structured content into data feeds that other applications, such as Commerce Cloud, can consume. You can design a content data model interactively or via the Contentstack API and populate the structures with the content interactively or via the same API.
In this article, I pay special attention to the data migration from Commerce Cloud’s CMS to Contentstack. This exercise helped me to see the complexity in all its glory. Because when you design a system from scratch, you can evade the platform’s constraints by designing the structures compatible only with the new CMS. When you need to migrate data, you have to find a solution on how to adapt concepts and data models to get them compatible with Commerce Cloud.
Read More »
Payments is a critical part of any e-commerce system. Integrating with a new payment processor is often full of unknowns for developers, and the documentation is either too cumbersome and time-consuming or, on the contrary, scanty and, to the extent it is available, tends to focus only on technical details.
The aim of this article is to provide a primer on the key concepts and terminology of online card payments as well as shed some light on the payment integration and related complexities. It provides brief context on the payment system and key regulatory principles and best practices you need to know before designing and implementing the payment module. We also covered the oft-forgotten edge cases you need to handle for large e-commerce solutions. You will learn what to expect and how to address the payment-related issues and problems. Read More »