Introduction

Varnish is an HTTP accelerator also known as a caching HTTP reverse proxy. You can install it in front of hybris and configure it to cache the contents. Varnish Cache is a very fast thing. It typically speeds up delivery with a factor of 300 – 1000x, depending on your architecture. However,  your backend system should be compatible with Varnish to use all its features and speed. Default hybris isn’t. Last weekend I managed to integrate Varnish with hybris. There is a module from hybris Professional service, that does the same task, but as to my knowledge it has some peculiarities that limit the areas where it can be used. Hybris professional services are able to identify these areas and offers this module only if it will work well with the customer data, traffic and website structure. Generally, their solution is good for most cases. Nevertheless, I decided to try Varnish with my page fragment caching infrastructure which I explained in the previous post. Below is a comparison between my solution and my understanding of hybris Varnish caching module. Unfortunately, due to the SAP licensing model these modules are not available for detailed study, so my comparison  is based on opinions of others I managed to get.

Varnish hybris extension	My solution
CMS objects only	can work with any page fragments. It introduces new tags for JSP that can be used  in JSP/TAG files without any limitations
no capabilities for cache invalidation	for some cases it has some capabilities to manually invalidate the caches
caching “live” HTML fragments is not possible (for example, list of customer addresses)	capable to work with “live” HTML fragments
well-tested and used on many projects, but it is not free	experimental thing (but free)

The last point is a strong one. My solution seems to be more flexible and cheaper, but it might also be more expensive in support. If you do have a choice, it would better go to hybris Professional Services.

Solution

The overall diagram of my hybris-Varnish implementation is below. It is based on the page caching concept that I explained in the previous post. According to that, the fragments are supposed to be cached should be marked with the tags. This can perhaps best be illustrated with a simple example.  Let’s take the following page layout: the cache markers might be placed as following:

Custom tags

The cache:cached tags are processed by my custom tag library class that extends BodyTagSupport and implements the DynamicAttributes interface. There are three methods to override, doStartTag(), doEndTag() and setDynamicAttribute(s,s1,o).

public class CacheTags extends BodyTagSupport implements DynamicAttributes

@Override
public void setDynamicAttribute(String s, String s1, Object o) throws JspException {
{ this.dynamicAttributes.put(s1, o.toString()); }

For the latter method, I used some pre-defined macros for attribute values: “sessionid”, “url+get”, “url”.  For example, “url” is processed by the following code:

ServletRequest sr = ((CacheTags) this).pageContext.getRequest();
String url = ((HttpServletRequest) sr ).getRequestURI();
this.dynamicAttributes.put("url", url);

So it means that for the tag “<cache:cached param1=”A” param=”url”>…</cache>” you will use page URL as a part of cache key:

"A_http://domain.ru/folder/folder" => "cached fragment"

The compound key consists of keys separated by “_”. It is convenient for debugging purposes, but for the real system, I recommend you to use a hash function to keep the key size constant. You can use any attribute names except “ttl” that is a reserved one for TTL value.

if (s1.equals("ttl"))
{
    ttl = Integer.parseInt(o.toString());
}

Varnish server

As I mentioned before, Varnish server is in the middle between the user and the application server(s). So for the debugging purposes you can request the “raw” response from the application. Note that ESI tags are mixed in the HTML tags. If you open this HTML in browser, the page will be broken, because some fragments were replaced with ESI tags. Your browser knows nothing about these tags, certainly. However, Varnish is capable of processing these tags properly. For each tag Varnish may make a request to the server to fetch the contains that we replaced with this tag. I said “may” because Varnish is a caching proxy and next time it will start looking in its cache storage first.

Cache data provider

Let’s look closer on the ESI tag I used:

<esi:include src="/cache/get?key=2_5_"/>

this URL is autogenerated by my JSP custom tag library. Varnish makes additional requests to the backend to retrieve the page fragments and cache them to speed up the next similar page requests. Cache Data Provider is a separate server whose solr purpose is to provide the data from NoSQL shared storage (MongoDB) to the requestor (Varnish).  The key is specified as a parameter (key). Note that the whole system operates with the different types of caches: browser cache, Varnish cache, Hybris cache, MongoDB cache. We added two of them (MongoDB and added). There are the following cache cases:

Varnish cache	MongoDB cache	Issue	Solution
Not presents/Stale	Presents	It is a typical first query or a query after TTL is over	Cache the fragment in Varnish
Presents	Presents or not presents	It is a repeated query (within TTL)	Varnish shouldn’t make a request to MongoDB Cache provider because the cached fragment had been successfully retrieved before. So this is why we shouldn’t care about MongoDB cache for this case
Not presents/stale	Not presents/stale	No data neither in MongoDB nor in Varnish	Varnish won’t be able to retrieve and deliver the fragment. This case is a bad case, actually. See below for the solution

Varnish cache – Mongo DB cache conflict resolution

If the fragment is removed from both caches (Varnish and MongoDB) there are no ways to retrieve it from hybris separately and that is an issue. In some situations these fragments can be generated separately, but you need to implement specific controllers for different types of the fragments. For example, for CMS content slots or for CMS components this implementation is trivial (and I suspect that hybris Professional Services went this way), but for the controller-related page fragments it is not so. The bad point is that the custom functionality for the latter is very project-specific. Let look deeper into this point.  To retrieve these fragments the hybris Professional Service’s extension possibly uses a custom controller to render CMS fragments separately of the page where they are placed. However, there are three issues with this approach.

• The first issue is about the “filters-page controller” interaction. Sometimes some data are prepared by custom/hybris/spring filters to be used in the page or component controller. So it means that filters must be used for each cache request. If you skip the filters, the rendered components might have a different look.
• The second issue is about resource-intensive filters, like BTG (personalization). Once you replace one slow request with (say) 10 slow, but cacheable requests, the efficiency might be questionable.
• Sometimes the cached fragments may depend on each other, on other components or the same shared information like javascript variables.

How to overcome this issues? My solution is very simple. When the issue happens?

1. when both the controller page’s TTL is longer than fragments’
2. and when the controller page is also cached.

These points is for plain structure, with no nested cacheable fragments. For nested ESIs replace “controller page” with “parent ESI” in the points above. If the controller page is not cached, each request from the user to the server will re-create the ESI fragments in MongoDB. If so, the case #3 will never happen. If the controller page’s TTL is not longer than fragments’ TTLs, the fragments TTL will never be removed from MongoDB. So the solution is to set TTL for controller page as a min (fragments TTLs). The same is for nested cacheable fragments: the TTL of parent ESI is calculated based on TTLs of the nested ESI which are specified as a tag parameter or calculated based on their substructures. For example, the product page has the following structure: So it means that the TTL of the product page will be calculated as min (10,3,5) =3. Certainly, you can programmatically set the TTL of the controller to “0” to get rid of the controller caching.

Varnish cache configuration

The configuration below is oversimplified – just to illustrate an approach. The backend response caching depends on requested URL and my custom X-VarnishCache header:

vcl 4.0;
import std;

backend default {
 .host = "electronics.local";
 .port = "9001";
 .connect_timeout = 60s;
 .first_byte_timeout = 60s;
 .between_bytes_timeout = 60s;
 .max_connections = 800;
}

sub vcl_recv {
 return (hash);
}
sub vcl_backend_response {

    if (bereq.url ~ "/cache/get") {
        set beresp.ttl = std.duration(beresp.http.X-VarnishCache+"s",0s);
    }
    else
    {
        set beresp.do_esi = true;
        set beresp.ttl = std.duration(beresp.http.X-VarnishCache+"s",0s);
     }
    return (deliver);
 }

Video (PoC)

The video demonstrates that the request rate on the application server side is 6 times less than the request rate from the browser. the high-resolution version of the diagram is here © Rauf Aliev, July 2016