Prerendering pages in browsers

Posted: Sep 27, 2016

I guess lots of people might have heard about the prerender feature supplied by most of the browsers. The idea of it is very simple. A site can tell the browser which the next page should be prerendered for the current user. As a result, when the user visits the prerendered page, most likely assets are downloaded, the DOM is rendered, hence, the user gets the instant transition. Ok, maybe this claim isn't correct. If the backend is very slow, prerendering won't help to achieve the instant transition, but, for others, it may be achievable.

To see how it works let's make a simple experiment. Let's try it out on the official site of Mozilla. Once you've opened it, execute this code in the console of your browser:

var link = document.createElement('link');
link.setAttribute('href', 'https://www.mozilla.org/en-US/firefox/products/');
link.setAttribute('rel', 'prerender');

var head = document.querySelector('head');
head.appendChild(link);

Once you've clicked on the firefox item of the top menu, you will see almost instant load of the page. Unfortunately, this feature isn't supported by all browsers, therefore, if you decide to try it out, you have to choose a browser which supports prerendering.

It worked for me like this:

Here how it worked without prerendering:

As you can see the example prerendering the page feels much better. Please, don't think this feature can solve performance problems. If the backend of a site is very slow, the browser may not be able to prerender the page before the user visits it. Actually, it depends on the time the user stays on the current page. For example, if the current page contains an article or a video, the user may stay there for a few minutes, in this case prerendering may help to give better experience even if the site is slow, but it isn't good to rely on this assumption.

Prediction

The prerender feature looks interesting, but, how can we predict the next page which may be visited by the current user? You can use Google Analytics (or any other analytics tool) to see how users navigate your site. Then you can manually add the link tags on pages in order to tell browsers which pages should be prerendered. But, it involves manual setup and you will have to support it. If the structure of your site changes, you will have to manually change the link tags. It doesn't seem good.

Fortunately, we can easily automate this process. First, we need to understand how users navigate the site. To do that we need to identify users' sessions. The session is a list of visited pages by a particular user. Again, you can extract this info from your analytics tool or you can implement a tracking mechanism on your own. Let's imagine we gathered following data:

Real sites have more pages and sessions are more diverse, but, to keep it simple, let's say the site has only 5 pages. Let's draw it to see the big picture.

Now it is clear what happens here. Users visiting the index page have 2 options either they proceed to the products page or the about page. Most of the users choose the products page. Once they've visited the products page, either they proceed to the contribute page or the index page. This guess doesn't feel right, because 31 users entered the products page and only 10 of them proceeded to the contribute page and 2 of them proceeded to the index page, hence, 19 users left the site on the products page.

The graph above shows that users use different pages to enter the site. Some of them start with the products page, others start with the index page. The same behavior we see when they leave the site. To demonstrate it on the graph, we need some virtual starting and exit points.

Now this graph shows which pages are used to enter and exit the site. We can correctly predict users' navigation. For example, most of the users entering the index page proceed to the products page, afterwards, most of them exit the site. It fixes the problem which we've seen with the first graph. Understanding exists of users will help to avoid prerendering when it doesn't make any sense. Prerendering of incorrect pages will lead to additional load on the backend without profit. Also, it may significantly drop the battery of users on mobile devices.

This graph can be converted to the transition probability matrix:

The first column represents the current page, the head of the table shows potential candidates. If the user is on the index page, there is 93.75% of probability the user will proceed to the products page.

In general, some graph DB is a good solution for this challenge. Pages will be represented by nodes, transitions will be represented by relations between nodes. The graph DB can be used to make predictions and to track sessions. For example, to make predictions we use the transition relation, to track sessions we use the session relation. Once the session gets expired, we increase the count of transitions on the transition relation.

For instance, here, once the session gets expired (green arrow), the transition relation (blue arrow) between the start and the products nodes will be increased by 1 and it will equal 13. The same operations should be performed to other transition relations matching the path of the session.

Inactive sessions can be expired after some time. To expire the inactive session, a new relation between the last visited page and the exit point has to be created.

When this model and prerendering doesn't work

Actually, it is easier to answer when it does work. We can only prerender the page if it gets loaded via the GET request. The pages which are loaded via the POST request or any other requests modifying state should not be tracked, hence, they cannot be predicted.

Unfortunately, this model cannot predict when users visiting the products page would proceed to the contribute page (there are some users doing that). To answer this question, we need more analytics data, it requires more personalized analytics. Anyway, using this simple model and the prerender feature we can give better experience to most of the users. It won't hurt if it doesn't work for others.

UPD (Feb 28, 2017): Last months I've been working on implementing a programming solution for the idea described above. If you would like to try it, it is accessible in the GitHub repo. Let me know how it works for you.