Gilberto Cocchi on web.dev

Automating audits with AutoWebPerf

2020-12-09T00:00:00Z

What is AutoWebPerf (AWP)? #

AutoWebPerf (AWP) is a modular tool that enables automatic gathering of performance data from multiple sources. Currently there are many tools available to measure website performance for different scopes (lab and field), such as Chrome UX Report, PageSpeed Insights, or WebPageTest. AWP offers integration with various audit tools with a simple setup so you can continuously monitor the site performance in one place.

The release of Web Vitals guidance means that close and active monitoring of web pages is becoming increasingly important. The engineers behind this tool have been doing performance audits for years and they created AWP to automate a manual, recurring, and time consuming part of their daily activities. Today, AWP has reached a level of maturity and it's ready to be shared broadly so anyone can benefit from the automation it brings.

The tool is accessible on the AutoWebPerf public repository on GitHub.

What is AWP for? #

Although several tools and APIs are available to monitor the performance of web pages, most of them expose data measured at a specific time. To adequately monitor a website and maintain good performance of key pages, it's recommended to continuously take measurements of Core Web Vitals over time and observe trends.

AWP makes that easier by providing an engine and pre-built API integrations which can be programmatically configured to automate recurrent queries to various performance monitoring APIs.

For example, with AWP, you can set a daily test on your home page to capture the field data from CrUX API and lab data from a Lighthouse report from PageSpeed Insights. This data can be written and stored over time, for example, in Google Sheets and then visualised in the Data Studio dashboard. AWP automates the heavy-lifting part of the entire process, making it a great solution to follow lab and field trends over time. See Visualising audit results in Data Studio below for more details).

Architecture overview #

AWP is a modular-based library with three different types of modules:

the engine
connector modules
gatherer modules

The engine takes a list of tests from a connector (for example, from a local CSV file), runs performance audits through selected gatherers (such as PageSpeed Insights), and writes results to the output connector (for example, Google Sheets).

AWP comes with a number of pre-implemented gatherers and connectors:

Pre-implemented gatherers:
Pre-implemented connectors:
- Google Sheets
- JSON
- CSV

Automating audits with AWP #

AWP automates the performance audits via your preferred audit platforms such as PageSpeed Insights, WebPageTest, or CrUX API. AWP offers the flexibility to choose where to load the list of tests, and where to write the results to.

For example, you can run audits for a list of tests stored in a Google Sheet, and write the results to a CSV file, with the command below:

PSI_APIKEY=<YOUR_KEY> SHEETS_APIKEY=<YOUR_KEY> ./awp run sheets:<SheetID> csv:output.csv

Recurring audits #

You can run recurring audits in daily, weekly, or monthly frequency. For example, you can run daily audits for a list of tests defined in a local JSON like below:

{
  "tests": [
    {
      "label": "web.dev",
      "url": "https://web.dev",
      "gatherer": "psi"
    }
  ]
}

The command below reads the list of audit tests from the local JSON file, runs audits on a local machine, then outputs results to a local CSV file:

PSI_APIKEY=<YOUR_KEY> ./awp run json:tests.json csv:output.csv

To run audits every day as a background service continuously, you can use the command below instead:

PSI_APIKEY=<YOUR_KEY> ./awp continue json:tests.json csv:output.csv

Alternatively, you can set up the crontab in a Unix-like environment to run AWP as a daily cron job:

0 0 * * * PSI_APIKEY=<YOUR_KEY> ./awp run json:tests.json csv:output.csv

You can find more ways to automate daily audits and result collection in the AWP GitHub repository.

Visualising audit results in Data Studio #

Along with continuously measuring Core Web Vitals, it is important to be able to evaluate the trends and discover potential regressions with real user metrics (RUM) or the Chrome UX Report (CrUX) data collected by AWP. Note that Chrome UX Report (CrUX) is a 28-day moving aggregation, hence it is recommended to also use your own RUM data along with CrUX so you can spot regressions sooner.

Data Studio is a free visualization tool that you can easily load performance metrics into and draw trends as charts. For example, the time series charts below show Core Web Vitals based on Chrome UX Report data. One of the charts shows increasing Cumulative Layout Shift in recent weeks, which means regressions in the layout stability for certain pages. In this scenario, you would want to prioritize the efforts to analyze the underlying issues of these pages.

To simplify the end-to-end process from data collection to visualization, you can run AWP with a list of URLs to automatically export results to Google Sheets with the following command:

PSI_APIKEY=<YOUR_KEY> SHEETS_APIKEY=<YOUR_KEY> ./awp run sheets:<SheetID> csv:output.csv

After collecting daily metrics in a spreadsheet, you can create a Data Studio dashboard that loads the data directly from the spreadsheet, and plots the trends into a time series chart. Check out Google Spreadsheets API Connector for detailed steps about how to set up AWP with spreadsheets as a data source to visualize on Data Studio.

What's next? #

AWP provides a simple and integrated way to minimize the efforts to set up a continuous monitoring pipeline to measure Core Web Vitals and other performance metrics. As for now, AWP covers the most common use cases and will continue to provide more features to address other use cases in the future.

Learn more in the AutoWebPerf repository.

Instant navigation experiences

2020-06-23T00:00:00Z

Performing a task on a site commonly involves several steps. For example, purchasing a product in an e-commerce website might involve searching for a product, picking an item from the list of results, adding the item to the cart, and completing the operation by checking out.

In technical terms, moving through different pages means making a navigation request. As a general rule, you don't want to use long-lived Cache-Control headers to cache the HTML response for a navigation request. They should normally be satisfied via the network, with Cache-Control: no-cache, to ensure that the HTML, along with the chain of subsequent network requests, is (reasonably) fresh. Having to go against the network each time the user navigates to a new page unfortunately means that each navigation might be slow—at the very least, it means that it won't be reliably fast.

To speed up these requests, if you can anticipate the user's action, you can request these pages and assets beforehand and keep them in the cache for a short period of time until the user clicks on these links. This technique is called prefetching and it's commonly implemented by adding <link rel="prefetch"> tags to pages, indicating the resource to prefetch.

In this guide we'll explore different ways in which service workers can be used as a complement of traditional prefetching techniques.

Production cases #

MercadoLibre is the biggest e-commerce site in Latin America. To speed up navigations, they dynamically inject <link rel="prefetch"> tags in some parts of the flow. For example, in listing pages, they fetch the next result page as soon as the user scrolls to the bottom of the listing:

Prefetched files are requested at the "Lowest" priority and stored in the HTTP cache or the memory cache (depending on whether the resource is cacheable or not), for an amount of time that varies by browsers. For example, as of Chrome 85, this value is 5 minutes. Resources are kept around for five minutes, after which the normal Cache-Control rules for the resource apply.

Using service worker caching can help you extend the lifetime of prefetch resources beyond the five-minute window.

For example, Italian sports portal Virgilio Sport uses service workers to prefetch the most popular posts in their home page. They also use the Network Information API to avoid prefetching for users that are on a 2G connection.

As a result of this, over 3 weeks of observation Virgilio Sport witnessed load times for navigation to articles improve 78%, and the number of article impressions increase 45%.

Implement precaching with Workbox #

In the following section we'll use Workbox to show how to implement different caching techniques in the service worker that can be used as a complement to <link rel="prefetch">, or even a replacement for it, by delegating this task completely to the service worker.

1. Precache static pages and page subresources #

Precaching is the ability of the service worker to save files to the cache while it's installing.

In the following cases precaching is used to achieve a similar goal as prefetching: making navigations faster.

Precaching static pages #

For pages that are generated at build time (e.g. about.html, contact.html), or in completely static sites, one can just add the site's documents to the precache list, so they are already available in the cache every time the user accesses them:

workbox.precaching.precacheAndRoute([
  {url: '/about.html', revision: 'abcd1234'},
  // ... other entries ...
]);

Precaching page subresources #

Precaching static assets that the different sections of the site might use (e.g. JavaScript, CSS, etc.), is a general best practice and can give an extra boost in prefetching scenarios.

To speed up navigations in an e-commerce site, you can use <link rel="prefetch"> tags in listing pages to prefetch product detail pages for the first few products of a listing page. If you have already precached the product page subresources, this can make the navigation even faster.

To implement this:

Add a <link rel="prefetch"> tag to the page:

 <link rel="prefetch" href="/phones/smartphone-5x.html" as="document">

Add the page subresources to the precache list in the service worker:

workbox.precaching.precacheAndRoute([
  '/styles/product-page.ac29.css',
  // ... other entries ...
]);

2. Extend the lifetime of prefetch resources #

As mentioned earlier, <link rel="prefetch"> fetches and keeps resources in the HTTP cache for a limited amount of time, after which point the Cache-Control rules for a resource apply. As of Chrome 85, this value is 5 minutes.

Service workers allow you to extend the lifetime of the prefetch pages, while providing the added benefit of making those resources available for offline usage.

In the previous example, one could complement the <link rel="prefetch"> used to prefetch a product page with a Workbox runtime caching strategy.

To implement that:

Add a <link rel="prefetch"> tag to the page:

 <link rel="prefetch" href="/phones/smartphone-5x.html" as="document">

Implement a runtime caching strategy in the service worker for these types of requests:

new workbox.strategies.StaleWhileRevalidate({
  cacheName: 'document-cache',
  plugins: [
    new workbox.expiration.Plugin({
      maxAgeSeconds: 30 * 24 * 60 * 60, // 30 Days
    }),
  ],
});

In this case, we have opted to use a stale-while-revalidate strategy. In this strategy, pages can be requested from both the cache and the network, in parallel. The response comes from the cache if available, otherwise from the network. The cache is always kept up to date with the network response with each successful request.

3. Delegate prefetching to the service worker #

In most cases the best approach is to use <link rel="prefetch">. The tag is a resource hint designed to make prefetching as efficient as possible.

In some cases, though, it might be better to delegate this task completely to the service worker. For example: to prefetch the first few products in a client-side rendered product listing page, one might need to inject several <link rel="prefetch"> tags dynamically in the page, based on an API response. This can momentarily consume time on the page's main thread and make the implementation more difficult.

In cases like this, use a "page to service worker communication strategy", to delegate the task of prefetching completely to the service worker. This type of communication can be achieved by using worker.postMessage():

The Workbox Window package simplifies this type of communication, abstracting many details of the underlying call being done.

Prefetching with Workbox Window can be implemented in the following way:

In the page: call the service worker passing it the type of message, and the list of URLs to prefetch:

const wb = new Workbox('/sw.js');
wb.register();

const prefetchResponse = await wb.messageSW({type: 'PREFETCH_URLS', urls: […]});

In the service worker: implement a message handler to issue a fetch() request for each URL to prefetch:

addEventListener('message', (event) => {
  if (event.data.type === 'PREFETCH_URLS') {
    // Fetch URLs and store them in the cache
  }
});