Antoine Bisch on web.dev

How Renault improved its bounce and conversion rates by measuring and optimizing Largest Contentful Paint

2021-10-08T00:00:00Z

Groupe Renault is a French multinational automobile manufacturer with a presence in over 130 countries. For an automotive group such as Renault, performant brand sites driving more user engagement and conversions mean more business. All of its brand sites aim at providing the best user experience at scale while maintaining flexibility of content and features for localized sites. In this context, performance monitoring is a key stake for the customer experience team which is in charge of developing and maintaining the global platform.

Measuring the business impact of Core Web Vitals #

Measurement in Google Analytics #

Working with fifty-five, its global data partner, Renault set up the web-vitals library, which allows sending to Google Analytics all the Web Vitals metrics from real users in a way that accurately matches how they're measured by Chrome and reported to other Google tools.

The following analysis showcases a dataset captured using these tools over four months between December 2020 and March 2021.

Optimized LCP strongly correlates with user engagement and business metrics #

The teams have identified a particularly strong correlation between a low Largest Contentful Paint (LCP) and favorable bounce rates and conversion rates, shown in the visualization below.

The dataset captures over 10 million visits in 33 countries over four months and shows how lower LCP measures correlates with:

Lower bounce rates
More conversions (lead forms completed)

Interestingly, as the website runs as a Single Page Application (SPA), all these measures are captured on landing pages only. The data shows that it is worth optimizing the website until the LCP reaches below 1 second. The group's brand sites can never be too optimized!

This dataset not only shows the negative correlation between LCP and business metrics, but also highlights performance discrepancies among the best performing landing pages. In the context of this website, getting LCP under 1 second leads to large increases in conversions and reductions in bounce.

Eja Rakotoarimanana, Consultant, fifty-five

1 second LCP improvement can lead to a 14 percentage points (ppt) decrease in bounce rate and 13% increase in conversions.

1 second LCP improvement	Result
LCP around 1 s	+13% CVR
LCP under 1.6 s	-14 ppt bounce rate
LCP above 1.6 s	-5 ppt bounce rate

Renault's approach to optimizing Core Web Vitals at scale #

Since early 2020, in the brand's top 5 European markets, the number of visitors experiencing a fast LCP (under 2.5 s) has improved by an average of 22 ppt for Renault domains (from 51% to 73%).

Here is how they've approached it.

A central optimization of the SPA #

From a platform standpoint, performance has been a priority for years and including Core Web Vitals (CWV) as key metrics was a smooth process. Central teams have set up a comprehensive monitoring solution (with Google Lighthouse and Chrome UX Report API) and established a performance culture across the organization. There were several strategies to optimize their Single Page Application, including:

Server side rendering (SSR) to ensure a fast First Contentful Paint (FCP).
Code splitting to deliver only the JS and CSS chunks needed for the landing page (for better LCP and FID).
CDN with a high level of resource caching (including a Lambda@Edge to sort and remove unnecessary query parameters). This helped avoid the drawbacks of SSR (slower TTFB due to server computation) and deliver content closer to the final user (for better TTFB and LCP).
Optimizing compression with brotli to reduce the code size.
HTTP2 to enable multiplexing of requests and responses.
Using responsive images with WebP support and srcset and sizes attribute to serve the most appropriate image size and format to users.
Lazy loading images, videos, and iframes using IntersectionObserver and FPOs (small 1 KB thumbnails).
Removing blocking scripts and adjusting transpilation to browser targets to reduce JS file sizes (by avoiding unnecessary polyfills).
Reducing the size of the Google Tag Manager container to load third-party scripts only where and when needed.
Reducing the number of custom fonts, using woff/woff2 formats with unicode-range, and font-display:swap to reduce font file sizes and show text as soon as possible even if custom fonts are not yet available.
Preloading hero images which are often LCP elements.

The team is still working on future improvements such as:

Server push to improve FCP by delivering CSS faster. (On standby due to lack of AWS support and deprecation proposal.
Progressive hydration to improve FID.
ES6 Module support to deliver a faster experience by using ES6 builds for modern browsers.

The SPA approach can be beneficial for performance as a full page reload is not necessary when users navigate across pages. That being said, current CWV measurement methodology on SPA can be perceived as a downside as route transitions are not being measured; hence comparatively faster page loads within a session, due to UI caching, are not taken into account. It also makes it difficult to compare CWV metrics against a Multi Page Application competitor website where a warm cache would bring the measures down on each page a user would browse during a session. Read the Web Vitals SPA FAQ for more details.

These are known limitations that are being investigated by the Chrome product teams. An update to the CLS metric has already been shipped to improve measurement on SPAs.

Performance requires constant monitoring as various technical teams can impact it. Despite limitations in how they are measured on SPAs, CWV metrics allow us to track the impact of the actions our teams take. Hopefully route transitions will be taken into account soon!

Cedric Bazureau, Tech Lead, Renault

Local guidelines promoting performance optimization as a shared responsibility #

Performance is communicated as both a global (central) and local responsibility. Teams have put together a series of best practices that local content owners are meant to follow. Here are a few examples from these guidelines:

Optimize local Google Tag Manager container to improve site performance. for example conditionally trigger certain tags.
Limit size of video content by compressing it using internal tools or hosting it on a external platform (such as YouTube).
Avoid uploading pictures via Google Tag Manager.

A deep understanding of our digital performance is key to ensuring a continuous optimization of our brand sites. Our customer experience team's approach is to provide a global platform that positively impacts local teams' business outcomes while empowering these teams with guidelines and best practices to maintain this performance at a high level.

Alexandre Perruche, Head of Performance, Renault

To conclude, website performance has always been a priority at Renault, and its website platform is continuously optimized. Measuring Core Web Vitals alongside business metrics has allowed them to promote this topic as a globally shared responsibility, and local guidelines equip teams with the ability to participate in this beneficial effort.

Core Web Vitals workflows with Google tools

2021-08-09T00:00:00Z

Core Web Vitals are a set of metrics that assess the user experience on criteria such as load performance, responsiveness to user input, and layout stability.

A workflow for improving Core Web Vitals for your website will be explored in this guide, but where that workflow begins depends on whether you're collecting your own field data. Where it ends may depend on which of Google's tools you'll find useful in diagnosing and fixing user experience problems.

Core Web Vitals are best measured in the field #

Core Web Vitals are specifically designed to measure how users experience your website—they are user centric metrics. Lab-based tools such as Lighthouse are diagnostic tools to highlight potential performance problems and best practices. Lab-based tools are run under certain, predefined conditions and may not reflect the real-life Core Web Vitals measurements your users experience.

For example, Lighthouse is a lab-based tool that runs tests with simulated throttling in a simulated desktop or mobile environment. While such simulations of slower network and device conditions are helpful when trying to diagnose performance problems, they're just a single slice of the large variety in network conditions and device capabilities, and so may not reflect what users on your sites are experiening.

Lab-based tools like Lighthouse also typically do a "cold load" of a webpage as a totally new visitor. This is often the slowest load, but in real life, visitors may have some assets cached if they have visited before, or when they are browsing around the site. New visitors, and tools, also may experience the site differently with cookie banners or other content present to them.

In short, while lab-based tools can give an indication of potential performance problems and help you debug and iterate, they may not represent how many of your visitors actually experience your website. Use field data for measuring real-world performance, and lab-based tools like Lighthouse for diagnostics of how to improve it. See also the When to use Lighthouse section.

Google measures Core Web Vitals through the Chrome User Experience Report (CrUX). This is a public dataset collected from real Chrome users. It is the backbone of many Google and third-party tools which report a site's Core Web Vitals.

CrUX has its limitations, though. It can often tell you when there is a problem, but often has insufficient data to tell you why.

Collect your own field data if possible #

The best dataset for improving website performance in the field is the one you build. That starts with collecting field data from your website's visitors. How you do this depends on the size of your organization, and whether you want to pay for a third-party solution or create your own.

Paid solutions will almost certainly measure Core Web Vitals (and other performance metrics) and usually provide a variety of tools to dig into the resulting data. In large organizations with significant resources, this may be the preferred method.

However, you may not be part of a large organization—or even one with the means to afford a third-party solution. In these cases, Google's web-vitals library will help you gather all Web Vitals. However, you'll be responsible for how that data is reported, stored, and analyzed.

If you're already using Google Analytics, but you haven't started collecting your own field data, there may be an opportunity for you to use the web-vitals library to send Web Vitals collected in the field to Google Analytics and use the BigQuery exports of GA4 to report on the data.

Understanding Google's tools #

Regardless of whether you're collecting your own field data, there are several Google tools that may be useful in analyzing Core Web Vitals. Before establishing a workflow, a high-level overview of each tool can help you to understand which tools may—or may not—be best for you.

Chrome User Experience Report (CrUX) #

As mentioned previously, CrUX is a public data set of field data gathered from a segment of real Google Chrome users from millions of websites. It includes Core Web Vital metrics and other metrics for websites with sufficient traffic.

CrUX is available as a monthly BigQuery dataset at the origin level, or as a daily API at the URL or origin-level, provided a URL or origin has enough samples in the CrUX dataset. The BigQuery data is also viewable in an easy to use CrUX Dashboard allowing sites to review historical trends for their site.

When to use CrUX #

Even if you gather your own field data, CrUX is still useful. Though CrUX represents a subset of Chrome users, it's helpful to compare your website's field data to see how it aligns with its CrUX data. There are advantages and disadvantages to each, which can result in differences. If you currently don't gather any field data for your website, CrUX is especially valuable to provide a high-level overview—provided your website is represented in its dataset.

You can use CrUX directly, or via another tool (including those mentioned below). Using the CrUX dataset directly, either via BigQuery or the API, is useful to surface data that is not currently shown in other tools—for example country-level data is often not available on other tools, or to view the additional metrics in CrUX which again are often not surfaced in other tooling.

When not to use CrUX #

CrUX only represents Chrome users, and even then, only a subset of Chrome users. A full RUM solution can include more experiences across Chrome and other browsers where they support the Web Vital metrics.

Websites that don't receive enough traffic are not represented in the CrUX dataset. If this is the case for you, you'll need to gather your own field data to understand how your website performs in the field, as CrUX won't be an option. Alternatively, you will need to depend on lab data, but with the limitations that it may not be representative above.

Since the data CrUX provides is a rolling average over the previous 28 days, it's not an ideal tool during development, as it will take a fair amount of time for improvements to be reflected in the CrUX dataset.

Finally, as a public dataset, CrUX is limited to how much information it can make available, and how this data can be queried. Capturing your own RUM data allows you to gather more details (for example, the LCP element), and slice and dice the data more to identify problems. Do logged in users experience better or worse Core Web Vitals than logged out users? Do users with a slow LCP have a particular LCP element? Which interactions are causing high FID and INP values?

PageSpeed Insights (PSI) #

PSI is a tool that reports field data from CrUX and lab from Lighthouse for a given page. See those individual sections for more details.

When to use PSI #

PSI is great for assessing CrUX performance at the page level or origin-level, for both mobile and desktop users. It's a good choice for an initial overview of Core Web Vitals for a page or site. It also allows you to easily view Core Web Vitals data for other sites like competitors.

PSI also provides Lighthouse data, which gives useful recommendations to improve your Core Web Vitals—if the metrics align. Where these do not align, Lighthouse recommendations may be less relevant.

Since Lighthouse is run from the server, it can form a more consistent baseline than running Lighthouse from DevTools.

When not to use PSI #

PSI is only available for public URLs. It cannot be used on development sites that are not publicly accessible.

CrUX data is only available when sites meet certain eligibility criteria, including site popularity thresholds. PSI is less useful when CrUX data is not available for a page or origin as it can only show the Lighthouse lab data in these cases.

Similarly, if you only have origin-level CrUX data rather than the specific URL being tested, then this also limits its usefulness of correlating the origin-level field data to the page-level lab diagnostics. Having the origin-level field data is still very useful information to have as a summary of the site's performance and the Lighthouse audits may help, but extra caution should be used in this case.

Finally, where page-level data is available in CrUX, but differs from the Lighthouse lab data, recommendations from Lighthouse may be of limited value. This can happen particularly for post-load CLS issues, and for interactivity Core Web Vitals (FID and INP) where lab-based audits are less useful.

Search Console #

Search Console measures your site's search traffic and performance, including Core Web Vitals. It is only available to site owners who have confirmed their ownership of the site.

A valuable feature of Search Console is that it groups similar pages (for example, pages that use the same template) into a single group assessment. Search Console also includes a Core Web Vitals report based on field data from CrUX.

When to use Search Console #

Search Console is well-suited for both developers and those in non-developer roles to assess both search and page performance in ways other Google tools don't. Its presentation of CrUX data and grouping of pages by similarity offers novel insight into how performance improvements impact entire categories of pages.

When not to use Search Console #

Search Console may not be a fit for projects that use different third-party tools which group pages by similarity, or if a website isn't represented in the CrUX dataset.

Page grouping can also be somewhat confusing when the example pages in a group have different characteristics than the rest of the group—for example, if the group fails particular Core Web Vitals overall, but the example pages all seem to pass the same Core Web Vitals. This can happen when a group contains a long tail or seldomly visited pages that may be slower to load, as they're less likely to be cached. When there are sufficient volumes of these pages in the long tail, they can influence the group's overall pass rate.

Lighthouse #

Lighthouse is a lab tool that provides specific opportunities for improving page performance. Lighthouse user flows also allow developers to script interaction flows for performance testing beyond page load.

Lighthouse-CI is a related tool that runs Lighthouse during project builds and deploys to assist with performance regression testing. It presents a Lighthouse report along with pull requests, and tracks performance metrics over time.

When to use Lighthouse #

Lighthouse is excellent for finding performance improvement opportunities during development in both local and staging environments. Lighthouse CI is similarly useful in the build and deploy phases to staging and production environments, where performance regression testing is needed to preserve good user experiences.

When not to use Lighthouse #

Lighthouse (or Lighthouse CI) is not a substitute for field data. Lighthouse is primarily a diagnostic tool listing potential issues and best practices from a predefined page load. The recommendations it surfaces may not always match the performance experienced by your users.

While Lighthouse can be used to diagnose production sites through tools like PageSpeed Insights, Lighthouse would ideally be used in development and continuous integration environments to address performance issues before they reach production.

Web Vitals extension #

The Web Vitals Chrome extension is a diagnostic tool that surfaces Core Web Vitals metrics as you browse the web. It also includes CrUX data for the current page if it is represented in the CrUX dataset, and provides debugging information to help you identify Core Web Vitals performance problems.

When to use the Web Vitals extension #

The Web Vitals extension can be used by anyone in any role to assess a page's Core Web Vitals at all points of the page lifecycle. It is useful as a "live" view of performance as you interact with the page to attempt to uncover performance issues—particularly for post-load issues you might see with the CLS and INP metrics.

When not to use the Web Vitals extension #

The Web Vitals extension isn't a holistic assessment of page performance. In addition, the metrics it reports are highly dependent the environment in which it runs, and developers often have higher powered machines or access to faster networks.

The Performance panel in Chrome DevTools #

Chrome DevTools is a collection of in-browser development tools, including the Performance panel. The Performance panel is a lab tool that profiles all page activity during page load or a recorded time period. It offers deep insight into everything it observes across dimensions such as network, rendering, painting, and scripting activity, as well as a page's Core Web Vitals.

When to use the Performance panel #

The Performance panel should be used by developers during development to gain deep insight into page performance. This is particularly useful to debug responsiveness issues affecting FID or INP. Once a poorly responding interaction is identified and repeatable, the Performance panel can provide a wealth of data as to what is going on in the browser to help understand the issue, from main thread blocking, to JavaScript call stacks, to rendering work.

When not to use the Performance panel #

The Performance panel is a developer tool that provides lab data only. It's not a substitute for field data. It contains a lot of debugging information, but because of that, it may not be easy to use for novice developers or those in non-developer roles.

A three step workflow for ensuring your website's Core Web Vitals stay healthy #

When working to improve the user experience, it's best to think of the process as a continuous cycle. For improving Core Web Vitals and other performance metrics, one approach could be:

Evaluate website health and identify pain points.
Debug and optimize.
Monitor with continuous integration tools to catch and prevent regressions.

Step 1: Evaluate website health and identify opportunities for improvement #

It is best to start with field data to evaluate website health.

Use PageSpeed Insights to view overall Core Web Vitals experience metrics on the origin, and specific information on an individual URL.
Search Console can be useful to identify pages which need improvement where its page grouping feature works well for your site.
If you have RUM data, then that is often the best option to be able to identify particular pages or traffic segments with issues.

Whether you analyze field data you collect yourself or CrUX data, this first step is vital. If you're not gathering field data, CrUX data may be enough to guide you—again, provided your website is represented in the dataset.

Analyze site performance with PageSpeed Insights #

PageSpeed Insights displays the CrUX data covering the last 28 days of user experience data at the 75th percentile. This means that if 75% of user experiences meet the threshold set for a given metric, then the experience is considered "good".

If you have a specific page in mind to look at the performance of, then use that. For an overall view of a site when you first start optimizing, you may wish to start with the home page, as it is typically one of the most popular pages on many sites.

Concentrate on the what your real users are experiencing section of PSI initially. You will see up to four views of the data: mobile and desktop for the URL entered and the whole origin. Compare these and see how they differ. Mobile is typically less performant than desktop since it is a more resource-constrained device operating under potentially less stable network conditions. If the URL and origin data are significantly different, then try to understand why: home pages are often the first pages visited (that is, a landing page) so can be slower than the origin users take the full brunt of an unprimed browser cache. Subsequent pages will likely load faster, as any shared assets will be cached, bringing down the aggregate origin-level data.

PSI also shows all three Core Web Vitals (LCP, CLS, and FID) and the pending INP metric, plus the diagnostic TTFB and FCP metrics. Are any of the Core Web Vitals failing, and by how much? This will indicate where to concentrate your efforts.

Understand the relationships between these numbers—particularly for LCP. If LCP is slow, as it is in this example, then look at TTFB and FCP which are both milestones to that metric. In this example we have a 1.8 second TTFB, which is going to make it very tough to meet the 2.5 second recommended threshold for good LCP. This suggests either a slow backend (server issues or a lack of CDN), slower networks, or redirects delaying the first HTML bytes. Look at the Optimize TTFB guide for more information. FCP takes another second on top of that, which again may be indicative of slower networks. LCP is not long after FCP in this example suggesting the LCP resource is well optimized once the page itself loads.

For CLS, look at the CrUX CLS and the Lighthouse CLS scores to see if this is a load CLS issue (which Lighthouse will catch and advise on), or a post-load CLS issue that Lighthouse won't catch. More for information see the Optimize CLS guide.

For responsiveness, look at the FID and INP scores. Look at the TBT audits in Lighthouse to see if a lot of JavaScript processing is happening during the initial page load, which is likely to impact INP. INP can be a tricky metric to improve, so consult the Optimize INP guide for more information.

Identify poorly performing pages in Search Console #

While PSI is useful when you have a specific URL you want to test or the site as a whole, Search Console can help target your efforts to particular types of pages. This is particularly useful if many pages share common themes or technologies and Search Console can successfully identify these.

The Core Web Vitals report in Search Console shows the big picture of your website's performance, but you can still drill down into specific pages that need attention. With Search Console, you can also:

Identify individual page groups that need improvement, and those that currently provide a good user experience.
Get granular data on performance by URL grouped by status, metric, and groups of similar web pages (such as product detail pages on an e-commerce website).
Get detailed reports that bucket URLs in each user experience quality category for both mobile and desktop.

Once you have some specific pages to look at, you can use PSI as explained previously to gather further understanding of the issues for those pages.

Step 2: Debug and optimize #

In step 1, you should have identified pages which require performance improvements, and also which of the Core Web Vitals metrics you want to improve. You can use the Google tooling to get further information to understand the root cause to identify the issue.

Run a Lighthouse audit to get page-level guidance
Use the Web Vitals extension to analyze Core Web Vitals in real time.
Use the Performance panel in Chrome DevTools to debug performance issues and test code changes.

For more detailed guidance, see these guides:

Uncover opportunities with Lighthouse #

PageSpeed Insights runs Lighthouse for you, but for local development it is also possible to run Lighthouse from Chrome DevTools, which is useful to validate fixes locally.

A key point is to validate that the Lighthouse audit replicates the issues you are trying to solve (for example, slow LCP, or CLS issues). Out of the box, Lighthouse only assesses the user experience during page load. Since it's a lab tool, it also excludes FID and INP in favor of TBT.

When the Lighthouse metrics suggest a similar problem as the one you are trying to solve, the wealth of information in its audits can help identify issues and suggest solutions.

You can filter the audits to just the Core Web Vitals you are interested in to focus on fixes for issues related to a specific metric:

For FID and INP, use the TBT audits to identify issues that can potentially affect those metrics, but be aware that without interactions, Lighthouse is limited in how much it can diagnose.

Analyze in real time with the Web Vitals extension #

The Web Vitals Chrome extension shows Core Web Vitals in real time during page load and while browsing a page. Because of this, it can capture FID and INP as well as layout shifts that occur after load. The debugging options show more detailed information on each metric:

It's best to think of the Web Vitals extension as a spot-checking tool to find performance issues, not a comprehensive debugging tool—that's a job for the Performance panel in Chrome DevTools.

Drill down with the Performance panel #

The Performance panel in Chrome DevTools profiles all page behavior during a recorded period of time.

Key timings—such as LCP, for example—are shown in the Timings track. Click on these for more details.

The Layout Shifts track highlights layout shifts and clicking on these provides more details about the elements that shifted for debugging CLS.

Long Tasks (that can lead to FID and INP issues) are also highlighted with red triangles.

These features—as well as information in other parts of the Performance panel—can help you determine whether fixes are having any effect on a page's Core Web Vitals.

Debug Core Web Vitals in the field #

Lab tools can't always identify the cause of all Core Web Vitals issues affecting your users. This is one reason why it's so important to collect your own field data, as it takes factors into account that lab data cannot.

See debug performance in the field for more information.

Step 3: Monitor for changes #

Once you have fixed any issues, you want to ensure they have the desired effect and that new issues do not disrupt your Core Web Vitals. This requires monitoring for performance issues as part of developer workflow to prevent performance issues being released to production, and monitoring the field data regularly to ensure this is the case.

Monitor for performance requisitions in Continuous Integration (CI) environments #

Lighthouse-CI allows you to automatically run Lighthouse audits on code commits to prevent performance regressions entering code. This can check performance timings (which are subject to variability), or for just the performance audits, as a linting tool to prevent bad practices in the code.

View website health trends with field data #

While you should aim to catch and fix all performance issues before they make it to production, monitoring your field data via RUM is essential to find any that slip through. There are many commercial RUM products available that can help with this. The web-vitals JavaScript library can automate a website's field data collection, and optionally use this data to power custom dashboards and alerting systems.

For sites without a RUM solution, you can use the CrUX Dashboard as a basic trend analysis of field data. It reports the following for sites in CrUX:

Site overview, which segments Core Web Vitals into desktop and mobile device types.
Historical trend by metric type, which is a distribution of metrics over time for each available monthly release of CrUX report data.
User demographics, which illustrates the distribution of page views across an entire origin for users in each demographic including device and effective connection types.

The CrUX Dashboard is based on the CrUX BigQuery dataset, which is updated once a month. This can be a good reminder to regularly check in on your Core Web Vitals.

Conclusion #

Ensuring fast and delightful user experiences requires a performance-first mindset and adoption of a workflow to ensure progress. With the right tools and processes to audit, debug, and monitor, building great user experiences and staying within the thresholds defined for good Core Web Vitals is within your reach.

Keeping third-party scripts under control

2021-04-22T00:00:00Z

Third-party scripts, or "tags" can be a source of performance problems on your site, and therefore a target for optimization. However, before you start optimizing the tags you have added, make sure that you are not optimizing tags you don't even need. This article shows you how to assess requests for new tags, and manage and review existing ones.

When discussing third-party tags, the conversation often quickly moves to performance problems, losing sight of the foundations of what the "core" role of these tags are. They provide a wide range of useful functionality, making the web more dynamic, interactive, and interconnected. However, third-party tags can be added by different teams across the organization and are often forgotten about over time. People move on, contracts expire, or the results are yielded, but the teams never get back in touch to have the scripts removed.

Before you start to think about third-party tag script execution, or which tags can be deferred, lazy-loaded or preconnected from a technical lens, there's an opportunity to govern which tags are added to a site/page from an organizational point of view. A common theme with websites that are being slowed down due to vast amounts of third-party tags, is this part of the website is not owned by a single person or team, and therefore falls between the cracks. There's nothing more frustrating than optimizing your website, being happy with the performance in a staging environment, only for the speed to regress in production because of tags that are being added. Implementing a "vetting process" for third-party tags can help prevent this, by building a workflow that creates cross-functional accountability and responsibility for these tags.

The manner in which you vet third-party tags depends solely on the organisation, its structure and its current processes. It could be as basic as having a single team who govern and act as the gatekeeper for analysing tags before they are added. Or more advanced and formal, for example by providing a form to teams to submit requests for a tag. This might ask for context in terms of why it needs to be on the website, for how long it should be present, and what benefit it would bring to the business.

Tag governance process #

However you choose to vet tags within your organization, the following stages should be considered as part of the lifecycle of a tag.

Compliance #

Before any tag is added onto a page, check that it has been thoroughly vetted by a legal team to ensure it passes all compliance requirements for it to be present. This might include checking that the tag is compliant with the EU General Data Protection Regulation (GDPR), and California Consumer Privacy Act (CCPA).

This is critical, if there is any doubt with this step it needs to be addressed before assessing the tag from a performance point of view.

Required #

The second step is to question whether a specific tag is needed on the page. Consider the following discussion points:

Is the tag actively being used? If not, can it be removed?
If the tag is loading sitewide, is this necessary? For example, if we're analysing an A/B testing suite and you are currently only testing on Landing Pages, can we only drop the tag on this page type?
Can we add further logic to this, can we detect if there is a live A/B test? If so, allow the tag to be added, but if not ensure that it is not present.

Ownership #

Having a clear person or team as an owner of a tag, helps to proactively keep track of tags. Usually this would be whomever has added the tag. By having an assignee next to the tag, this will ensure reviews and audits in the future can be conducted to re-visit whether the tag is required.

Purpose #

The fourth step begins to create cross-functional accountability and responsibility by ensuring people understand why the tag is added to the page. It's important for there to be a cross-functional understanding of what each tag is bringing to the website, and why it is being used. For example, if the tag is recording user session actions to allow personalization, do all teams know why this should be present?

Furthermore, have there been any commercial vs performance trade-off discussions? If there is a tag that is deemed as "required" because it brings in revenue, has there been an analysis to the potential revenue lost through speed regression

Review #

The fifth, final and arguably most important step is to ensure tags are being reviewed on a regular basis. This should be dependent on the size of the website, the number of tags that are on the site, and their turnaround time (e.g. weekly, monthly, quarterly). This should be treated in the same manner as optimizing other website assets (JS, CSS, images, etc.) and proactively checked on a regular basis. Failure to review could lead to a "bloated" tag manager, which slows down the pages. It can be a complex task to revert back to being performant, while not regressing the required functionality on the page

The vetting process should leave you with a final list of tags which are classified as needed for a specific page. At this stage, you can then delve into technical optimisation approaches. This also opens up the opportunity to define the number of tags in this final list within a performance budget, which can be monitored within Lighthouse CI and incorporated into performance-specific goal setting. For example:

If we stick to <5 tags on our Landing Pages along with our own optimized JS, we're confident the Total Blocking Time (TBT) can hit 'good' in the Core Web Vitals.

Automating audits with AutoWebPerf

2020-12-09T00:00:00Z

What is AutoWebPerf (AWP)? #

AutoWebPerf (AWP) is a modular tool that enables automatic gathering of performance data from multiple sources. Currently there are many tools available to measure website performance for different scopes (lab and field), such as Chrome UX Report, PageSpeed Insights, or WebPageTest. AWP offers integration with various audit tools with a simple setup so you can continuously monitor the site performance in one place.

The release of Web Vitals guidance means that close and active monitoring of web pages is becoming increasingly important. The engineers behind this tool have been doing performance audits for years and they created AWP to automate a manual, recurring, and time consuming part of their daily activities. Today, AWP has reached a level of maturity and it's ready to be shared broadly so anyone can benefit from the automation it brings.

The tool is accessible on the AutoWebPerf public repository on GitHub.

What is AWP for? #

Although several tools and APIs are available to monitor the performance of web pages, most of them expose data measured at a specific time. To adequately monitor a website and maintain good performance of key pages, it's recommended to continuously take measurements of Core Web Vitals over time and observe trends.

AWP makes that easier by providing an engine and pre-built API integrations which can be programmatically configured to automate recurrent queries to various performance monitoring APIs.

For example, with AWP, you can set a daily test on your home page to capture the field data from CrUX API and lab data from a Lighthouse report from PageSpeed Insights. This data can be written and stored over time, for example, in Google Sheets and then visualised in the Data Studio dashboard. AWP automates the heavy-lifting part of the entire process, making it a great solution to follow lab and field trends over time. See Visualising audit results in Data Studio below for more details).

Architecture overview #

AWP is a modular-based library with three different types of modules:

the engine
connector modules
gatherer modules

The engine takes a list of tests from a connector (for example, from a local CSV file), runs performance audits through selected gatherers (such as PageSpeed Insights), and writes results to the output connector (for example, Google Sheets).

AWP comes with a number of pre-implemented gatherers and connectors:

Pre-implemented gatherers:
Pre-implemented connectors:
- Google Sheets
- JSON
- CSV

Automating audits with AWP #

AWP automates the performance audits via your preferred audit platforms such as PageSpeed Insights, WebPageTest, or CrUX API. AWP offers the flexibility to choose where to load the list of tests, and where to write the results to.

For example, you can run audits for a list of tests stored in a Google Sheet, and write the results to a CSV file, with the command below:

PSI_APIKEY=<YOUR_KEY> SHEETS_APIKEY=<YOUR_KEY> ./awp run sheets:<SheetID> csv:output.csv

Recurring audits #

You can run recurring audits in daily, weekly, or monthly frequency. For example, you can run daily audits for a list of tests defined in a local JSON like below:

{
  "tests": [
    {
      "label": "web.dev",
      "url": "https://web.dev",
      "gatherer": "psi"
    }
  ]
}

The command below reads the list of audit tests from the local JSON file, runs audits on a local machine, then outputs results to a local CSV file:

PSI_APIKEY=<YOUR_KEY> ./awp run json:tests.json csv:output.csv

To run audits every day as a background service continuously, you can use the command below instead:

PSI_APIKEY=<YOUR_KEY> ./awp continue json:tests.json csv:output.csv

Alternatively, you can set up the crontab in a Unix-like environment to run AWP as a daily cron job:

0 0 * * * PSI_APIKEY=<YOUR_KEY> ./awp run json:tests.json csv:output.csv

You can find more ways to automate daily audits and result collection in the AWP GitHub repository.

Visualising audit results in Data Studio #

Along with continuously measuring Core Web Vitals, it is important to be able to evaluate the trends and discover potential regressions with real user metrics (RUM) or the Chrome UX Report (CrUX) data collected by AWP. Note that Chrome UX Report (CrUX) is a 28-day moving aggregation, hence it is recommended to also use your own RUM data along with CrUX so you can spot regressions sooner.

Data Studio is a free visualization tool that you can easily load performance metrics into and draw trends as charts. For example, the time series charts below show Core Web Vitals based on Chrome UX Report data. One of the charts shows increasing Cumulative Layout Shift in recent weeks, which means regressions in the layout stability for certain pages. In this scenario, you would want to prioritize the efforts to analyze the underlying issues of these pages.

To simplify the end-to-end process from data collection to visualization, you can run AWP with a list of URLs to automatically export results to Google Sheets with the following command:

PSI_APIKEY=<YOUR_KEY> SHEETS_APIKEY=<YOUR_KEY> ./awp run sheets:<SheetID> csv:output.csv

After collecting daily metrics in a spreadsheet, you can create a Data Studio dashboard that loads the data directly from the spreadsheet, and plots the trends into a time series chart. Check out Google Spreadsheets API Connector for detailed steps about how to set up AWP with spreadsheets as a data source to visualize on Data Studio.

What's next? #

AWP provides a simple and integrated way to minimize the efforts to set up a continuous monitoring pipeline to measure Core Web Vitals and other performance metrics. As for now, AWP covers the most common use cases and will continue to provide more features to address other use cases in the future.

Learn more in the AutoWebPerf repository.