Annie Sullivan on web.dev

Advancing Interaction to Next Paint

2023-05-10T00:00:00Z

Chrome's mission with the Web Vitals program is to help developers focus on the metrics that are essential to great user experiences—the Core Web Vitals. One important aspect of the user experience that we've been focusing on is responsiveness, which is the ability of a web page to react quickly to user interactions.

First Input Delay (FID) is the Core Web Vital metric that measures responsiveness, but as we've written in the past, there are known limitations of FID. This led us to explore and seek feedback on a (then) experimental metric that addresses these limitations more effectively. In 2022, we announced Interaction to Next Paint (INP) as that new metric, made it widely available in our tools, and started working with the community to test its efficacy.

After another year of testing and gathering feedback from the community, we're ready to take the training wheels off and announce that INP is no longer experimental. Furthermore, effective March 2024, we're also committed to promoting INP as the new Core Web Vital metric for responsiveness, replacing FID.

In this post, we'll cover more about the motivation behind the decision, exactly what's changing, and how to get started. For more on how this affects Google Search, refer to their blog post.

A better responsiveness metric #

FID was a huge step forward when we introduced it as a Core Web Vital in 2020. It offered developers a new way to measure responsiveness as real users experience it, which has always been an important part of the Web Vitals program. Unlike similar metrics that only approximate page interactivity—like Total Blocking Time (TBT) and Time To Interactive (TTI)—FID directly measures the user experience. Crucially, a page could have slow TBT or TTI and still be perceived as responsive, because of the way real users interact with the page.

While it did improve the way we measure responsiveness, FID wasn't without limitations. The name itself actually gives away two such limitations: "first input" and "delay". FID only reports the responsiveness of the first time a user interacts with the page. Even though first impressions are important, the first interaction is not necessarily representative of all interactions throughout the life of a page. Further, FID only measures the input delay portion of the first interaction, which is the amount of time the browser had to wait (due to main thread busyness) before even beginning to handle the interaction.

Enter INP. Rather than only measuring the first interaction, INP takes all interactions into account, reporting one of the slowest over the entire lifetime of the page. And, rather than only measuring the delay portion, INP measures the full duration from the start of the interaction, through the event handler, and until the browser is able to paint the next frame. Hence, Interaction to Next Paint. These implementation details make INP a much more comprehensive measure of user-perceived responsiveness than FID.

Evolving INP #

Starting today, INP is no longer experimental and will instead be considered a pending Core Web Vital metric. This is a new designation to indicate that INP has proven its readiness to replace FID—but we're not making the change just yet. To give the ecosystem time to adjust, INP will officially become a stable Core Web Vital metric in March 2024.

Graphic showing the timeline of INP phases.

Across our tools, you'll start to see INP without the experimental prefix or icon. If you programmatically consume INP data from any of the following APIs, you'll need to update your code to switch over to the non-experimental INP field.

Map of APIs to their old and new INP field name.
* Lighthouse will stop using the experimental INP field in version 11.
Tool	Deprecated field	New field
CrUX API	`experimental_interaction_to_next_paint`	`interaction_to_next_paint`
CrUX History API	`experimental_interaction_to_next_paint`	`interaction_to_next_paint`
CrUX BigQuery	`experimental.interaction_to_next_paint`	`interaction_to_next_paint`
PageSpeed Insights API	`experimental_interaction_to_next_paint`	`interaction_to_next_paint`
Lighthouse report*	`experimental-interaction-to-next-paint`	`interaction-to-next-paint`

To ease the transition and avoid breaking any downstream applications, these APIs will continue to serve the experimental INP field for another 90 days. API support for the experimental INP field will end on August 8, 2023.

Meanwhile, FID will remain a Core Web Vital until March 2024. At that time, INP will join Largest Contentful Paint (LCP) and Cumulative Layout Shift (CLS) as the three stable Core Web Vitals. In a future update, we'll share more information about the precise timing of the change and the plan for FID after it's replaced as a Core Web Vital.

Getting started with INP #

Graphic showing the thresholds for good and poor INP.

Don't wait for INP to become a Core Web Vital in 2024 to start improving it. Your website may very well be in the 93% of sites that have good FID performance on mobile devices; however, you might be surprised to learn that only 65% of sites have good INP on mobile devices. INP paints a much more accurate picture of responsiveness, so these numbers help us to more clearly see the room for improvement ahead of us.

To find out whether your site has INP issues, how to address them, and all things INP, the best place to start is the INP optimization guide. Whether you're learning about responsiveness for the first time or you're a performance expert, we've added a lot of new guidance to make it as easy as possible for everyone to learn how to measure and optimize INP.

INP might take some time to learn, and that's just fine. To help you focus on the optimizations with the biggest ROI, earlier this year we published a blog post to distill our guidance down to the most effective recommendations to improve Core Web Vitals. Specifically, we wrote about three must-know techniques that apply equally to optimizing FID and INP: avoid or break up long tasks, avoid unnecessary JavaScript, and avoid large rendering updates. You can learn more about these and many other INP optimization techniques in our documentation.

The road ahead #

A new bar for responsiveness has been set, and we're excited to see the community rise to the challenge for a better user experience. The early feedback and case studies have been promising, but we know that for many, it may be a long and unfamiliar road ahead. We strive to make that journey as easy as possible with improved diagnostics, tools, and documentation to help developers along the way.

Stay tuned for updates on INP's exact graduation date in March 2024. And even though the metric is not experimental anymore, we're always open to feedback at web-vitals-feedback@googlegroups.com to help us improve INP and the Web Vitals program as a whole.

Hero image from Unsplash, by Agê Barros.

Towards a better responsiveness metric

2021-06-21T00:00:00Z

On the Chrome Speed Metrics team, we're working on deepening our understanding of how quickly web pages respond to user input. We'd like to share some ideas for improving responsiveness metrics and hear your feedback.

This post will cover two main topics:

Review our current responsiveness metric, First Input Delay (FID), and explain why we chose FID rather than some of the alternatives.
Present some improvements we've been considering that should better capture the end-to-end latency of individual events. These improvements also aim to capture a more holistic picture of the overall responsiveness of a page throughout its lifetime.

What is First Input Delay? #

The First Input Delay (FID) metric measures how long it takes the browser to begin processing the first user interaction on a page. In particular, it measures the difference between the time when the user interacts with the device and the time when the browser is actually able to begin processing event handlers. FID is just measured for taps and key presses, which means that it only considers the very first occurrence of the following events:

click
keydown
mousedown
pointerdown (only if it is followed by pointerup)

The following diagram illustrates FID:

FID does not include the time spent running those event handlers, nor any work done by the browser afterwards to update the screen. It measures the amount of time the main thread was busy before having the chance to handle an input. This blocking time is usually caused by long JavaScript tasks, as these can't just be stopped at any time, so the current task must complete before the browser can start processing the input.

Why did we choose FID? #

We believe it is important to measure actual user experience in order to ensure that improvements on the metric result in real benefits to the user. We chose to measure FID because it represents the part of the user experience when the user decides to interact with a site that has just been loaded. FID captures some of the time that the user has to wait in order to see a response from their interaction with a site. In other words, FID is a lower bound on the amount of time a user waits after interacting.

Other metrics like Total Blocking Time (TBT) and Time To Interactive (TTI) are based on long tasks and, like FID, also measure main thread blocking time during load. Since these metrics can be measured in both the field and the lab, many developers have asked why we don't prefer one of these over FID.

There are several reasons for this. Perhaps the most important reason is that these metrics do not measure the user experience directly. All of these metrics measure how much JavaScript runs on the page. While long running JavaScript does tend to cause problems to sites, these tasks don't necessarily impact the user experience if the user is not interacting with the page when they occur. A page can have a great score on TBT and TTI but feel slow or it can have a poor score while feeling fast for users. In our experience, these indirect measurements result in metrics that work great for some sites but not for most sites. In short, the fact that long tasks and TTI are not user-centric makes these weaker candidates.

While lab measurement is certainly important and an invaluable tool for diagnostics, what really matters is how users experience sites. By having a user-centric metric that reflects real-user conditions, you are guaranteed to capture something meaningful about the experience. We decided to start with a small portion of that experience, even though we know this portion is not representative of the full experience. This is why we're working on capturing a larger chunk of the time a user waits for their inputs to be handled.

A note on measuring TTI in the field

Measuring TTI on real users in the field is problematic because it occurs very late in the page load. A 5-second network quiet window is required before TTI can even be computed. In the lab, you can choose to unload the page whenever you have all the data that you need, but that's not the case with real-user monitoring in the field. A user may choose to leave the page or interact with it at any time. In particular, users may choose to leave pages that take a long time to load, and an accurate TTI will not be recorded in those cases. When we measured TTI for real users in Chrome, we found that only about half of page loads reached TTI.

What improvements are we considering? #

We would like to develop a new metric that extends what FID measures today yet still retains its strong connection to user experience.

We want the new metric to:

Consider the responsiveness of all user inputs (not just the first one)
Capture each event's full duration (not just the delay).
Group events together that occur as part of the same logical user interaction and define that interaction's latency as the max duration of all its events.
Create an aggregate score for all interactions that occur on a page, throughout its full lifecycle.

To be successful, we should be able to say with high confidence that if a site scores poorly on this new metric, it is not responding quickly to user interactions.

Capture the full event duration #

The first obvious improvement is to try to capture broader end-to-end latency of an event. As mentioned above, FID only captures the delay portion of the input event. It does not account for the time it takes the browser to actually process the event handlers.

There are various stages in the lifecycle of an event, as illustrated in this diagram:

The following are steps Chrome takes to process an input:

The input from the user occurs. The time at which this occurs is the event's timeStamp.
The browser performs hit testing to decide which HTML frame (main frame or some iframe) an event belongs to. Then the browser sends the event to the appropriate renderer process in charge of that HTML frame.
The renderer receives the event and queues it so that it can process when it becomes available to do so.
The renderer processes the event by running its handlers. These handlers may queue additional asynchronous work, such as setTimeout and fetches, that are part of the input handling. But at this point, the synchronous work is complete.
A frame is painted to the screen that reflects the result of event handlers running. Note that any asynchronous tasks queued by the event handlers may still be unfinished.

The time between steps (1) and (3) above is an event's delay, which is what FID measures.

The time between steps (1) and (5) above is an event's duration. This is what our new metric will measure.

The event's duration includes the delay, but it also includes the work occurring in event handlers and the work the browser needs to do to paint the next frame after those handlers have run. The duration of an event is currently available in the Event Timing API via the entry's duration attribute.

A note on asynchronous tasks

Ideally we would love to also capture asynchronous work triggered by the event. But the problem is that the definition of asynchronous work triggered by the event is extremely tricky to get right. As an example, a developer may choose to begin some animation on event handlers and use a setTimeout to begin such animation. If we captured all tasks posted on the handlers, the animation would delay the completion time for as long as the animation runs. We believe it is worthwhile to investigate options on how to use heuristics to capture work that is asynchronous and which should be completed ASAP. However, we want to be really careful when doing so because we don't want to penalize work that is meant to take a long time to be finished. Thus, our initial effort will look at step 5 as the end point: it will only consider synchronous work and the amount of time it takes to paint after such work is completed. That is, we're not going to apply heuristics to guess the work that would be kicked off asynchronously in step 4 in our initial effort.

It's worth noting that, in many cases, work should be executed synchronously. In fact, this may be unavoidable because events are sometimes dispatched one after the other and the event handlers need to be executed in order. That said, we will still miss important work, like events which trigger fetching or which rely on important work to be done at the next requestAnimationFrame callback, for example.

Group events into interactions #

Extending the metric measurement from delay to duration is a good first step, but it still leaves a critical gap in the metric: it focuses on individual events and not the user experience of interacting with the page.

Many different events can fire as a result of a single user interaction, and separately measuring each doesn't build a clear picture of what the user experiences. We want to make sure our metric captures the full amount of time a user has to wait for a response when tapping, pressing keys, scrolling, and dragging as accurately as possible. So we're introducing the concept of interactions to measure the latency of each.

Interaction types #

The following table lists the four interactions we want to define along with the DOM events that they're associated with. Note that this is not quite the same as the set of all events that are dispatched when such user interaction occurs. For instance, when a user scrolls, a scroll event is dispatched, but it happens after the screen has been updated to reflect the scrolling, so we don't consider it part of the interaction latency.

DOM events for each interaction type.
Interaction	Start / end	Desktop events	Mobile events
Keyboard	Key pressed	`keydown`	`keydown`
	Key pressed	`keypress`	`keypress`
	Key released	`keyup`	`keyup`
Tap or drag	Tap start or drag start	`pointerdown`	`pointerdown`
	Tap start or drag start	`mousedown`	`touchstart`
	Tap up or drag end	`pointerup`	`pointerup`
		`mouseup`	`touchend`
		`click`	`mousedown`
			`mouseup`
			`click`
Scroll	N/A

The first three interactions listed above (keyboard, tap, and drag) are currently covered by FID. For our new responsiveness metric, we want to include scrolling as well, since scrolling is extremely common on the web and is a critical aspect of how responsive a page feels to users.

A note on start and end

Note that each of these interactions has two parts: when the user presses the mouse, finger, or key down and when they lift it up. We need to ensure our metric doesn't count time the user spends holding the finger down between these two actions as part of the page's latency!

Keyboard #

A keyboard interaction has two parts to it: when the user presses the key and when they release it. There are three associated events with this user interaction: keydown, keyup, and keypress. The following diagram illustrates the keydown and keyup delays and durations for a keyboard interaction:

In the diagram above, the durations are disjoint because the frame from keydown updates is presented before the keyup occurs, but this does not need to be the case always. In addition, note that a frame can be presented in the middle of a task in the renderer process since the last steps required to produce the frame are done outside of the renderer process.

The keydown and keypress occur when the user presses the key, while the keyup occurs when the user releases the key. Generally the main content update occurs when the key is pressed: text appears on the screen, or the modifier effect is applied. That said, we want to capture the more rare cases where keyup would also present interesting UI updates, so we want to look at the overall time taken.

In order to capture the overall time taken by the keyboard interaction, we can compute the maximum of the duration of the keydown and the keyup events.

A note on repeating keypresses

There is an edge case here worth mentioning: there may be cases where the user presses a key and takes a while to release it. In this case, the sequence of events dispatched can vary. In these cases, we would consider there to be one interaction per keydown, which may or may not have a corresponding keyup.

Tap #

Another important user interaction is when the user taps or clicks on a website. Similar to keypress, some events are fired as the user presses down, and others as they release, as shown in the diagram above, Note the events associated with a tap are a little different on desktop vs mobile.

For a tap or click, the release is generally the one which triggers the majority of reactions, but, as with keyboard interactions, we want to capture the full interaction. And in this case it's more important to do so because having some UI updates upon tap press is not actually that uncommon.

We'd like to include the event durations for all of these events, but as many of them overlap completely, we need to measure just pointerdown, pointerup, and click to cover the full interaction.

Can we narrow further to just pointerdown and pointerup?

One initial thought would be to use the pointerdown and pointerup events and assume that they cover all of the durations that we're interested in. Sadly, this is not the case, as this edge case shows. Try opening this site on mobile, or with mobile emulation, and tapping where it says "Click me". This site triggers the browser tap delay. It can be seen that the pointerdown, pointerup, and touchend are dispatched quickly, whereas the mousedown, mouseup, and click wait for the delay before being dispatched. This means that if we only looked at pointerdown and pointerup then we'd miss the duration from the synthetic events, which is large due to the browser tap delay and should be included. So we should measure pointerdown, pointerup, and click to cover the full interaction.

Drag #

We decided to include dragging as well since it has similar associated events and since it generally causes important UI updates to sites. But for our metric we intend to only consider the drag start and the drag end - the initial and final parts of the drag. This is to make it easier to reason about as well as make the latencies comparable with the other interactions considered. This is consistent with our decision to exclude continuous events such as mouseover.

We're also not considering drags implemented via the Drag and Drop API because they only work on desktop.

Scrolling #

One of the most common forms of interacting with a site is via scrolling. For our new metric, we'd like to measure the latency for the initial scrolling interaction of the user. In particular, we care about the initial reaction of the browser to the fact that the user requested a scroll. We will not cover the whole scrolling experience. That is, scrolling produces many frames, and we'll focus our attention on the initial frame produced as a reaction to the scroll.

Why just the first one? For one, subsequent frames may be captured by a separate smoothness proposal. That is, once the user has been shown the first result of scrolling, the rest should be measured in terms of how smooth the scrolling experience feels. Therefore, we think that the smoothness effort could better capture this. So, as with FID, we choose to stick to discrete user experiences: user experiences that have clear points in time associated with them and for which we can easily compute their latency. Scrolling as a whole is a continuous experience, so we do not intend to measure all of it in this metric.

So why measure scrolls? The scrolling performance we've gathered in Chrome shows that scrolling is generally very fast. That said, we still want to include initial scroll latencies in our new metric for various reasons. First, scrolling is fast only because it has been optimized so much, because it is so important. But there are still ways for a website to bypass some of the performance gains that the browser offers. The most common one in Chrome is to force scrolling to happen on the main thread. So our metric should be able to say when this happens and causes poor scrolling performance for users. Second, scrolling is just too important to ignore. We worry that if we exclude scrolling then we'll have a big blindspot, and scrolling performance could decrease over time without web developers properly noticing.

There are several events that are dispatched when a user scrolls, such as touchstart, touchmove, and scroll. Except for the scroll event, this is largely dependent on the device used for scrolling: touch events are dispatched when scrolling with the finger on mobile devices, while wheel events occur when scrolling with a mouse wheel. The scroll events are fired after initial scrolling has completed. And in general, no DOM event blocks scrolling, unless the website uses non-passive event listeners. So we think of scrolling as decoupled from DOM Events altogether. What we want to measure is the time from when the user moves enough to produce a scroll gesture until the first frame that shows that scrolling happened.

How to define the latency of an interaction? #

As we noted above, interactions that have a "down" and "up" component need to be considered separately in order to avoid attributing the time the user spent holding their finger down.

For these types of interactions, we'd like the latency to involve the durations of all events associated with them. Since event durations for each "down" and "up" part of the interaction can overlap, the simplest definition of interaction latency that achieves this is the maximum duration of any event associated with it. Referring back to the keyboard diagram from earlier, this would be the keydown duration, as it is longer than the keyup:

The keydown and keyup durations may overlap as well. This may happen for instance when the frame presented for both events is the same, as in the following diagram:

There's are pros and cons to this approach of using the maximum, and we're interested in hearing your feedback:

Pro: It is aligned with how we intend to measure scroll in that it only measures a single duration value.
Pro: It aims to reduce noise for cases like keyboard interactions, where the keyup usually does nothing and where the user may execute the key press and release quickly or slowly.
Con: It does not capture the full wait time of the user. For instance, it will capture the start or end of a drag, but not both.

For scrolling (which just has a single associated event) we'd like to define its latency as the time it takes for the browser to produce the first frame as a result of scrolling. That is, the latency is the delta between the event timeStamp of the first DOM event (like touchmove, if using a finger) that is large enough to trigger a scroll and the first paint which reflects the scrolling taking place.

Aggregate all interactions per page #

Once we've defined what the latency of an interaction is, we'll need to compute an aggregate value for a page load, which may have many user interactions. Having an aggregated value enables us to:

Form correlations with business metrics.
Evaluate correlations with other performance metrics. Ideally, our new metric will be sufficiently independent that it adds value to the existing metrics.
Easily expose values in tooling in ways that are easy to digest.

In order to perform this aggregation we need to solve two questions:

What numbers do we try to aggregate?
How do we aggregate those numbers?

We're exploring and evaluating several options. We welcome your thoughts on this aggregation.

One option is to define a budget for the latency of an interaction, which may depend on the type (scroll, keyboard, tap, or drag). So for example if the budget for taps is 100 ms and the latency of a tap is 150 ms then the amount over budget for that interaction would be 50 ms. Then we could compute the maximum amount of latency that goes over the budget for any user interaction in the page.

Another option is to compute the average or median latency of the interactions throughout the life of the page. So if we had latencies of 80 ms, 90 ms, and 100 ms, then the average latency for the page would be 90 ms. We could also consider the average or median "over budget" to account for different expectations depending on the type of interaction.

How does this look like on web performance APIs? #

What's missing from Event Timing? #

Unfortunately not all of the ideas presented in this post can be captured using the Event Timing API. In particular, there's no simple way to know the events associated with a given user interaction with the API. In order to do this, we've proposed adding an interactionID to the API.

Another shortcoming of the Event Timing API is that there is no way to measure the scroll interaction, so we're working on enabling these measurements (via Event Timing or a separate API).

What can you try right now? #

Right now, it is still possible to compute the maximum latency for taps/drags and for keyboard interactions. The following code snippet would produce these two metrics.

let maxTapOrDragDuration = 0;
let maxKeyboardDuration = 0;
const observer = new PerformanceObserver(list => {
  list.getEntries().forEach(entry => {
    switch(entry.name) {
      case "keydown":
      case "keyup":
        maxKeyboardDuration = Math.max(maxKeyboardDuration,
            entry.duration);
        break;
      case "pointerdown":
      case "pointerup":
      case "click":
        maxTapOrDragDuration = Math.max(maxTapOrDragDuration,
            entry.duration);
        break;
    }
  });
});
observer.observe({type: "event", durationThreshold: 16, buffered: true});
// We can report maxTapDragDuration and maxKeyboardDuration when sending
// metrics to analytics.

Feedback #

Let us know what you think about these ideas by emailing: web-vitals-feedback@googlegroups.com!

Evolving the CLS metric

2021-04-07T00:00:00Z

We (the Chrome Speed Metrics Team) recently outlined our initial research into options for making the CLS metric more fair to pages that are open for a long time. We've received a lot of very helpful feedback and after completing the large-scale analysis, we've finalized the change we plan to make to the metric: maximum session window with 1 second gap, capped at 5 seconds.

Read on for the details!

How did we evaluate the options? #

We reviewed all the feedback received from the developer community and took it into account.

We also implemented the top options in Chrome and did a large-scale analysis of the metrics over millions of web pages. We checked what types of sites each option improved, and how the options compared, especially looking into the sites which were scored differently by different options. Overall, we found that:

All the options reduced the correlation between time spent on page and layout shift score.
None of the options resulted in a worse score for any page. So there is no need to be concerned that this change will worsen the scores for your site.

Decision points #

Why a session window? #

In our earlier post, we covered a few different windowing strategies for grouping together layout shifts while ensuring the score doesn't grow unbounded. The feedback we received from developers overwhelmingly favored the session window strategy because it groups the layout shifts together most intuitively.

To review session windows, here's an example:

Example of a session window.

In the example above, many layout shifts occur over time as the user views the page. Each is represented by a blue bar. You'll notice above that the blue bars have different heights; those represent the score of each individual layout shift. A session window starts with the first layout shift and continues to expand until there is a gap with no layout shifts. When the next layout shift occurs, a new session window starts. Since there are three gaps with no layout shifts, there are three session windows in the example. Similar to the current definition of CLS, the scores of each shift are added up, so that each window's score is the sum of its individual layout shifts.

Based on the initial research, we chose a 1 second gap between session windows, and that gap worked well in our large-scale analysis. So the "Session Gap" shown in the example above is 1 second.

Why the maximum session window? #

We narrowed the summarization strategies down to two options in our initial research:

The average score of all the session windows, for very large session windows (uncapped windows with 5 second gaps between them).
The maximum score of all the session windows, for smaller session windows (capped at 5 seconds, with 1 second gaps between them).

After the initial research, we added each metric to Chrome so that we could do a large-scale analysis over millions of URLs. In the large-scale analysis, we found a lot of URLs with layout shift patterns like this:

On the bottom right, you can see there is only a single, tiny layout shift in Session Window 2, giving it a very low score. That means that the average score is pretty low. But what if the developer fixes that tiny layout shift? Then the score is calculated just on Session Window 1, which means that the page's score nearly doubles. It would be really confusing and discouraging to developers to improve their layout shifts only to find that the score got worse. And removing this small layout shift is obviously slightly better for the user experience, so it shouldn't worsen the score.

Because of this problem with averages, we decided to move forward with the smaller, capped, maximum windows. So in the example above, Session Window 2 would be ignored and only the sum of the layout shifts in Session Window 1 would be reported.

Why 5 seconds? #

We evaluated multiple window sizes and found two things:

For short windows, slower page loads and slower responses to user interactions could break layout shifts into multiple windows and improve the score. We wanted to keep the window large enough so it doesn't reward slowdowns!
There are some pages with a continual stream of small layout shifts. For example, a sports score page that shifts a bit with each score update. These shifts are annoying, but they don't get more annoying as time passes. So we wanted to ensure that the window was capped for these types of layout shifts.

With these two things in mind, comparing a variety of window sizes on many real-world web pages, we concluded that 5 seconds would be a good limit to the window size.

How will this affect my page's CLS score? #

Since this update caps the CLS of a page, no page will have a worse score as a result of this change.

And based on our analysis, 55% of origins will not see a change in CLS at all at the 75th percentile. This is because their pages either do not currently have any layout shifts or the shifts they do have are already confined to a single session window.

The rest of the origins will see improved scores at the 75th percentile with this change. Most will only see a slight improvement, but about 3% will see their scores improve from having a "needs improvement" or "poor" rating to having a "good" rating. These pages tend to use infinite scrollers or have many slow UI updates, as described in our earlier post.

How can I try it out? #

We'll be updating our tools to use the new metric definition soon! Until then, you can try out the updated version of CLS on any site using the example JavaScript implementations or the fork of the Web Vitals extension.

Thanks to everyone who took the time to read the previous post and give their feedback!

Feedback wanted: The road to a better layout shift metric for long-lived pages

2021-01-25T00:00:00Z

Cumulative Layout Shift (CLS) is a metric that measures the visual stability of a web page. The metric is called cumulative layout shift because the score of every individual shift is summed throughout the lifespan of the page.

While all layout shifts are poor user experiences, they do add up more on pages that are open longer. That's why the Chrome Speed Metrics Team set out to improve the CLS metric to be more neutral to the time spent on a page.

It's important that the metric focuses on user experience through the full page lifetime, as we've found that users often have negative experiences after load, while scrolling or navigating through pages. But we've heard concerns about how this impacts long-lived pages—pages which the user generally has open for a long time. There are several different types of pages which tend to stay open longer; some of the most common are social media apps with infinite scroll and single-page applications.

An internal analysis of long-lived pages with high CLS scores found that most problems were caused by the following patterns:

Infinite scrollers shifting content as the user scrolls.
Input handlers taking longer than 500 ms to update the UI in response to a user interaction, without any kind of placeholder or skeleton pattern.

While we encourage developers to improve those user experiences, we're also working towards improving the metric and looking for feedback on possible approaches.

How would we decide if a new metric is better? #

Before diving into metric design, we wanted to ensure that we evaluated our ideas on a wide variety of real-world web pages and use cases. To start, we designed a small user study.

First, we recorded videos and Chrome traces of 34 user journeys through various websites. In selecting the user journeys, we aimed for a few things:

A variety of different types of sites, such as news and shopping sites.
A variety of user journeys, such as initial page load, scrolling, single-page app navigations, and user interactions.
A variety of both number and intensity of individual layout shifts on the sites.
Few negative experiences on the sites apart from layout shifts.

We asked 41 of our colleagues to watch two videos at a time, rating which of the pair was better in terms of layout shift. From these ratings, we created an idealized ranking order of the sites. The results of the user ranking confirmed our suspicions that our colleagues, like most users, are really frustrated by layout shifts after load, especially during scrolling and single-page app navigations. We saw that some sites have much better user experiences during these activities than others.

Since we recorded Chrome traces along with the videos, we had all the details of the individual layout shifts in each user journey. We used those to compute metric values for each idea for each user journey. This allowed us to see how each metric variant ranked the user journeys, and how different each was from the ideal ranking.

What metric ideas did we test? #

Windowing strategies #

Often pages have multiple layout shifts bunched closely together, because elements can shift multiple times as new content comes in piece by piece. This prompted us to try out techniques for grouping shifts together. To accomplish that, we looked at three windowing approaches:

Tumbling windows
Sliding windows
Session windows

In each of these examples, the page has layout shifts of varying severity over time. Each blue bar represents a single layout shift, and the length represents the score of that shift. The images illustrate the ways different windowing strategies group the layout shifts over time.

Tumbling windows #

Example of a tumbling window.

The simplest approach is just to break the page into windows of equal-sized chunks. These are called tumbling windows. You'll notice above that the fourth bar really looks like it should be grouped into the second tumbling window, but because the windows are all a fixed size it is in the first window instead. If there are slight differences in timing of loads or user interactions on the page, the same layout shifts might fall on different sides of the tumbling window boundaries.

Sliding windows #

Example of a sliding window.

An approach that lets us see more possible groupings of the same length is to continuously update the potential window over time. The image above shows one sliding window at a time, but we could look at all possible sliding windows or a subset of them to create a metric.

Session windows #

Example of a session window.

If we wanted to focus on identifying areas of the page with bursts of layout shifts, we could start each window at a shift, and keep growing it until we encountered a gap of a given size between layout shifts. This approach groups the layout shifts together, and ignores most of the non-shifting user experience. One potential problem is that if there are no gaps in the layout shifts, a metric based on session windows could grow unbounded just like the current CLS metric. So we tried this out with a maximum window size as well.

Window sizes #

The metric might give very different results depending on how big the windows actually are, so we tried multiple different window sizes:

Each shift as its own window (no windows)
100 ms
300 ms
1 second
5 seconds

Summarization #

We tried out many ways to summarize the different windows.

Percentiles #

We looked at the maximum window value, as well as the 95th percentile, 75th percentile, and median.

Average #

We looked at the mean window value.

Budgets #

We wondered if maybe there was some minimum layout shift score that users wouldn't notice, and we could just count layout shifts over that "budget" in the score. So for various potential "budget" values, we looked at the percentage of shifts over budget, and the total shift score over budget.

Other strategies #

We also looked at many strategies that didn't involve windows, like the total layout shift divided by time on page, and the average of the worst N individual shifts.

The initial results #

Overall, we tested 145 different metric definitions based on permutations of the above ideas. For each metric, we ranked all the user journeys by their score on the metric, and then ranked the metrics by how close they were to the ideal ranking.

To get a baseline, we also ranked all the sites by their current CLS score. CLS placed 32nd, tied with 13 other strategies, so it was better than most permutations of the strategies above. To ensure the results were meaningful, we also added in three random orderings. As expected, the random orderings did worse than every strategy tested.

To understand if we might be overfitting for the data set, after our analysis we recorded some new layout shift videos and traces, manually ranked those, and saw that the metric rankings were very similar for the new data set and the original one.

A few different strategies stood out in the rankings.

Best strategies #

When we ranked the strategies, we found that three types of strategies topped the list. Each had roughly the same performance, so we plan to move forward with a deeper analysis on all three. We'd also like to hear developer feedback to understand if there are factors outside of user experience we should be considering when deciding between them. (See below for how to give feedback.)

High percentiles of long windows #

A few windowing strategies worked well with long window sizes:

1 second sliding windows
Session windows capped at 5 seconds with 1 second gap
Session windows uncapped with 1 second gap

These all ranked really well at both the 95th percentile and the maximum.

But for such large window sizes, we were concerned about using the 95th percentile—often we were looking at only 4-6 windows, and taking the 95th percentile of that is a lot of interpolation. It's unclear what the interpolation is doing in terms of the metric value. The maximum value is a lot clearer, so we decided to move forward with checking the maximum.

Average of session windows with long gaps #

Averaging the scores of all uncapped session windows with 5 second gaps between them performed really well. This strategy has a few interesting characteristics:

If the page doesn't have gaps between layout shifts, it ends up being one long session window with the exact same score as the current CLS.
This metric didn't take idle time into account directly; it only looked at the shifts that happened on the page, and not at points in time when the page was not shifting.

High percentiles of short windows #

The maximum 300 ms sliding window ranked very highly, as well as the 95th percentile. For the shorter window size, there is less percentile interpolation than larger window sizes, but we were also concerned about "repeat" sliding windows—if a set of layout shifts occurs over two frames, there are multiple 300 ms windows that include them. Taking the maximum is much clearer and simpler than taking the 95th percentile one. So again we decided to move forward with checking the maximum.

Strategies that didn't work out #

Strategies that tried to look at the "average" experience of time spent both without layout shifts and with layout shifts did very poorly. None of the median or 75th percentile summaries of any windowing strategy ranked the sites well. Neither did the sum of layout shifts over time.

We evaluated a number of different "budgets" for acceptable layout shifts:

Percent of layout shifts above some budget. For various budgets, these all ranked quite poorly.
Average layout shift above some excess. Most variations on this strategy did poorly, but average excess over a long session with a large gap did almost as well as the average of session windows with long gaps. We decided to move forward with only the latter because it is simpler.

Next steps #

Larger-scale analysis #

We've implemented the top strategies listed above in Chrome, so that we can get data on real-world usage for a much larger set of websites. We plan to use a similar approach of ranking sites based on their metric scores to do the larger-scale analysis:

Rank all the sites by CLS, and by each new metric candidate.
- Which sites are ranked most differently by CLS and each candidate? Do we find anything unexpected when we look at these sites?
- What are the largest differences between the new metric candidates? Do any of the differences stand out as advantages or disadvantages of a specific candidate?
Repeat the above analysis, but bucketing by time spent on each page load. Do we see an expected improvement for long-lived page loads with acceptable layout shift? Do we see any unexpected results for short-lived pages?

Feedback on our approach #

We'd love to get feedback from web developers on these approaches. Some things to keep in mind while considering the new approaches:

What's not changing #

We do want to clarify that a lot of things will not be changing with a new approach:

None of our metric ideas change the way layout shift scores for individual frames are calculated, only the way we summarize multiple frames. This means that the JavaScript API for layout shifts will stay the same, and the underlying events in Chrome traces that developer tools use will also stay the same, so layout shift rects in tools like WebPageTest and Chrome DevTools will continue to work the same way.
We'll continue to work hard on making the metrics easy for developers to adopt, including them in the web-vitals library, documenting on web.dev, and reporting them in our developer tooling like Lighthouse.

Trade-offs between metrics #

One of the top strategies summarizes the layout shift windows as an average, and the rest report the maximum window. For pages which are open a very long time, the average will likely report a more representative value, but in general it will likely be easier for developers to act on a single window—they can log when it occurred, the elements that shifted, and so on. We'd love feedback on which is more important to developers.

Do you find sliding or session windows easier to understand? Are the differences important to you?

How to give feedback #

You can try out the new layout shift metrics on any site using our example JavaScript implementations or our fork of the Core Web Vitals extension.

Please email feedback to our web-vitals-feedback Google group, with "[Layout Shift Metrics]" in the subject line. We're really looking forward to hearing what you think!