Bojan Pavic on web.dev

How to measure speed?

2019-05-01T00:00:00Z

Real-world performance is highly variable due to differences in users' devices, network connections, and other factors. For example, if you load your website using a cable network connection in your office and compare it against the load using WiFi in a coffee shop, the experiences are likely to be very different. There are many tools on the market that can help you collect lab or field data to assess page performance.

Lab data vs Field data #

Lab data is performance data collected within a controlled environment with predefined device and network settings, while Field data is performance data collected from real page loads experienced by your users in the wild. Each type has its own strengths and limitations.

Lab data offers reproducible results and a debugging environment, but might not capture real-world bottlenecks and cannot correlate against real-world page KPIs. With lab data, you need to understand your users' typical devices and networks and appropriately mirror those conditions when you test performance. Have in mind that even in areas with 4G, users may still experience slower or intermittent connections when in elevators, while commuting, or in comparable environments.

Field data (also called Real User Monitoring or RUM) captures true real-world user experience and enables correlation to business KPIs, but has a restricted set of metrics and limited debugging capabilities.

Tools #

Lab data #

Lighthouse takes a URL and runs a series of audits against the page, generating a report on how well the page did. There are multiple ways to run Lighthouse, including an option to easily audit a page from within Chrome DevTools.

Field data #

Chrome User Experience Report (CrUX) provides metrics showing how real-world Chrome users experience popular destinations on the web.

Other tools #

PageSpeed Insights provides both lab and field data about a page. It uses Lighthouse to collect and analyze lab data about the page, while real-world field data is based on the Chrome User Experience Report dataset.

Chrome Developer Tools is a set of web developer tools built directly into the Google Chrome browser. It allows you to profile the runtime of a page, as well as identify and debug performance bottlenecks.

Why does speed matter?

2019-05-01T00:00:00Z

Consumers increasingly rely on mobile to access digital content and services, and if you look at your site analytics, you'll probably see this story playing out in your own data. Consumers are also more demanding than they've ever been, and when they weigh the experience on your site, they aren't just comparing you with your competitors, they're rating you against the best-in-class services they use every day.

This post rounds up some of the research that has been done on the relationship between performance and business success.

Performance is about retaining users #

Performance has directly impacted the company's bottom line.
Pinterest

Performance plays a major role in the success of any online venture. High-performing sites engage and retain users better than low-performing ones.

Pinterest reduced perceived wait times by 40% and this increased search engine traffic and sign-ups by 15%.

COOK reduced average page load time by 850 milliseconds which increased conversions by 7%, decreased bounce rates by 7%, and increased pages per session by 10%.

Studies have also shown the negative impact poor performance can have on business goals. For example, the BBC found they lost an additional 10% of users for every additional second their site took to load.

Performance is about improving conversions #

Retaining users is crucial to improving conversions. Slow sites have a negative impact on revenue, and fast sites are shown to increase conversion rates.

For Mobify, every 100ms decrease in homepage load speed worked out to a 1.11% increase in session-based conversion, yielding an average annual revenue increase of nearly $380,000. Additionally, a 100ms decrease in checkout page load speed amounted to a 1.55% increase in session-based conversion, which in turn yielded an average annual revenue increase of nearly $530,000.

When AutoAnything reduced page load time by half, they saw a boost of 12% to 13% in sales.

Retailer Furniture Village audited their site speed and developed a plan to address the problems they found, leading to a 20% reduction in page load time and a 10% increase in conversion rate.

Performance is about user experience #

When it comes to user experience, speed matters. A consumer study shows that the stress response to delays in mobile speed are similar to that of watching a horror movie or solving a mathematical problem, and greater than waiting in a checkout line at a retail store.

As a site begins to load, there's a period of time where users wait for content to appear. Until this happens, there's no user experience to speak of. This lack of an experience is fleeting on fast connections. On slower connections, however, users are forced to wait. Users may experience more problems as page resources slowly trickle in.

A comparison of page load on a very slow connection (top) versus a faster connection (bottom).

Performance is a foundational aspect of good user experiences. When sites ship a lot of code, browsers must use megabytes of the user's data plan to download it. Mobile devices have limited CPU power and memory. They often get overwhelmed with what we might consider a small amount of unoptimized code. This creates poor performance which leads to unresponsiveness. Knowing what we know about human behavior, users will only tolerate low performing applications for so long before abandoning them.

Performance is about people #

Poorly performing sites and applications can also pose real costs for the people who use them.

As mobile users continue to make up a larger portion of internet users worldwide, it's important to bear in mind that many of these users access the web through mobile LTE, 4G, 3G, and even 2G networks. As Ben Schwarz of Calibre points out in this study of real world performance, the cost of prepaid data plans is decreasing, which in turn is making access to the internet more affordable in places where it once wasn't. Mobile devices and internet access are no longer luxuries. They are common tools necessary to navigate and function in an increasingly interconnected world.

Total page size has been steadily increasing since at least 2011, and the trend appears to be continuing. As the typical page sends more data, users must replenish their metered data plans more often, which costs them money.

In addition to saving your users money, fast and lightweight user experiences can also prove crucial for users in crisis. Public resources such as hospitals, clinics, and crisis centers have online resources that give users important and specific information that they need during a crisis. While design is pivotal in presenting important information efficiently in stressful moments, the importance of delivering this information fast can't be understated. It's part of our job.

Get started with improving your website speed #

Read up on the Core Web Vitals to learn about the metrics that Google believes all websites should focus on.

Today, we’re building on this work and providing an early look at an upcoming Search ranking change that incorporates these page experience metrics. We will introduce a new signal that combines Core Web Vitals with our existing signals for page experience to provide a holistic picture of the quality of a user’s experience on a web page.
Evaluating page experience for a better web , Official Google Webmaster Central Blog

Then check out Fast load times for lots of tips and tricks related to getting fast and staying fast.

How to stay fast?

2019-05-01T00:00:00Z

Brands that optimize speed will often find they regress quickly. This is because website performance is a lot like getting fit: it's not enough to make a one time effort; you have to change your lifestyle.

Performance budgets are one way to address this. A performance budget is a set of limits on metrics that affect site performance. The concept is similar to a financial budget: you set a limit and make sure you stay within it. In general, a good performance budget combines different types of metrics; so, for example, the performance budget for a product page might look as follows:

Example performance budget.
Time To Interactive on slow 3G	Less than	5 seconds
First Contentful Paint on slow 3G	Less than	2 seconds
Lighthouse performance score	Greater than	80
Total JavaScript size	Less than	170 kb

Once set, a performance budget has to be enforced, which means for example incorporating the budget into your build process and reporting. Tools like Lighthouse can be included in your continuous integration, and you can write tests that fail a build if key metrics drop below a set threshold. Additionally, regular reporting through dashboards or summary reports can help with visibility and accountability. Pinterest are one example of a business that have implemented performance budgets to make sure their fast experience stays fast, while brands like Experian are now using site speed as a key metric in their monthly executive KPI reports.

You can find more details on performance budgets here. A guide that describes how to further instill performance culture and make speed metrics visible and tangible for all stakeholders can be found here.

What is speed?

2019-05-01T00:00:00Z

So, speed matters, but what exactly do we mean by it? What does it mean to have a fast site?

It's common to hear people talk in terms of their website loading in x.xx seconds or similar, but a load is not a single moment in time; it's an experience that no single metric can fully capture. There are multiple moments during the load experience that can affect whether a user perceives it as 'fast', and if you just focus solely on one, you might miss bad experiences that happen during the rest of the time.

Rather than measuring load with just one metric, you should time each moment during the experience that affects the user's perception of load speed. When a user navigates to a web page, they're typically looking for certain types of feedback:

Traditional performance metrics like load time or DOMContentLoaded time are unreliable, since their occurrence may or may not correspond with these feedback milestones. So, additional metrics have emerged that could be used to understand when a page delivers this feedback to its users:

It's important to understand the different insights offered by these metrics, then establish the ones that are important to your user experience. Some brands even define additional custom metrics specific to the expectations people have of their service. In the case of Pinterest, users want to see images, so they defined a custom metric, Pinner Wait Time, that combines Time to Interactive and Above the Fold Image load times.

Even though the load is more than one moment in time, it can still be useful to have a single metric for the purposes of simplified reporting or comparison: Speed Index and Lighthouse score can both be used in this way.