André Cipriani Bandarra on web.dev

Deep dive into top web developer pain points

2022-04-25T00:00:00Z

A few months ago, Paul Kinlan posted about the top developer pain points in 2021, so it feels appropriate to start this article with an update on the last 2 quarters. The numbers have moved around a bit, but the ranking hasn't changed.

Challenge	Q1 2021	Q2 2021	Q3 2021	Q4 2021
Keeping up with changes to the web platform or web standards.	27%	26%	27%	22%
Keeping up with a large number of new and existing tools or frameworks.	26%	26%	25%	21%
Making a design or experience work the same across browsers.	26%	28%	24%	21%
Testing across browsers.	23%	24%	20%	20%
Understanding and implementing security measures.	23%	25%	20%	19%

As mentioned in Paul's blog post we need to address these pain points. As part of a larger effort to do so, my colleague Kadir Topal and I have interviewed over 18 developers. Our aim is to investigate and start making sense of the path to fixing top developer pain points.

Developer Discussions #

Disclaimer: those insights are based on a small number of conversations with developers. When using "all" or "some", this refers to the developers interviewed, not the entire community. More research is needed to extrapolate those insights more widely.

These conversations were a great reminder of how amazing and diverse the web developer community is, and I'd like to thank all the developers who talked to us. Some developers had over 25 years of experience, while others started as recently as 2020. Some developers started their careers via a formal computer science degree, while others started their careers independently. Some developers actively seek what is new and keep up by reading browser release notes, while others learn about new things via colleagues and friends. Some think complexity is part of the job and enjoy being challenged, while others just want to get their job done. When thinking about solving those pain points, it's important to keep this diversity in our minds!

One of the common things amongst all developers is that all of them are using a CMS or a framework to do their work. Wordpress, React, Bootstrap, Angular, and Tailwind were all mentioned, none of the developers were using the vanilla web platform in production. Choosing a framework when starting a project is a challenge, and developers frequently take into account non-technical requirements. For example, whether it will be easy to hire a developer to work with that framework. We cannot improve developer pain points if frameworks and CMSs are not included in the solution.

Speaking of the web platform, most developers understand the platform as the thing they are developing on top of. This includes not only the classical definition of the web platform, but also the CMSs, framework, tools, and polyfills. In many cases, keeping up to date with those is where the biggest difficulties are. This changed our interpretation of that question, and we now know we need to update our survey to break it down into different parts that are less ambiguous.

Another area of ambiguity is the definition of web standards. When asked about examples around keeping up with standards, many developers pointed out difficulties with keeping up with best practices instead. This is another area we need to clarify on the survey.

Developers look for best practices when implementing specific use-cases and patterns. Blog posts and StackOverflow are mentioned as sources for best practices, but developers often wonder if the information they are reading is indeed the best practice and if it is up to date with the latest features and APIs. They would like a more official source to read those.

Keeping up with features and APIs that enable new use-cases is a smaller problem. Developers struggle more with features, APIs, and changes to the platform that result in a change in best practices.

Most developers agree that compatibility is one of the biggest challenges. Things are improving via efforts like Compat 2021 and Interop 2022, but it's clear that developers don't see it as a solved problem yet.

Most developers use polyfills in one way or another. In many cases, however, usage is transparent to developers, since the polyfill can be automatically added by a tool like Babel or a framework. For those who are managing their polyfills themselves, figuring out if a polyfill is "good" can be a problem. Developers mentioned using the number of installs on NPM and the creator of the polyfill as signals. A couple of developers mentioned doing work to remove polyfills that became unnecessary due to dropping support for IE 11.

Frameworks introduce fragmentation issues. We heard reports where developers were "stuck" into an older version of a framework, and limited on the features they could use because of that, but that migrating to a newer version of the same framework could be costly and hard to justify.

Conclusion #

Modern web development has many moving pieces including, standards, browsers, libraries, polyfills, CMSs, frameworks, best practices, and tooling. This diversity is one of the great things about the web but, right now, it's up to each developer individually to make sense of each piece and how they are compatible with each other.

I wonder if there's a way to bring more clarity to developers on how everything ties together and more alignment between all the pieces, without compromising on the diversity. It's a large, complex problem, and hard to do all at once. But where to even start?

If you have views and opinions you'd like to share. I'd love to talk to you too. I'll set up a way for booking conversations directly but, in the meantime, my DMs are open on Twitter. Reach out and we can grab the time to chat!

How Terra improved user engagement thanks to Dark Mode

2021-12-18T00:00:00Z

Terra, one of Brazil's largest media companies with 75 million monthly users, reduced the bounce rate by 60% and increased the pages read per session by 170% on desktop for users that prefer dark mode by providing a custom dark theme.

In this article, we'll analyze Terra's journey from identifying the size of the "dark mode" cohort, to applying a custom dark theme, and finally measuring the impact of this implementation on their main KPIs.

60_%

Reduction in Bounce Rates

170_%

More pages per session

What is dark mode? #

Historically user interfaces in devices are displayed in "light mode", which usually means displaying black text on top of light interfaces. The alternative is "dark mode", with light text on a dark background, such as gray or black.

Dark Mode has benefits for user experience. Some people prefer it for aesthetic or accessibility reasons. It has other advantages, such as preserving battery life in devices. Users can express that they prefer dark mode via a setting in their devices, which depends on the operating system. For example, the following screenshot shows what the Dark Theme configuration option looks like in devices that run Android Q:

Android Q dark theme settings.

To offer a better experience to users who prefer "dark mode", you can use the prefers-color-scheme media query, with a value of light or dark. This media query reflects the user's choice in their device. You can then load a custom dark theme for those that prefer dark. For example, by loading a "dark" CSS file, and changing values such as font and background colors. The following code shows an example of that:

@media (prefers-color-scheme: dark) {
   // apply a dark theme
}

@media (prefers-color-scheme: light) {
  // apply a light theme
}

Browser support

Chrome 76, Supported 76
Firefox 67, Supported 67
Edge 79, Supported 79
Safari 12.1, Supported 12.1

Source

Identifying the "prefers light" vs "dark" user cohorts #

At the time of writing (December 2021), Chrome Platform Status determines that approximately 22% of the web traffic comes from users with the "prefer dark" setting.

This is aggregated data, so the real percentage of users who prefer dark that come to a site can vary. For that reason, to understand the size of this group it is advisable to run in house measurement.

The following code creates an analytics dimension, to measure the performance of users that prefer light vs. dark:

function getColorScheme() {
    let colorScheme = 'Unknown';
    if (window.matchMedia) {
        if (window.matchMedia('(prefers-color-scheme: dark)').matches) {
            colorScheme = 'Dark';
        } else if (window.matchMedia('(prefers-color-scheme: light)').matches) {
            colorScheme = 'Light';
        }
    }
    return colorScheme;
}

window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};
ga.l=+new Date; ga('create', 'UA-ID', 'auto');
ga('set', 'color-scheme-preference', getColorScheme());
ga('send', 'pageview');

Terra implemented this code in their site and compared the behavior of both groups in mobile (Android) and desktop (Windows) devices. At that moment Terra wasn't providing a custom dark theme, so the goals of this experiment were:

Determining the size of the group of users who prefer dark.
Identifying patterns: for example, do users that prefer dark leave the site more quickly compared to those that prefer light?

Knowing this can inform decisions, for example: if it's necessary to provide a custom dark theme. These are the results Terra obtained after testing for 14 days:

Mobile (Android) #

In the case of mobile (Android) the numbers for bounce rate and pages per session didn't show big differences between the users that prefer "light", compared to those that prefer "dark":

Display Mode	Bounce Rate	Pages Per Session
Prefers Light	25.84%	3.96
Prefers Dark	26.10%	3.75

Desktop (Windows) #

In the case of desktop (Windows), the group of users that preferred "dark" stayed much less on the site: they had almost twice the bounce rate and read a little more than half of the pages than those users that preferred "light":

Display Mode	Bounce Rate	Pages Per Session
Prefers Light	13.20%	7.42
Prefers Dark	24.12%	4.68

Based on this data, Terra hypothesized that users who prefer "dark" stay less in desktop devices, due to the lack of support of a dark theme in their site.

As a next step Terra decided to work on a "dark theme" strategy to see if they could improve the engagement for the group of users that preferred dark.

Implementing a dark theme #

Most websites implement a dark theme by using the simple strategy shown previously of listening to user's configuration changes via the prefers-color-scheme media query and changing styles based on that.

Terra decided to give more control to the user: when they detect that they have the "prefer dark" setting turned on in their devices (via the media query), they show them a prompt to ask them if they want to navigate in "night mode". As long as the user doesn't deny the prompt (by clicking on the "Ignore" button), they honor the user's OS-setting, and apply a custom dark theme:

Terra shows a prompt to the user asking if they want to navigate in dark mode after detecting that they prefer dark in their devices.

As a complement of this strategy they provide additional configuration options in the "settings" screen, where the user can decide if they explicitly prefer "light", "dark", or want to rely on the underlying device settings.

Terra's themes configurations allow users to choose between "Dark" and "Light" themes or rely on the device's settings.

This is how Terra's "Night Mode" looks like:

Terra's dark theme, "Night Mode".

Measuring the impact of Terra's dark theme #

To measure the impact of the dark theme, Terra took the group of users that have the "Prefer Dark" setting turned on in their devices and compared engagement metrics when showing a "Light" vs. a "DarK" theme. Here are the results for mobile (Android) and desktop (Windows):

Mobile (Android) #

While bounce rates remained similar, pages and sessions almost doubled (from 2.47 to 5.24) when users were exposed to a dark theme:

Display Mode	Bounce Rate	Pages Per Session
Prefers Light	26.91%	2.47
Prefers Dark	23.91%	5.24

2_X

More pages per session

Desktop (Windows) #

Both metrics improved when showing a dark theme: bounce rates went from 27.25% to 10.82% and pages per session almost tripled (from 3.7 to 9.99).

Display Mode	Bounce Rate	Pages Per Session
Prefers Light	27.5%	3.7
Prefers Dark	10.82%	9.99

60_%

Reduction in Bounce Rates

170_%

More pages per session

Based on this data, Terra could confirm the benefits for the users from implementing a dark theme, and has decided to continue maintaining it as a core feature of the site.

Conclusion #

Dark Mode is a preference that users can turn on in their devices to change the style of the screens into dark themes. This technique has reported benefits from the user experience and for different aspects of the user's devices such as saving battery life.

In this article we saw how providing a custom dark theme improved engagement metrics for the group of Terra's users that have the preferred dark mode setting turned on in their devices.

For companies with the resources to implement an alternative dark theme this is the recommended approach. For those that don't have the time to invest in such a feature, Chrome is starting to roll out an Auto Dark Mode feature.

Web on Android

2020-07-30T00:00:00Z

The Android platform has been around for more than ten years, and since its early days it has had great support for the Web. It shipped with WebView, a component that allows developers to use the web inside their own Android Apps. More than that, Android allows developers to bring their own browser engine into the platform, fostering competition and innovation.

Developers can include the web in their Android applications in many ways. WebView is frequently used to render ads, as a layout component used along with Android UI elements, or for packaging HTML 5 games. Custom Tabs allows developers to build in-app browsers and provide a seamless navigation experience to third-party web content, and Trusted Web Activity allows developers to use their Progressive Web Apps (PWAs) in Android apps, which can be downloaded from the Play Store.

Android WebView #

WebView gives developers access to modern HTML, CSS, and JavaScript inside their Android apps, and allows content to be shipped inside the APK or hosted on the internet. It's one of Android's most flexible and powerful components, which can be used for most of the use-cases where web content is included in an Android app. From powering ad services like AdMob to building and shipping complete HTML5 games that use modern APIs such as WebGL.

But, when used to create an in-app-browser or including a PWA in an Android application, WebView lacks the security, features, and capabilities of the web platform.

The in-app browser challenge #

Over time, more and more developers have built browser experiences incorporating third-party content into their Android application, with the goal of creating a more seamless experience for their users when navigating to third-party websites. Those experiences became known as in-app browsers.

WebView has extensive support for the modern web tech stack and supports many modern web APIs, like WebGL. But WebView is primarily a web UI toolkit. It is not meant to - and does not - support all features of the web platform. When an API already has an OS-level alternative, like Web Bluetooth, or it requires browser UI to be implemented, like push notifications, it may not be supported. As the web platform evolves and adds more features that were only available to Android apps, this gap will become even larger. As app developers don't control which features are used when opening third-party content, it makes WebView a poor choice for in-app browsers or opening Progressive Web Apps. Even if WebView implemented support for all web platform features, developers would still need to write code and implement their own UI for functionality like permissions or push notifications, making it hard to achieve consistency for users.

Security Considerations for using WebView as an in-app browser #

WebView gives the embedder application full access to the rendered content, including cookies and the DOM. Those are powerful features that require a high level of trust from users.

Since WebView is not intended as a framework for building browsers, it lacks security features available in modern browsers.

Multi-process architecture and site isolation #

Browsers are designed to be secure while rendering and executing content that is untrusted. To ensure the user stays safe while navigating content that is potentially untrustworthy or even malicious, modern browsers employ techniques such as using multi-process architecture and site isolation.

Without the multi-process architecture, a crash caused by the web page can crash the entire browser app, or a vulnerability can be exploited to take control of the entire device. Site isolation adds another layer of security that makes it harder for untrustworthy sites to access and steal information from other sites.

Until Android 8.0 Oreo, the WebView renderer used the same process as the embedder application. On newer versions of the OS, and when devices are capable enough, the renderer runs in a different process. However, a single process is still shared between all pages and WebView instances running them, making it impossible to fully implement site isolation.

The lack of a multi-process architecture and site isolation is not an issue for applications that render content that they own and trust, but can be a problem for applications running untrusted third-party content, like in-app browsers, and leaves users exposed to vulnerabilities like Meltdown and Spectre, which could be used for stealing cookies, banking details, personal information, and more.

Secure UI Indicators #

It is also important to provide good security indicators to users, and browsers put a lot of effort and are always evolving in this area. However, the WebView lacks an API for checking if a site's connection is secure that allows application developers to build trustworthy security indicators. The lack of such an API could cause, for instance, a URL displayed in the address bar to not match the page displayed to the user, even over secure HTTPS connections.

Another option available to developers is embedding a browser engine in their application. Besides leading to increased application size, this approach is both complex and time-consuming.

Custom Tabs as a solution for in-app browsers #

Custom Tabs was introduced in Chrome 45 and allows developers to use a tab from the user's default browser as part of their application. Custom Tabs was originally launched by Chrome, and was therefore known as "Chrome Custom Tabs". Today it's an Android API and most popular browsers support Custom Tabs, including Chrome, Firefox, Edge, and Samsung Internet, so it's more appropriate to just call it "Custom Tabs".

Custom Tabs helps developers seamlessly integrate web content into their app experience. They also allow developers to customise the activity in which web content is shown by allowing them to customize the toolbar color, action buttons, transition animation, and more.

They also offer features that were previously unavailable when using WebView or embedding a browser engine. Since the in-app browser is powered by the user's browser, Custom Tabs shares storage with the browser, so users don't need to re-login to their favourite websites every time one of their installed apps starts an In-App browsing session.

Unlike WebViews, Custom Tabs supports all web platform features and APIs that are supported by the browser powering it.

Open Progressive Web Apps using Trusted Web Activity #

Progressive Web Apps bring many behaviors and capabilities that were once only available to platform-specific apps to the web. With the introduction of app-like behaviour, the desire from developers to re-use those experiences on Android increased, and developers started asking for ways to integrate PWAs into their apps.

Custom Tabs has support for all modern web capabilities and APIs but, since it was primarily designed to open third-party content, it has a toolbar on the top that tells the users which URL they are visiting, as well as the lock icon indicating whether the site is secure. When opening an app's own experience, the toolbar prevents the application from feeling like it is integrated with the operating system.

Trusted Web Activities was introduced in Chrome 72 and allows developers to use their PWA inside an Android app. Its protocol is similar to the Custom Tabs protocol, but introduces APIs that allow developers to verify (through Digital Asset Links) that they control both the Android app and the URL being opened and remove the URL bar when both are true.

They also introduced APIs for creating splash screens when opening the PWA or delegating web notifications to be handled by Android code. More features like support for Play Billing are coming soon.

Since URLs opened in Trusted Web Activities are expected to be PWAs and have a set of behaviors and performance characteristics, Trusted Web Activities introduces quality criteria for PWAs being opened inside it.

Limitations of the current solutions #

Developer feedback showed a need for the platform compatibility of Custom Tabs combined with the flexibility of WebView so they could, for instance, access the DOM or inject JavaScript, into their in-app browsers.

Custom Tabs is effectively a tab rendered by the user's browser, with a custom UI or with no UI at all. This means that the browser needs to honour the user's expectations around privacy and security towards the browser, making some of those features impossible.

The Web on Android team at Google is looking into alternatives and experimenting with solutions to solve those use-cases. Stay tuned for details!

Summary #

WebView is useful when an application needs HTML, CSS, and JavaScript inside their Android app, but doesn't use more advanced features and capabilities available on the modern web such as Push Notifications, Web Bluetooth and others. It is not recommended when opening content that has been designed for the modern web platform, as it may not be displayed in the way the developer intended. WebView is not recommended for creating in-app browsers. On the other hand displaying first-party web content is an area where WebViews really shine.

Trusted Web Activity should be used when the developers want to render their own Progressive Web App in fullscreen inside their Android application. It can be used as the only activity in the app or used along with other Android activities.

Custom Tabs is the recommended way for opening third-party content that is designed for the web platform, also known as in-app browsers.

Using a PWA in your Android app

2020-03-19T00:00:00Z

Start a PWA in an Android app #

Progressive Web Apps (PWA) are web applications that use app-like features to create high quality experiences that are fast, reliable, and engaging.

The web has incredible reach and offers powerful ways for users to discover new experiences. But users are also used to searching for applications in their operating system store. Those users are, in many cases, already familiar with the brand or service they're seeking and have a high level of intentionality that results in higher than average engagement metrics.

The Play Store is a store for Android apps, and developers often want to open their Progressive Web Apps from their Android apps.

Trusted Web Activity is an open standard that allows browsers to provide a fully web platform compatible container that renders PWAs inside an Android app. The feature is available in Chrome, and in development in Firefox Nightly.

Existing solutions were limited #

It has always been possible to include web experiences in an Android app, using technologies like the Android WebView or frameworks like Cordova.

The limitation with Android WebView is that it's not intended as a browser replacement. The Android WebView is a developer tool for using web UI in an Android app and it doesn't provide complete access to modern web platform features such as contact picker, or filesystem, among others.

Cordova was designed to augment the shortcomings of WebView, but the APIs are then limited to the Cordova environment. That means you need to maintain an additional codebase for using Cordova APIs for your Android app, separate from your PWA on the open web.

In addition, feature discoverability often doesn't always work as expected and compatibility issues between Android versions and OEMs can also be a problem. When using one of those solutions, developers need additional quality assurance processes and incur an extra development cost to detect and create workarounds.

Trusted Web Activity is a new container for Web apps on Android #

Developers can now use a Trusted Web Activity as a container to include a PWA as a launch activity for an Android app. The technology leverages the browser to render the PWA in full screen, ensuring the Trusted Web Activity has the same compatibility with the Web Platform features and APIs that the underlying browser does. There are also open source utilities to make implementing an Android app using a Trusted Web Activity even easier.

Another advantage not available in other solutions is that the container shares storage with the browser. Login states and users preferences are shared seamlessly across experiences.

Browser Compatibility #

The feature has been available in Chrome since version 75, with Firefox implementing it in their nightly version.

Quality Criteria #

Web developers should use a Trusted Web Activity when they want to include web content in an Android app.

Web content in a Trusted Web Activity must meet the PWA installability criteria.

Additionally, to match the behavior users expect from Android applications, Chrome 86 introduced a change where failing to handle the following scenarios is considered a crash:

Failure to verify digital asset links at application launch.
Failure to return HTTP 200 for an offline network resource request.
A navigation request returning an HTTP 404 or 5xx error".

When one of those scenarios happens in the Trusted Web Activity, it causes a user visible crash of the Android application. Check out the guidance on handling those scenarios in your service worker.

The application must also meet additional Android-specific criteria such as policy compliance.

The PWA badge in Lighthouse shows if your PWA passes the installability criteria.

Tooling #

Web developers who want to take advantage of Trusted Web Activity don't need to learn new technologies or APIs to transform their PWA into an Android Application. Together, Bubblewrap and PWABuilder provide developer tooling in the form of a library, Command Line Interface (CLI) and Graphical User Interface (GUI).

Bubblewrap #

The Bubblewrap project generates Android apps in the form of a NodeJS library and a Command Line Interface (CLI).

Bootstrapping a new project is achieved by running the tool and passing the URL of the Web Manifest:

npx @bubblewrap/cli init --manifest=https://pwa-directory.appspot.com/manifest.json

The tool can also build the project, and running the command below will output an Android application ready to be uploaded to the Play Store:

npx @bubblewrap/cli build

After running this command, a file called app-release-signed.apk will be available in the root directory for the project. This is the file that will be uploaded to the Play Store.

PWABuilder #

PWABuilder helps developers transform existing websites into Progressive Web Apps. It also integrates with Bubblewrap to provide a GUI interface to wrap those PWAs into an Android app. The PWABuilder team has put together a great blog post on how to generate an Android application using the tool.

Verifying ownership of the PWA in the Android app #

A developer building a great Progressive Web App wouldn't want another developer to build an Android app with it without their permission. To ensure this doesn't happen, the Android application must be paired with the Progressive Web App using a tool called Digital Asset Links.

Bubblewrap and PWABuilder take care of the necessary configuration on the Android application, but a last step remains, which is adding the assetlinks.json file to the PWA.

To generate this file, developers need the SHA-256 signature of the key used to sign the APK that is being downloaded by the users.

The key can be generated in multiple ways, and the easiest way to find which key that signed the APK being served to end users is to download it from the Play Store itself.

To avoid showing a broken application to users, deploy the application to a closed test channel, install it into a test device then use Peter's Asset Link Tool to generate the correct assetlinks.json file for the app. Make the generated assetlinks.json file available at /.well-known/assetlinks.json, in the domain being validated.

Where to go next #

A Progressive Web App is a high quality web experience. Trusted Web Activity is a new way to open those high quality experiences from an Android app when they meet the minimum quality criteria.

If you are getting started with Progressive Web Apps, read our guidance on how to build a great PWA. For developers who already have a PWA, use Lighthouse to verify if it meets the quality criteria.

Then, use Bubblewrap or PWABuilder to generate the Android application, upload the application to a closed test channel on the Play Store and pair it with the PWA using Peter's Asset Link Tool.

Finally, move your application from the closed test channel to production!