Thomas Steiner on web.dev

Compiling mkbitmap to WebAssembly

2023-06-29T00:00:00Z

In What is WebAssembly and where did it come from?, I explained how we ended up with the WebAssembly of today. In this article, I will show you my approach of compiling an existing C program, mkbitmap, to WebAssembly. It's more complex than the hello world example, as it includes working with files, communicating between the WebAssembly and JavaScript lands, and drawing to a canvas, but it's still manageable enough to not overwhelm you.

The article is written for web developers who want to learn WebAssembly and shows step-by-step how you might proceed if you wanted to compile something like mkbitmap to WebAssembly. As a fair warning, not getting an app or library to compile on the first run is completely normal, which is why some of the steps described below ended up not working, so I needed to backtrack and try again differently. The article doesn't show the magic final compilation command as if it had dropped from the sky, but rather describes my actual progress, some frustrations included.

About `mkbitmap` #

The mkbitmap C program reads an image and applies one or more of the following operations to it, in this order: inversion, highpass filtering, scaling, and thresholding. Each operation can be individually controlled and turned on or off. The principal use of mkbitmap is to convert color or grayscale images into a format suitable as input for other programs, particularly the tracing program potrace that forms the basis of SVGcode. As a preprocessing tool, mkbitmap is particularly useful for converting scanned line art, such as cartoons or handwritten text, to high-resolution bilevel images.

You use mkbitmap by passing it a number of options and one or multiple file names. For all details, see the tool's man page:

$ mkbitmap [options] [filename...]

The original image (Source).

First scaled, then thresholded: mkbitmap -f 2 -s 2 -t 0.48 (Source).

Get the code #

The first step is to obtain the source code of mkbitmap. You can find it on the project's website. At the time of this writing, potrace-1.16.tar.gz is the latest version.

Compile and install locally #

The next step is to compile and install the tool locally to get a feeling for how it behaves. The INSTALL file contains the following instructions:

cd to the directory containing the package's source code and type ./configure to configure the package for your system.

Running configure might take a while. While running, it prints some messages telling which features it is checking for.
Type make to compile the package.
Optionally, type make check to run any self-tests that come with the package, generally using the just-built uninstalled binaries.
Type make install to install the programs and any data files and documentation. When installing into a prefix owned by root, it is recommended that the package be configured and built as a regular user, and only the make install phase executed with root privileges.

By following these steps, you should end up with two executables, potrace and mkbitmap—the latter is the focus of this article. You can verify it worked correctly by running mkbitmap --version. Here is the output of all four steps from my machine, heavily trimmed for brevity:

Step 1, ./configure:

$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
[…]
config.status: executing libtool commands

Step 2, make:

$ make
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-recursive
Making all in src
clang -DHAVE_CONFIG_H -I. -I..     -g -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c
mv -f .deps/main.Tpo .deps/main.Po
[…]
make[2]: Nothing to be done for `all-am'.

Step 3, make check:

$ make check
Making check in src
make[1]: Nothing to be done for `check'.
Making check in doc
make[1]: Nothing to be done for `check'.
[…]
============================================================================
Testsuite summary for potrace 1.16
============================================================================
# TOTAL: 8
# PASS:  8
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[1]: Nothing to be done for `check-am'.

Step 4, sudo make install:

$ sudo make install
Password:
Making install in src
 .././install-sh -c -d '/usr/local/bin'
  /bin/sh ../libtool   --mode=install /usr/bin/install -c potrace mkbitmap '/usr/local/bin'
[…]
make[2]: Nothing to be done for `install-data-am'.

To check if it worked, run mkbitmap --version:

$ mkbitmap --version
mkbitmap 1.16. Copyright (C) 2001-2019 Peter Selinger.

If you get the version details, you have successfully compiled and installed mkbitmap. Next, make the equivalent of these steps work with WebAssembly.

Compile `mkbitmap` to WebAssembly #

Emscripten is a tool for compiling C/C++ programs to WebAssembly. Emscripten's Building Projects documentation states the following:

Building large projects with Emscripten is very easy. Emscripten provides two simple scripts that configure your makefiles to use emcc as a drop-in replacement for gcc—in most cases the rest of your project's current build system remains unchanged.

The documentation then goes on (a little edited for brevity):

Consider the case where you normally build with the following commands:

./configure
make

To build with Emscripten, you would instead use the following commands:

emconfigure ./configure
emmake make

So essentially ./configure becomes emconfigure ./configure and make becomes emmake make. The following demonstrates how to do this with mkbitmap.

Step 0, make clean:

$ make clean
Making clean in src
 rm -f potrace mkbitmap
test -z "" || rm -f
rm -rf .libs _libs
[…]
rm -f *.lo

Step 1, emconfigure ./configure:

$ emconfigure ./configure
configure: ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
[…]
config.status: executing libtool commands

Step 2, emmake make:

$ emmake make
make: make
/Applications/Xcode.app/Contents/Developer/usr/bin/make  all-recursive
Making all in src
/opt/homebrew/Cellar/emscripten/3.1.36/libexec/emcc -DHAVE_CONFIG_H -I. -I..     -g -O2 -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c
mv -f .deps/main.Tpo .deps/main.Po
[…]
make[2]: Nothing to be done for `all'.

If everything went well, there should now be .wasm files somewhere in the directory. You can find them by running find . -name "*.wasm":

$ find . -name "*.wasm"
./a.wasm
./src/mkbitmap.wasm
./src/potrace.wasm

The two last ones look promising, so cd into the src/ directory. There are now also two new corresponding files, mkbitmap and potrace. For this article, only mkbitmap is relevant. The fact that they don't have the .js extension is a little confusing, but they are in fact JavaScript files, verifiable with a quick head call:

$ cd src/
$ head -n 20 mkbitmap
// include: shell.js
// The Module object: Our interface to the outside world. We import
// and export values on it. There are various ways Module can be used:
// 1. Not defined. We create it here
// 2. A function parameter, function(Module) { ..generated code.. }
// 3. pre-run appended it, var Module = {}; ..generated code..
// 4. External script tag defines var Module.
// We need to check if Module already exists (e.g. case 3 above).
// Substitution will be replaced with actual code on later stage of the build,
// this way Closure Compiler will not mangle it (e.g. case 4. above).
// Note that if you want to run closure, and also to use Module
// after the generated code, you will need to define   var Module = {};
// before the code. Then that object will be used in the code, and you
// can continue to use Module afterwards as well.
var Module = typeof Module != 'undefined' ? Module : {};

// --pre-jses are emitted after the Module integration code, so that they can
// refer to Module (if they choose; they can also define Module)

Rename the JavaScript file to mkbitmap.js by calling mv mkbitmap mkbitmap.js (and mv potrace potrace.js respectively if you want). Now it's time for the first test to see if it worked by executing the file with Node.js on the command line by running node mkbitmap.js --version:

$ node mkbitmap.js --version
mkbitmap 1.16. Copyright (C) 2001-2019 Peter Selinger.

You have successfully compiled mkbitmap to WebAssembly. Now the next step is to make it work in the browser.

`mkbitmap` with WebAssembly in the browser #

Copy the mkbitmap.js and the mkbitmap.wasm files to a new directory called mkbitmap and create an index.html HTML boilerplate file that loads the mkbitmap.js JavaScript file.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>mkbitmap</title>
  </head>
  <body>
    <script src="mkbitmap.js"></script>
  </body>
</html>

Start a local server that serves the mkbitmap directory and open it in your browser. You should see a prompt that asks you for input. This is as expected, since, according to the tool's man page, "[i]f no filename arguments are given, then mkbitmap acts as a filter, reading from standard input", which for Emscripten by default is a prompt().

Prevent automatic execution #

To stop mkbitmap from executing immediately and instead make it wait for user input, you need to understand Emscripten's Module object. Module is a global JavaScript object with attributes that Emscripten-generated code calls at various points in its execution. You can provide an implementation of Module to control the execution of code. When an Emscripten application starts up, it looks at the values on the Module object and applies them.

In the case of mkbitmap, set Module.noInitialRun to true to prevent the initial run that caused the prompt to appear. Create a script called script.js, include it before the <script src="mkbitmap.js"></script> in index.html and add the following code to script.js. When you now reload the app, the prompt should be gone.

var Module = {
  // Don't run main() at page load
  noInitialRun: true,
};

Create a modular build with some more build flags #

To provide input to the app, you can use Emscripten's file system support in Module.FS. The Including File System Support section of the documentation states:

Emscripten decides whether to include file system support automatically. Many programs don't need files, and file system support is not negligible in size, so Emscripten avoids including it when it doesn't see a reason to. That means that if your C/C++ code does not access files, then the FS object and other file system APIs will not be included in the output. And, on the other hand, if your C/C++ code does use files, then file system support will be automatically included.

Unfortunately mkbitmap is one of the cases where Emscripten does not automatically include file system support, so you need to explicitly tell it to do so. This means you need to follow the emconfigure and emmake steps described previously, with a couple more flags set via a CFLAGS argument. The following flags may come in useful for other projects, too.

Set -sFILESYSTEM=1 so file system support is included.
Set -sEXPORTED_RUNTIME_METHODS=FS,callMain so Module.FS and Module.callMain are exported.
Set -sMODULARIZE=1 and -sEXPORT_ES6 to generate a modern ES6 module.
Set -sINVOKE_RUN=0 to prevent the initial run that caused the prompt to appear.

Also, in this particular case, you need to set the --host flag to wasm32 to tell the configure script that you are compiling for WebAssembly.

The final emconfigure command looks like this:

$ emconfigure ./configure --host=wasm32 CFLAGS='-sFILESYSTEM=1 -sEXPORTED_RUNTIME_METHODS=FS,callMain -sMODULARIZE=1 -sEXPORT_ES6 -sINVOKE_RUN=0'

Do not forget to run emmake make again and copy the freshly created files over to the mkbitmap folder.

Modify index.html so that it only loads the ES module script.js, from which you then import the mkbitmap.js module.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>mkbitmap</title>
  </head>
  <body>
    <!-- No longer load `mkbitmap.js` here -->
    <script src="script.js" type="module"></script>
  </body>
</html>

// This is `script.js`.
import loadWASM from './mkbitmap.js';

const run = async () => {
  const Module = await loadWASM();
  console.log(Module);
};

run();

When you open the app now in the browser, you should see the Module object logged to the DevTools console, and the prompt is gone, since the main() function of mkbitmap is no longer called at the start.

Manually execute the main function #

The next step is to manually call mkbitmap's main() function by running Module.callMain(). The callMain() function takes an array of arguments, which match one-by-one what you would pass on the command line. If on the command line you would run mkbitmap -v, you would call Module.callMain(['-v']) in the browser. This logs the mkbitmap version number to the DevTools console.

// This is `script.js`.
import loadWASM from './mkbitmap.js';

const run = async () => {
  const Module = await loadWASM();
  Module.callMain(['-v']);
};

run();

Redirect the standard output #

The standard output (stdout) by default is the console. However, you can redirect it to something else, for example, a function that stores the output to a variable. This means you can add the output to the HTML by setting the Module.print property.

// This is `script.js`.
import loadWASM from './mkbitmap.js';

const run = async () => {
  let consoleOutput = 'Powered by ';
  const Module = await loadWASM({
    print: (text) => (consoleOutput += text),
  });
  Module.callMain(['-v']);
  document.body.textContent = consoleOutput;
};

run();

Get the input file into the memory file system #

To get the input file into the memory file system, you need the equivalent of mkbitmap filename on the command line. To understand how I approach this, first some background on how mkbitmap expects its input and creates its output.

Supported input formats of mkbitmap are PNM (PBM, PGM, PPM) and BMP. The output formats are PBM for bitmaps, and PGM for graymaps. If a filename argument is given, mkbitmap will by default create an output file whose name is obtained from the input file name by changing its suffix to .pbm. For example, for the input file name example.bmp, the output file name would be example.pbm.

Emscripten provides a virtual file system that simulates the local file system, so that native code using synchronous file APIs can be compiled and run with little or no change. For mkbitmap to read an input file as if it was passed as a filename command line argument, you need to use the FS object that Emscripten provides.

The FS object is backed by an in-memory file system (commonly referred to as MEMFS) and has a writeFile() function that you use to write files to the virtual file system. You use writeFile() as shown in the following code sample.

To verify the file write operation worked, run the FS object's readdir() function with the parameter '/'. You will see example.bmp and a number of default files that are always created automatically.

Note that the previous call to Module.callMain(['-v']) for printing the version number was removed. This is due to the fact that Module.callMain() is a function that generally expects to only be run once.

// This is `script.js`.
import loadWASM from './mkbitmap.js';

const run = async () => {
  const Module = await loadWASM();
  const buffer = await fetch('https://example.com/example.bmp').then((res) => res.arrayBuffer());
  Module.FS.writeFile('example.bmp', new Uint8Array(buffer));
  console.log(Module.FS.readdir('/'));
};

run();

First actual execution #

With everything in place, execute mkbitmap by running Module.callMain(['example.bmp']). Log the contents of the MEMFS' '/' folder, and you should see the newly created example.pbm output file next to the example.bmp input file.

// This is `script.js`.
import loadWASM from './mkbitmap.js';

const run = async () => {
  const Module = await loadWASM();
  const buffer = await fetch('https://example.com/example.bmp').then((res) => res.arrayBuffer());
  Module.FS.writeFile('example.bmp', new Uint8Array(buffer));
  Module.callMain(['example.bmp']);
  console.log(Module.FS.readdir('/'));
};

run();

Get the output file out of the memory file system #

The FS object's readFile() function enables getting the example.pbm created in the last step out of the memory file system. The function returns a Uint8Array that you convert to a File object and save to disk, as browsers don't generally support PBM files for direct in-browser viewing. (There are more elegant ways to save a file, but using a dynamically created <a download> is the most widely supported one.) Once the file is saved, you can open it in your favorite image viewer.

// This is `script.js`.
import loadWASM from './mkbitmap.js';

const run = async () => {
  const Module = await loadWASM();
  const buffer = await fetch('https://example.com/example.bmp').then((res) => res.arrayBuffer());
  Module.FS.writeFile('example.bmp', new Uint8Array(buffer));
  Module.callMain(['example.bmp']);
  const output = Module.FS.readFile('example.pbm', { encoding: 'binary' });
  const file = new File([output], 'example.pbm', {
    type: 'image/x-portable-bitmap',
  });
  const a = document.createElement('a');
  a.href = URL.createObjectURL(file);
  a.download = file.name;
  a.click();
};

run();

Add an interactive UI #

To this point, the input file is hardcoded and mkbitmap runs with default parameters. The final step is to let the user dynamically select an input file, tweak the mkbitmap parameters, and then run the tool with the selected options.

// Corresponds to `mkbitmap -o output.pbm input.bmp -s 8 -3 -f 4 -t 0.45`.
Module.callMain(['-o', 'output.pbm', 'input.bmp', '-s', '8', '-3', '-f', '4', '-t', '0.45']);

The PBM image format is not particularly hard to parse, so with some JavaScript code, you could even show a preview of the output image. See the source code of the embedded demo below for one way to do this.

Conclusion #

Congratulations, you have successfully compiled mkbitmap to WebAssembly and made it work in the browser! There were some dead ends and you had to compile the tool more than once until it worked, but as I wrote above, that's part of the experience. Also remember the StackOverflow's webassembly tag if you get stuck. Happy compiling!

Acknowledgements #

This article was reviewed by Sam Clegg and Rachel Andrew.

What is WebAssembly and where did it come from?

2023-06-29T00:00:00Z

Ever since the web became a platform not just for documents but also for apps, some of the most advanced apps have pushed web browsers to their limits. The approach of going "closer to the metal" by interfacing with lower-level languages in order to improve performance is encountered in many higher-level languages. As an example, Java has the Java Native Interface. For JavaScript, this lower-level language is WebAssembly. In this article, you will discover what assembly language is, and why it can be useful on the web, then learn how WebAssembly was created via the interim solution of asm.js.

Assembly language #

Have you ever programmed in assembly language? In computer programming, assembly language, often referred to simply as Assembly and commonly abbreviated as ASM or asm, is any low-level programming language with a very strong correspondence between the instructions in the language and the architecture's machine code instructions.

For example, looking at the Intel® 64 and IA-32 Architectures (PDF), the MUL instruction (for multiplication) performs an unsigned multiplication of the first operand (destination operand) and the second operand (source operand), and stores the result in the destination operand. Very simplified, the destination operand is an implied operand located in register AX, and the source operand is located in a general-purpose register like CX. The result is stored again in register AX. Consider the following x86 code example:

mov ax, 5  ; Set the value of register AX to 5.
mov cx, 10 ; Set the value of register CX to 10.
mul cx     ; Multiply the value of register AX (5)
           ; and the value of register CX (10), and
           ; store the result in register AX.

For comparison, if tasked with the objective of multiplying 5 and 10, you would probably write code similar to the following in JavaScript:

const factor1 = 5;
const factor2 = 10;
const result = factor1 * factor2;

The advantage of going the assembly route is that such low-level and machine-optimized code is much more efficient than high-level and human-optimized code. In the preceding case it doesn't matter, but you can imagine that for more complex operations, the difference can be significant.

As the name suggests, x86 code is dependent on the x86 architecture. What if there were a way of writing assembly code that was not dependent on a specific architecture, but that would inherit the performance benefits of assembly?

asm.js #

The first step to writing assembly code with no architecture dependencies was asm.js, a strict subset of JavaScript that could be used as a low-level, efficient target language for compilers. This sub-language effectively described a sandboxed virtual machine for memory-unsafe languages like C or C++. A combination of static and dynamic validation allowed JavaScript engines to employ an ahead-of-time (AOT) optimizing compilation strategy for valid asm.js code. Code written in statically-typed languages with manual memory management (such as C) was translated by a source-to-source compiler such as the early Emscripten (based on LLVM).

Performance was improved by limiting language features to those amenable to AOT. Firefox 22 was the first browser to support asm.js, released under the name OdinMonkey. Chrome added asm.js support in version 61. While asm.js still works in browsers, it has been superseded by WebAssembly. The reason to use asm.js at this point would be as an alternative for browsers that don't have WebAssembly support.

WebAssembly #

WebAssembly is a low-level assembly-like language with a compact binary format that runs with near-native performance and provides languages such as C/C++ and Rust, and many more with a compilation target so that they run on the web. Support for memory-managed languages such as Java, Kotlin, and Dart is in the works and should become available soon. WebAssembly is designed to run alongside JavaScript, allowing both to work together.

Apart from the browser, WebAssembly programs also run in other runtimes thanks to WASI, the WebAssembly System Interface, a modular system interface for WebAssembly. WASI is created to be portable across operating systems, with the objective of being secure and the ability to run in a sandboxed environment.

WebAssembly code (binary code, that is, bytecode) is intended to be run on a portable virtual stack machine (VM). The bytecode is designed to be faster to parse and execute than JavaScript and to have a compact code representation.

Conceptual execution of instructions proceeds by way of a traditional program counter that advances through the instructions. In practice, most Wasm engines compile the Wasm bytecode to machine code, and then execute that. Instructions fall into two categories:

Control instructions that form control constructs and pop their argument value(s) off the stack, may change the program counter, and push result value(s) onto the stack.
Simple instructions that pop their argument value(s) from the stack, apply an operator to the values, and then push the result value(s) onto the stack, followed by an implicit advancement of the program counter.

Going back to the example before, the following WebAssembly code would be equivalent to the x86 code from the beginning of the article:

i32.const 5  ; Push the integer value 5 onto the stack.
i32.const 10 ; Push the integer value 10 onto the stack.
i32.mul      ; Pop the two most recent items on the stack,
             ; multiply them, and push the result onto the stack.

While asm.js is implemented all in software, that is, its code can run in any JavaScript engine (even if unoptimized), WebAssembly required new functionality that all browser vendors agreed on. Announced in 2015 and first released in March 2017, WebAssembly became a W3C recommendation on December 5, 2019. The W3C maintains the standard with contributions from all major browser vendors and other interested parties. Since 2017, browser support is universal.

Browser support

Chrome 57, Supported 57
Firefox 52, Supported 52
Edge 16, Supported 16
Safari 11, Supported 11

Source

WebAssembly has two representations: textual and binary. What you see above is the textual representation.

Textual representation #

The textual representation is based on S-expressions and commonly uses the file extension .wat (for WebAssembly text format). If you really wanted to, you could write it by hand. Taking the multiplication example from above and making it more useful by no longer hardcoding the factors, you can probably make sense of the following code:

(module
  (func $mul (param $factor1 i32) (param $factor2 i32) (result i32)
    local.get $factor1
    local.get $factor2
    i32.mul)
  (export "mul" (func $mul))
)

Binary representation #

The binary format that uses the file extension .wasm is not meant for human consumption, let alone human creation. Using a tool like wat2wasm, you can convert the above code to the following binary representation. (The comments are not usually part of the binary representation, but added by the wat2wasm tool for better understandability.)

0000000: 0061 736d                             ; WASM_BINARY_MAGIC
0000004: 0100 0000                             ; WASM_BINARY_VERSION
; section "Type" (1)
0000008: 01                                    ; section code
0000009: 00                                    ; section size (guess)
000000a: 01                                    ; num types
; func type 0
000000b: 60                                    ; func
000000c: 02                                    ; num params
000000d: 7f                                    ; i32
000000e: 7f                                    ; i32
000000f: 01                                    ; num results
0000010: 7f                                    ; i32
0000009: 07                                    ; FIXUP section size
; section "Function" (3)
0000011: 03                                    ; section code
0000012: 00                                    ; section size (guess)
0000013: 01                                    ; num functions
0000014: 00                                    ; function 0 signature index
0000012: 02                                    ; FIXUP section size
; section "Export" (7)
0000015: 07                                    ; section code
0000016: 00                                    ; section size (guess)
0000017: 01                                    ; num exports
0000018: 03                                    ; string length
0000019: 6d75 6c                          mul  ; export name
000001c: 00                                    ; export kind
000001d: 00                                    ; export func index
0000016: 07                                    ; FIXUP section size
; section "Code" (10)
000001e: 0a                                    ; section code
000001f: 00                                    ; section size (guess)
0000020: 01                                    ; num functions
; function body 0
0000021: 00                                    ; func body size (guess)
0000022: 00                                    ; local decl count
0000023: 20                                    ; local.get
0000024: 00                                    ; local index
0000025: 20                                    ; local.get
0000026: 01                                    ; local index
0000027: 6c                                    ; i32.mul
0000028: 0b                                    ; end
0000021: 07                                    ; FIXUP func body size
000001f: 09                                    ; FIXUP section size
; section "name"
0000029: 00                                    ; section code
000002a: 00                                    ; section size (guess)
000002b: 04                                    ; string length
000002c: 6e61 6d65                       name  ; custom section name
0000030: 01                                    ; name subsection type
0000031: 00                                    ; subsection size (guess)
0000032: 01                                    ; num names
0000033: 00                                    ; elem index
0000034: 03                                    ; string length
0000035: 6d75 6c                          mul  ; elem name 0
0000031: 06                                    ; FIXUP subsection size
0000038: 02                                    ; local name type
0000039: 00                                    ; subsection size (guess)
000003a: 01                                    ; num functions
000003b: 00                                    ; function index
000003c: 02                                    ; num locals
000003d: 00                                    ; local index
000003e: 07                                    ; string length
000003f: 6661 6374 6f72 31            factor1  ; local name 0
0000046: 01                                    ; local index
0000047: 07                                    ; string length
0000048: 6661 6374 6f72 32            factor2  ; local name 1
0000039: 15                                    ; FIXUP subsection size
000002a: 24                                    ; FIXUP section size

Compiling to WebAssembly #

As you see, neither .wat nor .wasm are particularly very human-friendly. This is where a compiler like Emscripten comes into play. It lets you compile from higher-level languages like C and C++. There are other compilers for other languages like Rust and many more. Consider the following C code:

#include <stdio.h>

int main() {
  printf("Hello World\n");
  return 0;
}

Usually, you would compile this C program with the compiler gcc.

$ gcc hello.c -o hello

With Emscripten installed, you compile it to WebAssembly using the emcc command and almost the same arguments:

$ emcc hello.c -o hello.html

This will create a hello.wasm file and the HTML wrapper file hello.html. When you serve the file hello.html from a web server, you will see "Hello World" printed to the DevTools console.

There's also a way to compile to WebAssembly without the HTML wrapper:

$ emcc hello.c -o hello.js

As before, this will create a hello.wasm file, but this time a hello.js file instead of the HTML wrapper. To test, you run the resulting JavaScript file hello.js with, for example, Node.js:

$ node hello.js
Hello World

Learn more #

This brief introduction to WebAssembly is just the tip of the iceberg. Learn more about WebAssembly in the WebAssembly documentation on MDN and consult the Emscripten documentation. Truth be told, working with WebAssembly can feel a bit like the How to draw an owl meme, especially since web developers familiar with HTML, CSS, and JavaScript are not necessarily versed in the to-be-compiled-from languages like C. Luckily there are channels like StackOverflow's webassembly tag where experts are often happy to help if you ask nicely.

Acknowledgements #

This article was reviewed by Jakob Kummerow, Derek Schuff, and Rachel Andrew.

The origin private file system

2023-06-08T00:00:00Z

Browser support #

The origin private file system is supported by modern browsers and is standardized by the Web Hypertext Application Technology Working Group (WHATWG) in the File System Living Standard.

Browser support

Chrome 86, Supported 86
Firefox 111, Supported 111
Edge 86, Supported 86
Safari 15.2, Supported 15.2

Source

Motivation #

When you think of files on your computer, you probably think about a file hierarchy: files organized in folders that you can explore with your operating system's file explorer. For example, on Windows, for a user called Tom, their To Do list might live in C:\Users\Tom\Documents\ToDo.txt. In this example, ToDo.txt is the file name, and Users, Tom, and Documents are folder names. C:\ on Windows represents the root directory of the drive.

Traditional way of working with files on the web #

To edit the To Do list in a web application, this is the traditional flow:

The user uploads the file to a server or opens it on the client with <input type="file">.
The user makes their changes, and then downloads the resulting file with an injected <a download="ToDo.txt> that you programmatically click() via JavaScript.
For opening folders, you use a special attribute in <input type="file" webkitdirectory>, which, despite its proprietary name, has practically universal browser support.

Modern way of working with files on the web #

This flow is not representative of how users think of editing files, and means users end up with downloaded copies of their input files. Therefore, the File System Access API introduced three picker methods—showOpenFilePicker(), showSaveFilePicker(), and showDirectoryPicker()—that do exactly what their name suggests. They enable a flow as follows:

Open ToDo.txt with showOpenFilePicker(), and get a FileSystemFileHandle object.
From the FileSystemFileHandle object, get a File by calling the file handle's getFile() method.
Modify the file, then call requestPermission({mode: 'readwrite'}) on the handle.
If the user accepts the permission request, save the changes back to the original file.
Alternatively, call showSaveFilePicker() and let the user pick a new file. (If the user picks a previously opened file, its contents will be overwritten.) For repeat saves, you can keep the file handle around, so you don't have to show the file save dialog again.

Restrictions of working with files on the web #

Files and folders that are accessible via these methods live in what can be called the user-visible file system. Files saved from the web, and executable files specifically, are marked with the mark of the web, so there's an additional warning the operating system can show before a potentially dangerous file gets executed. As an additional security feature, files obtained from the web are also protected by Safe Browsing, which, for the sake of simplicity and in the context of this article, you can think of as a cloud-based virus scan. When you write data to a file using the File System Access API, writes are not in-place, but use a temporary file. The file itself is not modified unless it passes all these security checks. As you can imagine, this work makes file operations relatively slow, despite improvements applied where possible, for example, on macOS. Still every write() call is self-contained, so under the hood it opens the file, seeks to the given offset, and finally writes data.

Files as the foundation of processing #

At the same time, files are an excellent way to record data. For example, SQLite stores entire databases in a single file. Another example are mipmaps used in image processing. Mipmaps are pre-calculated, optimized sequences of images, each of which is a progressively lower resolution representation of the previous, which makes many operations like zooming faster. So how can web applications get the benefits of files, but without the performance costs of traditional web-based file processing? The answer is the origin private file system.

The user-visible versus the origin private file system #

Unlike the user-visible file system browsed via the operating system's file explorer, with files and folders you can read, write, move, and rename, the origin private file system is not meant to be seen by users. Files and folders in the origin private file system, as the name suggests, are private, and more concretely, private to the origin of a site. Discover the origin of a page by typing location.origin in the DevTools Console. For example, the origin of the page https://developer.chrome.com/articles/ is https://developer.chrome.com (that is, the part /articles is not part of the origin). You can read more about the theory of origins in Understanding "same-site" and "same-origin". All pages that share the same origin can see the same origin private file system data, so https://developer.chrome.com/docs/extensions/mv3/getstarted/extensions-101/ can see the same details as the previous example. Each origin has its own independent origin private file system, which means the origin private file system of https://developer.chrome.com is completely distinct from the one of, say, https://web.dev. On Windows, the root directory of the user-visible file system is C:\. The equivalent for the origin private file system is an initially empty root directory per origin accessed by calling the asynchronous method navigator.storage.getDirectory(). For a comparison of the user-visible file system and the origin private file system, see the following diagram. The diagram shows that apart from the root directory, everything else is conceptually the same, with a hierarchy of files and folders to organize and arrange as needed for your data and storage needs.

Specifics of the origin private file system #

Just like other storage mechanisms in the browser (for example, localStorage or IndexedDB), the origin private file system is subject to browser quota restrictions. When a user clears all browsing data or all site data, the origin private file system will be deleted, too. Call navigator.storage.estimate() and in the resulting response object see the usage entry to see how much storage your app already consumes, which is broken down by storage mechanism in the usageDetails object, where you want to look at the fileSystem entry specifically. Since the origin private file system is not visible to the user, there are no permissions prompts and no Safe Browsing checks.

Getting access to the root directory #

To get access to the root directory, run the command below. You end up with an empty directory handle, more specifically, a FileSystemDirectoryHandle.

const opfsRoot = await navigator.storage.getDirectory();
// A FileSystemDirectoryHandle whose type is "directory"
// and whose name is "".
console.log(opfsRoot);

Main thread or Web Worker #

There are two ways of using the origin private file system: on the main thread or in a Web Worker. Web Workers cannot block the main thread, which means in this context APIs can be synchronous, a pattern generally disallowed on the main thread. Synchronous APIs can be faster as they avoid having to deal with promises, and file operations are typically synchronous in languages like C that can be compiled to WebAssembly.

// This is synchronous C code.
FILE *f;
f = fopen("example.txt", "w+");
fputs("Some text\n", f);
fclose(f);

If you need the fastest possible file operations and/or you deal with WebAssembly, skip down to Using the origin private file system in a Web Worker. Else, you can read on.

Using the origin private file system on the main thread #

Creating new files and folders #

Once you have a root folder, create files and folders using the getFileHandle() and the getDirectoryHandle() methods respectively. By passing {create: true}, the file or folder will be created if it doesn't exist. Build up a hierarchy of files by calling these functions using a newly created directory as the starting point.

const fileHandle = await opfsRoot
    .getFileHandle('my first file', {create: true});
const directoryHandle = await opfsRoot
    .getDirectoryHandle('my first folder', {create: true});
const nestedFileHandle = await directoryHandle
    .getFileHandle('my first nested file', {create: true});
const nestedDirectoryHandle = await directoryHandle
    .getDirectoryHandle('my first nested folder', {create: true});

Accessing existing files and folders #

If you know their name, access previously created files and folders by calling the getFileHandle() or the getDirectoryHandle() methods, passing in the name of the file or folder.

const existingFileHandle = await opfsRoot.getFileHandle('my first file');
const existingDirectoryHandle = await opfsRoot
    .getDirectoryHandle('my first folder);

Getting the file associated with a file handle for reading #

A FileSystemFileHandle represents a file on the file system. To obtain the associated File, use the getFile() method. A File object is a specific kind of Blob, and can be used in any context that a Blob can. In particular, FileReader, URL.createObjectURL(), createImageBitmap(), and XMLHttpRequest.send() accept both Blobs and Files. If you will, obtaining a File from a FileSystemFileHandle "frees" the data, so you can access it and make it available to the user-visible file system.

const file = await fileHandle.getFile();
console.log(await file.text());

Writing to a file by streaming #

Stream data into a file by calling createWritable() which creates a FileSystemWritableFileStream to that you then write() the contents. At the end, you need to close() the stream.

const contents = 'Some text';
// Get a writable stream.
const writable = await fileHandle.createWritable();
// Write the contents of the file to the stream.
await writable.write(contents);
// Close the stream, which persists the contents.
await writable.close();

Deleting files and folders #

Delete files and folders by calling their file or directory handle's particular remove() method. To delete a folder including all subfolders, pass the {recursive: true} option.

await fileHandle.remove();
await directoryHandle.remove({recursive: true});

As an alternative, if you know the name of the to-be-deleted file or folder in a directory, use the removeEntry() method.

directoryHandle.removeEntry('my first nested file');

Moving and renaming files and folders #

Rename and move files and folders using the move() method. Moving and renaming can happen together or in isolation.

// Rename a file.
await fileHandle.move('my first renamed file');
// Move a file to another directory.
await fileHandle.move(nestedDirectoryHandle);
// Move a file to another directory and rename it.
await fileHandle
    .move(nestedDirectoryHandle, 'my first renamed and now nested file');

Resolving the path of a file or folder #

To learn where a given file or folder is located in relation to a reference directory, use the resolve() method, passing it a FileSystemHandle as the argument. To obtain the full path of a file or folder in the origin private file system, use the root directory as the reference directory obtained via navigator.storage.getDirectory().

const relativePath = await opfsRoot.resolve(nestedDirectoryHandle);
// `relativePath` is `['my first folder', 'my first nested folder']`.

Checking if two file or folder handles point to the same file or folder #

Sometimes you have two handles and don't know if they point at the same file or folder. To check whether this is the case, use the isSameEntry() method.

fileHandle.isSameEntry(nestedFileHandle);
// Returns `false`.

Listing the contents of a folder #

FileSystemDirectoryHandle is an asynchronous iterator that you iterate over with a for await…of loop. As an asynchronous iterator, it also supports the entries(), the values(), and the keys() methods, from which you can choose depending on what information you need:

for await (let [name, handle] of directoryHandle) {}
for await (let [name, handle] of directoryHandle.entries()) {}
for await (let handle of directoryHandle.values()) {}
for await (let name of directoryHandle.keys()) {}

Recursively listing the contents of a folder and all subfolders #

Dealing with asynchronous loops and functions paired with recursion is easy to get wrong. The function below can serve as a starting point for listing the contents of a folder and all its subfolders, including all files and their sizes. You can simplify the function if you don't need the file sizes by, where it says directoryEntryPromises.push, not pushing the handle.getFile() promise, but the handle directly.

  const getDirectoryEntriesRecursive = async (
    directoryHandle,
    relativePath = '.',
  ) => {
    const fileHandles = [];
    const directoryHandles = [];
    const entries = {};
    // Get an iterator of the files and folders in the directory.
    const directoryIterator = directoryHandle.values();
    const directoryEntryPromises = [];
    for await (const handle of directoryIterator) {
      const nestedPath = `${relativePath}/${handle.name}`;
      if (handle.kind === 'file') {
        fileHandles.push({ handle, nestedPath });
        directoryEntryPromises.push(
          handle.getFile().then((file) => {
            return {
              name: handle.name,
              kind: handle.kind,
              size: file.size,
              type: file.type,
              lastModified: file.lastModified,
              relativePath: nestedPath,
              handle
            };
          }),
        );
      } else if (handle.kind === 'directory') {
        directoryHandles.push({ handle, nestedPath });
        directoryEntryPromises.push(
          (async () => {
            return {
              name: handle.name,
              kind: handle.kind,
              relativePath: nestedPath,
              entries:
                  await getDirectoryEntriesRecursive(handle, nestedPath),
              handle,
            };
          })(),
        );
      }
    }
    const directoryEntries = await Promise.all(directoryEntryPromises);
    directoryEntries.forEach((directoryEntry) => {
      entries[directoryEntry.name] = directoryEntry;
    });
    return entries;
  };

Using the origin private file system in a Web Worker #

As outlined before, Web Workers can't block the main thread, which is why in this context synchronous methods are allowed.

Getting a synchronous access handle #

The entry point to the fastest possible file operations is a FileSystemSyncAccessHandle, obtained from a regular FileSystemFileHandle by calling createSyncAccessHandle().

const fileHandle = await opfsRoot
    .getFileHandle('my highspeed file.txt', {create: true});
const syncAccessHandle = await fileHandle.createSyncAccessHandle();

Synchronous in-place file methods #

Once you have a synchronous access handle, you get access to fast in-place file methods that are all synchronous.

getSize(): Returns the size of the file in bytes.
write(): Writes the content of a buffer into the, optionally at a given offset, and returns the number of written bytes. Checking the returned number of written bytes allows callers to detect and handle errors and partial writes.
read(): Reads the contents of the file into a buffer, optionally at a given offset.
truncate(): Resizes the file to the given size.
flush(): Ensures that the contents of the file contain all the modifications done through write().
close(): Closes the access handle.

Here is an example that uses all the methods mentioned above.

const opfsRoot = await navigator.storage.getDirectory();
const fileHandle = await opfsRoot.getFileHandle('fast', {create: true});
const accessHandle = await fileHandle.createSyncAccessHandle();

const textEncoder = new TextEncoder();
const textDecoder = new TextDecoder();

// Initialize this variable for the size of the file.
let size;
// The current size of the file, initially `0`.
size = accessHandle.getSize();
// Encode content to write to the file.
const content = textEncoder.encode('Some text');
// Write the content at the beginning of the file.
accessHandle.write(content, {at: size});
// Flush the changes.
accessHandle.flush();
// The current size of the file, now `9` (the length of "Some text").
size = accessHandle.getSize();

// Encode more content to write to the file.
const moreContent = textEncoder.encode('More content');
// Write the content at the end of the file.
accessHandle.write(moreContent, {at: size});
// Flush the changes.
accessHandle.flush();
// The current size of the file, now `21` (the length of
// "Some textMore content").
size = accessHandle.getSize();

// Prepare a data view of the length of the file.
const dataView = new DataView(new ArrayBuffer(size));

// Read the entire file into the data view.
accessHandle.read(dataView);
// Logs `"Some textMore content"`.
console.log(textDecoder.decode(dataView));

// Read starting at offset 9 into the data view.
accessHandle.read(dataView, {at: 9});
// Logs `"More content"`.
console.log(textDecoder.decode(dataView));

// Truncate the file after 4 bytes.
accessHandle.truncate(4);

Copying a file from the origin private file system to the user-visible file system #

As mentioned above, moving files from the origin private file system to the user-visible file system isn't possible, but you can copy files. Since showSaveFilePicker() is only exposed on the main thread, but not in the Worker thread, be sure to run the code there.

// On the main thread, not in the Worker. This assumes
// `fileHandle` is the `FileSystemFileHandle` you obtained
// the `FileSystemSyncAccessHandle` from in the Worker
// thread. Be sure to close the file in the Worker thread first.
const fileHandle = await opfsRoot.getFileHandle('fast');
try {
  // Obtain a file handle to a new file in the user-visible file system
  // with the same name as the file in the origin private file system.
  const saveHandle = await showSaveFilePicker({
    suggestedName: fileHandle.name || ''
  });
  const writable = await saveHandle.createWritable();
  await writable.write(await fileHandle.getFile());
  await writable.close();
} catch (err) {
  console.error(err.name, err.message);
}

Debugging the origin private file system #

Until built-in DevTools support is added (see crbug/1284595), use the OPFS Explorer Chrome extension to debug the origin private file system. The screenshot above from the section Creating new files and folders is taken straight from the extension by the way.

After installing the extension, open the Chrome DevTools, select the OPFS Explorer tab, and you're then ready to inspect the file hierarchy. Save files from the origin private file system to the user-visible file system by clicking the file name and delete files and folders by clicking the trash can icon.

Demo #

See the origin private file system in action (if you install the OPFS Explorer extension) in a demo that uses it as a backend for a SQLite database compiled to WebAssembly. Be sure to check out the source code on Glitch. Note how the embedded version below does not use the origin private file system backend (because the iframe is cross-origin), but when you open the demo in a separate tab, it does.

Conclusions #

The origin private file system, as specified by the WHATWG, has shaped the way we use and interact with files on the web. It has enabled new use cases that were impossible to achieve with the user-visible file system. All major browser vendors—Apple, Mozilla, and Google—are on-board and share a joint vision. The development of the origin private file system is very much a collaborative effort, and feedback from developers and users is essential to its progress. As we continue to refine and improve the standard, feedback on the whatwg/fs repository in the form of Issues or Pull Requests is welcome.

Acknowledgements #

This article was reviewed by Austin Sully, Etienne Noël, and Rachel Andrew. Hero image by Christina Rumpf on Unsplash.

PWAs in app stores

2023-03-31T00:00:00Z

PWAs can be accessed through a web browser, but they can also be installed on a user's home screen as outlined in the articles in the section Provide an installable experience. One of the challenges with PWAs, however, is distributing them to app stores. This is where PWABuilder comes in.
PWABuilder is a powerful tool that allows developers to create packages that can be submitted to various app stores:

One of the major advantages of using PWABuilder to create packages is that it simplifies the process of publishing your web application to app stores. Normally, submitting an app to app stores requires a lot of work, including writing code in languages web developers may not necessarily be familiar with, creating app icons, configuring various settings, and testing the app across different devices and operating systems. PWABuilder takes care of many of these tasks automatically, reducing the amount of time and effort required to publish apps.

Prerequisites #

If your web app meets a few baseline PWA requirements, you can use PWABuilder to validate, score, and package your application for stores.

Your PWA needs to be published at a public URL.
It must have a complete web app manifest.
The app has to be served with HTTPS.

Packaging #

You can have an application package for your PWA in a few steps:

From the homepage of PWABuilder, enter a URL to start the packaging process. PWABuilder will take you to the report card page for your application, where you can view scores and action items for your progressive web app.
To proceed to packaging your app, click the Package for Stores button on the top right of the scorecard page.
Browse the options for packaging, and select a platform by clicking Generate Package. You will be prompted for metadata related to your application, which will vary depending on platform.
Lastly, select Download Package to download your package.

Submitting #

For guidance on how to publish a PWA to a specific store, be sure to check out the platform-specific articles:

Example #

I have generated store packages for one of my apps, SVGcode, using PWABuilder.

Apart from these stores, you can of course get the app in the browser as well.

JavaScript import maps are now supported cross-browser

2023-03-28T00:00:00Z

ES modules are a modern way to include and reuse JavaScript code in web applications. They are supported by modern browsers and provide several benefits over older, non-modular approaches to JavaScript development.

A modern way to use ES modules is with the <script type="importmap"> tag. This tag allows you to define a mapping of external module names to their corresponding URLs, which makes it easier to include and use external modules in your code.

To use the <script type="importmap"> approach, you first need to add it to the <head> section of your HTML document. Inside the tag, you can define a JSON object that maps module names to their corresponding URLs. For example:

<script type="importmap">
  {
    "imports": {
      "browser-fs-access": "https://unpkg.com/browser-fs-access@0.33.0/dist/index.modern.js"
    }
  }
</script>

This code defines a single external module named "browser-fs-access" and maps it to the URL of the browser-fs-access library, hosted on the unpkg CDN. With this mapping in place, you can now use the import keyword to include the browser-fs-access library in your code. Note that the import keyword is only available inside a script tag with the type="module" attribute.

<button>Select a text file</button>
<script type="module">
  import {fileOpen} from 'browser-fs-access';

  const button = document.querySelector('button');
  button.addEventListener('click', async () => {
    const file = await fileOpen({
      mimeTypes: ['text/plain'],
    });
    console.log(await file.text());
  });
</script>

Using the <script type="importmap"> tag and the import keyword provides several benefits over older, non-modular approaches to JavaScript development. It allows you to clearly and explicitly specify the external modules your code depends on, which makes it easier to understand and maintain your code. Overall, using ES modules with the <script type="importmap"> tag is a modern and powerful way to include and reuse JavaScript code in web applications. You can feature-detect support as follows:

if (HTMLScriptElement.supports('importmap')) {
  // The importmap feature is supported.
}

Browser support

Chrome 89, Supported 89
Firefox 108, Supported 108
Edge 89, Supported 89
Safari 16.4, Supported 16.4

Acknowledgements #

Hero image by CHUTTERSNAP on Unsplash.

Push notifications are now supported cross-browser

2023-03-28T00:00:00Z

Push notifications were standardized in 2016 with the release of the Push API and the Notification API, which are part of the W3C's Web Applications Working Group. These APIs provide the necessary functionality for web developers to incorporate push notifications into their web applications and for users to receive and interact with notifications on their web browsers. Push messages are notifications that are sent to a user's web browser from a website or application that the user has previously granted permission to send notifications. These messages can be used to alert the user of new content or updates, remind them of upcoming events or deadlines, or provide other important information. Push messages can be particularly useful for applications that need to deliver timely, relevant information to their users, such as news or sports apps, or for e-commerce websites that want to send users notifications about special offers or sales.

To sign up for push notifications, first check if your browser supports them by checking for the serviceWorker and PushManager objects in the navigator and window objects.

If push notifications are supported, use the async and await keywords to register the service worker and subscribe for push notifications. Here is an example of how you can do this using JavaScript:

// Check if the browser supports push notifications.
if ("serviceWorker" in navigator && "PushManager" in window) {
  try {
    // Register the service worker.
    const swReg = await navigator.serviceWorker.register("/sw.js");

    // Subscribe for push notifications.
    const pushSubscription = await swReg.pushManager.subscribe({
      userVisibleOnly: true
    });

    // Save the push subscription to the database.
    savePushSubscription(pushSubscription);
  } catch (error) {
    // Handle errors.
    console.error("Error subscribing for push notifications.", error);
  }
} else {
  // Push notifications are not supported by the browser.
  console.error("Push notifications are not supported by the browser.");
}

Browser support

Chrome 42, Supported 42
Firefox 44, Supported 44
Edge 17, Supported 17
Safari 16, Supported 16

Source

Acknowledgements #

Hero image by Jean Bach on Unsplash.

How to use multiple screens

2022-10-10T00:00:00Z

The modern way #

Using `window.getScreenDetails()` #

To make sure your browser supports the getScreenDetails() method, first check if it exists on the window object. Then, call window.getScreenDetails() to get attached screens. Adding an event listener to adapt to changed screen details allows you to manage multiple screens according to the needs of your app.

if ('getScreenDetails' in window) {
  // The Window Management API is supported.
  const screenDetails = await window.getScreenDetails();
  screenDetails.addEventListener('screenschange', (event) => {
    // Handle screens change.
  });
}

Browser support

Chrome 100, Supported 100
Firefox, Not supported
Edge 100, Supported 100
Safari, Not supported

The classic way #

Using the window.screen method #

There's no classic way to control multi-screen layouts, but you can fall back to controlling the current screen by using the window.screen property and polyfill the new API shape.

if (!('getScreenDetails' in window)) {
  // Returning a one-element array with the current screen,
  // noting that there might be more.
  window.getScreenDetails = async () => [window.screen];
  // Set to `false`, noting that this might be a lie.
  window.screen.isExtended = false;
}

Browser support

Chrome 1, Supported 1
Firefox 1, Supported 1
Edge 12, Supported 12
Safari 1, Supported 1

Source

Progressive enhancement #

The demo below shows how you can handle multiple screens with the Window Management API. The code checks the browser capability first and then falls back to the classic way.

const detectButton = document.querySelector('#detectScreen');
const createButton = document.querySelector('#create');
const permissionLabel = document.querySelector('#permissionStatus');
const screensAvailLabel = document.querySelector('#screensAvail');
const popupUrl = './popup.html';
let screenDetails = undefined;
let permission = undefined;
let currentScreenLength = undefined;

detectButton.addEventListener('click', async () => {
  if ('getScreenDetails' in window) {
    screenDetails = await window.getScreenDetails();
    screenDetails.addEventListener('screenschange', (event) => {
      if (screenDetails.screens.length !== currentScreenLength) {
        currentScreenLength = screenDetails.screens.length;
        updateScreenInfo();
      }
    });
    try {
      permission =
        (await navigator.permissions.query({ name: 'window-management' })).state === 'granted'
          ? 'Granted'
          : 'No Permission';
    } catch (err) {
      console.error(err);
    }
    currentScreenLength = screenDetails.screens.length;
    updateScreenInfo();
  } else {
    screenDetails = window.screen;
    permission = 'Window Management API - NOT SUPPORTED';
    currentScreenLength = 1;
    updateScreenInfo();
  }
});

createButton.addEventListener('click', () => {
  let screen = screenDetails.screens[Math.floor(Math.random() * currentScreenLength)];
  options = {
    x: screen.availLeft,
    y: screen.availTop,
    width: screen.availWidth,
    height: screen.availHeight,
  };
  window.open(popupUrl, '_blank', getFeaturesFromOptions(options));
});

function getFeaturesFromOptions(options) {
  return (
    'left=' +
    options.x +
    ',top=' +
    options.y +
    ',width=' +
    options.width +
    ',height=' +
    options.height
  );
}

updateScreenInfo = () => {
  screensAvailLabel.innerHTML = currentScreenLength;
  permissionLabel.innerHTML = permission;
  if ('getScreenDetails' in window && screenDetails.screens.length > 1) {
    createButton.disabled = false;
  } else {
    createButton.disabled = true;
  }
};

Demo #

How to let the user share the website they are on

2022-10-10T00:00:00Z

Letting the user share the website they are on is a common web apps pattern that you can find on many news sites, blogs, or shopping sites. Since linking is one of the web's super powers, the hope is to acquire traffic from users who see the shared link on social networking sites, or who receive it via chat messages or even plain old school via email.

The modern way #

The Web Share API lets the user share data like the URL of the page they are on, along with a title and descriptive text. The navigator.share() method of the Web Share API invokes the native sharing mechanism of the device. It returns a promise and takes a single argument with the to-be-shared data. Possible values are:

url: A string representing the URL to be shared.
text: A string representing text to be shared.
title: A string representing a title to be shared. May be ignored by the browser.

Browser support

Chrome 89, Supported 89
Firefox 71, Behind a flag
Edge 93, Supported 93
Safari 12.1, Supported 12.1

Source

The classic way #

Not all browsers support the Web Share API yet. A fallback is thus to integrate with your target audience's most popular social networking sites. A popular example is Twitter, whose Web Intent URL allows for a text and a URL to be shared. The method typically consists of crafting a URL and opening it in a browser.

UI considerations #

It is a nice touch to respect the platform's established share icon according to the UI guidelines of the operating system vendors.

Windows:
Apple:
Android and other operating systems:

Progressive enhancement #

The snippet below uses the Web Share API when it is supported, then falls back to Twitter's Web Intent URL.

// DOM references
const button = document.querySelector('button');
const icon = button.querySelector('.icon');
const canonical = document.querySelector('link[rel="canonical"]');

// Find out if the user is on a device made by Apple.
const IS_MAC = /Mac|iPhone/.test(navigator.platform);
// Find out if the user is on a Windows device.
const IS_WINDOWS = /Win/.test(navigator.platform);
// For Apple devices or Windows, use the platform-specific share icon.
icon.classList.add(`share${IS_MAC? 'mac' : (IS_WINDOWS? 'windows' : '')}`);

button.addEventListener('click', async () => {
  // Title and text are identical, since the title may actually be ignored.
  const title = document.title;
  const text = document.title;
  // Use the canonical URL, if it exists, else, the current location.
  const url = canonical?.href || location.href;

  // Feature detection to see if the Web Share API is supported.
  if ('share' in navigator) {
    try {
      await navigator.share({
        url,
        text,
        title,
      });
      return;
    } catch (err) {
      // If the user cancels, an `AbortError` is thrown.
      if (err.name !== "AbortError") {
        console.error(err.name, err.message);
      }
    }
  }
  // Fallback to use Twitter's Web Intent URL.
  // (https://developer.twitter.com/en/docs/twitter-for-websites/tweet-button/guides/web-intent)
  const shareURL = new URL('https://twitter.com/intent/tweet');
  const params = new URLSearchParams();
  params.append('text', text);
  params.append('url', url);
  shareURL.search = params;
  window.open(shareURL, '_blank', 'popup,noreferrer,noopener');
});

Demo #

New patterns for amazing apps

2022-10-10T00:00:00Z

No matter what you build—be it a next generation video editing app, an addictive word game, or a future online social networking app—you will always find yourself in need of a few basic building blocks:

The video editing app will probably allow the user to save the edited video.
Your game will maybe allow the user to share game progress with friends.
An online social networking app will highly likely allow the user to paste images into a post.

No universal way to realize these patterns #

These were just a couple of examples of such patterns, and there are many more. But all of these have one thing in common: there is no universal way to realize them.

For example, not all browsers implement the Web Share API, so in some cases you will have to fall back to a different approach, like Twitter's Web Intents, or copying to the clipboard, which is the approach chosen in Wordle when the Web Share API isn't implemented. Phew, barely got this one:

Wordle 471 6/6

⬛⬛⬛⬛🟨
🟩⬛⬛⬛🟨
🟩🟩🟩⬛⬛
🟩🟩🟩⬛⬛
🟩🟩🟩🟩⬛
🟩🟩🟩🟩🟩

Saving files #

When it comes to saving, the go-to approach is to use the File System Access API, so you end up with a FileSystemFileHandle, which allows you to implement a true save, edit, save flow. The next best thing is to fall back to a classic <a download>, which likewise lets the user save data, but has the downside of creating new files on each download, so they end up with my-video.mp4, my-video (1).mp4, my-video (2).mp4, etc.

Pasting images #

To conclude the introductory examples, not all browsers support pasting images into a web app, so you can fall back to using the Drag and Drop API or showing a file picker, which is not as elegant as the Async Clipboard API, but at least it works.

The new patterns #

With this out of the way, the new pattern sections are:

Clipboard patterns #

Clipboard patterns for everything concerned with the system clipboard like copying and pasting of all sorts of things.

Files patterns #

Files patterns for everything concerned with files and directories; be it saving, opening, dragging and dropping, receiving or sharing.

Web apps patterns #

Web apps patterns for everything concerned with advanced app features like providing app shortcuts, periodically syncing data in the background, showing app badges, and many more.

Feedback #

I hope these patterns will help you build amazing apps, and I'm looking forward to your feedback! You can provide feedback by tweeting at @ChromiumDev or filing an Issue. In both cases, tag @tomayac to make sure I see it.

Acknowledgements #

I'm grateful to Joe Medley for his help with reviewing and editing the patterns. Thanks to Pete LePage, Ewa Gasperowicz, Rachel Andrew, Ken Pascal, and Matthias Rohmer for all their technical and organizational support and encouragements to land this. The entire patterns project would not have been possible without the help of the authors of the individual patterns, namely Harry Theodoulou, Tony Conway, Palances Liao, Cecilia Cong, François Beaufort, and Joe Medley.

Sample pattern

2022-10-10T00:00:00Z

The pattern layout supports the full range of markdown elements/blocks as demonstrated in the example post.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin dictum a massa sit amet ullamcorper. Suspendisse auctor ultrices ante, nec tempus nibh varius at.

Image, Inline #

Inline right, outlined image.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin dictum a massa sit amet ullamcorper. Suspendisse auctor ultrices ante, nec tempus nibh varius at. Cras ligula lacus, porta vitae maximus a, ultrices a mauris. Vestibulum porta dolor erat, vel molestie dolor posuere in. Nam vel elementum augue. Nam quis enim blandit, posuere justo dignissim, scelerisque diam. Fusce aliquet urna ac blandit ullamcorper. Proin et semper nibh, sit amet imperdiet velit. Morbi at quam sem. Integer et erat ac mi scelerisque suscipit et vitae nulla. Aliquam scelerisque efficitur ante ut facilisis. Aenean et risus fringilla, hendrerit sapien et, tincidunt orci. Aenean sed tellus aliquam, consectetur metus in, tempus enim.

Sample snippet #

const button = document.querySelector('button');

button.addEventListener('click', async () => {
  console.log('Testing 123');
});

Demo #

How to copy images

2022-10-10T00:00:00Z

Many modern browsers support copying images to the clipboard in the formats PNG and SVG. Other formats are not yet supported for security reasons.

The modern way #

Using the Async Clipboard API #

The Clipboard.write() method takes an array of ClipboardItem objects and returns a Promise that resolves when the image is successfully written to the clipboard. Clipboard.write() can only be used from the window object that has focus.

Browser support

Chrome 66, Supported 66
Firefox 87, Behind a flag
Edge 79, Supported 79
Safari 13.1, Supported 13.1

Source

The classic way #

Using `navigator.clipboard.writeText()` #

While not all browsers support navigator.clipboard.write() for binary data yet, they all support navigator.clipboard.writeText(). If you want to copy an SVG image, instead of copying the image itself, you can copy the SVG source code. For PNG images, you are unfortunately out of luck.

Browser support

Chrome 66, Supported 66
Firefox 63, Supported 63
Edge 79, Supported 79
Safari 13.1, Supported 13.1

Source

Progressive enhancement #

const button = document.querySelector('button');
const img = document.querySelector('img');

button.addEventListener('click', async () => {
  const responsePromise = fetch(img.src);
  try {
    if ('write' in navigator.clipboard) {
      await navigator.clipboard.write([
        new ClipboardItem({
          'image/svg+xml': new Promise(async (resolve) => {
            const blob = await responsePromise.then(response => response.blob());
            resolve(blob);
          }),
        }),
      ]);
      // Image copied as image.
    } else {
      const text = await responsePromise.then(response => response.text());
      await navigator.clipboard.writeText(text);
      // Image copied as source code.
    }
  } catch (err) {
    console.error(err.name, err.message);
  }
});

Demo #

How to drag and drop directories

2022-10-10T00:00:00Z

The HTML Drag and Drop interfaces enable web applications to accept dragged and dropped files on a web page. During a drag and drop operation, dragged file and directory items are associated with file entries and directory entries respectively. When it comes to dragging and dropping files into the browser, there are two ways of doing it: the modern and the classic way.

The modern way #

Using the File System Access API's `DataTransferItem.getAsFileSystemHandle()` method #

The DataTransferItem.getAsFileSystemHandle() method returns a promise with a FileSystemFileHandle object if the dragged item is a file, and a promise with a FileSystemDirectoryHandle object if the dragged item is a directory. These handles let you read, and optionally write back to the file or directory. Note that the Drag and Drop interface's DataTransferItem.kind will be "file" for both files and directories, whereas the File System Access API's FileSystemHandle.kind will be "file" for files and "directory" for directories.

Browser support

Chrome 86, Supported 86
Firefox, Not supported
Edge 86, Supported 86
Safari, Not supported

Source

The classic way #

Using the non-standard `DataTransferItem.webkitGetAsEntry()` method #

The DataTransferItem.webkitGetAsEntry() method returns the drag data item's FileSystemFileEntry if the item is a file, and FileSystemDirectoryEntry if the item is a directory. While you can read the file or directory, there is no way to write back to them. This method has the disadvantage that is not on the standards track, but has the advantage that it supports directories.

Browser support

Chrome 13, Supported 13
Firefox 50, Supported 50
Edge 14, Supported 14
Safari 11.1, Supported 11.1

Source

Progressive enhancement #

The snippet below uses the modern File System Access API's DataTransferItem.getAsFileSystemHandle() method when it is supported, then falls back to the non-standard DataTransferItem.webkitGetAsEntry() method, and finally falls back to the classic DataTransferItem.getAsFile() method. Be sure to check the type of each handle, since it could be either of:

FileSystemDirectoryHandle when the modern code path is chosen.
FileSystemDirectoryEntry when the non-standard code path is chosen.

All types have a name property, so logging it is fine and will always work.

// Run feature detection.
const supportsFileSystemAccessAPI =
  'getAsFileSystemHandle' in DataTransferItem.prototype;
const supportsWebkitGetAsEntry =
  'webkitGetAsEntry' in DataTransferItem.prototype;

// This is the drag and drop zone.
const elem = document.querySelector('main');

// Prevent navigation.
elem.addEventListener('dragover', (e) => {
  e.preventDefault();
});

// Visually highlight the drop zone.
elem.addEventListener('dragenter', (e) => {
  elem.style.outline = 'solid red 1px';
});

// Visually unhighlight the drop zone.
elem.addEventListener('dragleave', (e) => {
  elem.style.outline = '';
});

// This is where the drop is handled.
elem.addEventListener('drop', async (e) => {
  // Prevent navigation.
  e.preventDefault();
  if (!supportsFileSystemAccessAPI && !supportsWebkitGetAsEntry) {
    // Cannot handle directories.
    return;
  }
  // Unhighlight the drop zone.
  elem.style.outline = '';

  // Prepare an array of promises…
  const fileHandlesPromises = [...e.dataTransfer.items]
    // …by including only files (where file misleadingly means actual file _or_
    // directory)…
    .filter((item) => item.kind === 'file')
    // …and, depending on previous feature detection…
    .map((item) =>
      supportsFileSystemAccessAPI
        // …either get a modern `FileSystemHandle`…
        ? item.getAsFileSystemHandle()
        // …or a classic `FileSystemFileEntry`.
        : item.webkitGetAsEntry(),
    );
  // Loop over the array of promises.
  for await (const handle of fileHandlesPromises) {
    // This is where we can actually exclusively act on the directories.
    if (handle.kind === 'directory' || handle.isDirectory) {
      console.log(`Directory: ${handle.name}`);
    }
  }
});

Demo #

How to drag and drop files

2022-10-10T00:00:00Z

The modern way #

Using the File System Access API's `DataTransferItem.getAsFileSystemHandle()` method #

Browser support

Chrome 86, Supported 86
Firefox, Not supported
Edge 86, Supported 86
Safari, Not supported

Source

The classic way #

Using the classic `DataTransferItem.getAsFile()` method #

The DataTransferItem.getAsFile() method returns the drag data item's File object. If the item is not a file, this method returns null. While you can read the file, there is no way to write back to it. This method has the disadvantage that it does not support directories.

Browser support

Chrome 11, Supported 11
Firefox 50, Supported 50
Edge 12, Supported 12
Safari 5.1, Supported 5.1

Source

Progressive enhancement #

FileSystemFileHandle when the modern code path is chosen.
File when the classic code path is chosen.

All types have a name property, so logging it is fine and will always work.

// Run feature detection.
const supportsFileSystemAccessAPI =
  'getAsFileSystemHandle' in DataTransferItem.prototype;

// This is the drag and drop zone.
const elem = document.querySelector('main');

  // Prevent navigation.
elem.addEventListener('dragover', (e) => {
  e.preventDefault();
});

// Visually highlight the drop zone.
elem.addEventListener('dragenter', (e) => {
  elem.style.outline = 'solid red 1px';
});

// Visually unhighlight the drop zone.
elem.addEventListener('dragleave', (e) => {
  elem.style.outline = '';
});

// This is where the drop is handled.
elem.addEventListener('drop', async (e) => {
  // Prevent navigation.
  e.preventDefault();
  // Unhighlight the drop zone.
  elem.style.outline = '';
  // Prepare an array of promises…
  const fileHandlesPromises = [...e.dataTransfer.items]
    // …by including only files (where file misleadingly means actual file _or_
    // directory)…
    .filter((item) => item.kind === 'file')
    // …and, depending on previous feature detection…
    .map((item) =>
      supportsFileSystemAccessAPI
        // …either get a modern `FileSystemHandle`…
        ? item.getAsFileSystemHandle()
        // …or a classic `File`.
        : item.getAsFile(),
    );
  // Loop over the array of promises.
  for await (const handle of fileHandlesPromises) {
    // This is where we can actually exclusively act on the files.
    if (handle.kind === 'file' || handle.isFile) {
      console.log(`File: ${handle.name}`);
    }
  }
});

Demo #

How to open a directory

2022-10-10T00:00:00Z

Dealing with directories is not something you will cope with on a daily basis, but occasionally the use case arises, such as wanting to process all images in a directory. With the File System Access API, users can now open directories in the browser and decide if they need write access or not.

The modern way #

Using the File System Access API's `showDirectoryPicker()` method #

To open a directory, call showDirectoryPicker(), which returns a promise with the picked directory. If you need write access, you can pass { mode: 'readwrite' } to the method.

Browser support

Chrome 86, Supported 86
Firefox, Not supported
Edge 86, Supported 86
Safari, Not supported

Source

The classic way #

Using the `<input type="file" webkitdirectory>` element #

The <input type="file" webkitdirectory> element on a page allows the user to click it and open a directory. The trick now consists of inserting the element invisibly into a page with JavaScript and click it programmatically.

Browser support

Chrome 7, Supported 7
Firefox 50, Supported 50
Edge 13, Supported 13
Safari 11.1, Supported 11.1

Source

Progressive enhancement #

The method below uses the File System Access API when it's supported and else falls back to the classic approach. In both cases the function returns a directory, but in case of where the File System Access API is supported, each file object also has a FileSystemDirectoryHandle stored in the directoryHandle property and a FileSystemFileHandle stored in the handle property, so you can optionally serialize the handles to disk.

const openDirectory = async (mode = "read") => {
  // Feature detection. The API needs to be supported
  // and the app not run in an iframe.
  const supportsFileSystemAccess =
    "showDirectoryPicker" in window &&
    (() => {
      try {
        return window.self === window.top;
      } catch {
        return false;
      }
    })();
  // If the File System Access API is supported…
  if (supportsFileSystemAccess) {
    let directoryStructure = undefined;

    // Recursive function that walks the directory structure.
    const getFiles = async (dirHandle, path = dirHandle.name) => {
      const dirs = [];
      const files = [];
      for await (const entry of dirHandle.values()) {
        const nestedPath = `${path}/${entry.name}`;
        if (entry.kind === "file") {
          files.push(
            entry.getFile().then((file) => {
              file.directoryHandle = dirHandle;
              file.handle = entry;
              return Object.defineProperty(file, "webkitRelativePath", {
                configurable: true,
                enumerable: true,
                get: () => nestedPath,
              });
            })
          );
        } else if (entry.kind === "directory") {
          dirs.push(getFiles(entry, nestedPath));
        }
      }
      return [
        ...(await Promise.all(dirs)).flat(),
        ...(await Promise.all(files)),
      ];
    };

    try {
      // Open the directory.
      const handle = await showDirectoryPicker({
        mode,
      });
      // Get the directory structure.
      directoryStructure = getFiles(handle, undefined);
    } catch (err) {
      if (err.name !== "AbortError") {
        console.error(err.name, err.message);
      }
    }
    return directoryStructure;
  }
  // Fallback if the File System Access API is not supported.
  return new Promise((resolve) => {
    const input = document.createElement('input');
    input.type = 'file';
    input.webkitdirectory = true;

    input.addEventListener('change', () => {
      let files = Array.from(input.files);
      resolve(files);
    });
    if ('showPicker' in HTMLInputElement.prototype) {
      input.showPicker();
    } else {
      input.click();
    }
  });
};

Demo #

How to open one or multiple files

2022-10-10T00:00:00Z

Dealing with files is one of the most common operations for apps on the web. Traditionally, users needed to upload a file, make some changes to it, and then download it again, resulting in a copy in the Downloads folder. With the File System Access API, users can now open files directly, make modifications, and save back the changes to the original file.

The modern way #

Using the File System Access API's `showOpenFilePicker()` method #

To open a file, call showOpenFilePicker(), which returns a promise with an array of the picked file or files. If you need multiple files, you can pass { multiple: true, } to the method.

Browser support

Chrome 86, Supported 86
Firefox, Not supported
Edge 86, Supported 86
Safari, Not supported

Source

The classic way #

Using the `<input type="file">` element #

The <input type="file"> element on a page allows the user to click it and open one or multiple files. The trick now consists of inserting the element invisibly into a page with JavaScript and clicking it programmatically.

Browser support

Chrome 1, Supported 1
Firefox 1, Supported 1
Edge 12, Supported 12
Safari 1, Supported 1

Source

Progressive enhancement #

The method below uses the File System Access API when it's supported and else falls back to the classic approach. In both cases the function returns an array of files, but in case of where the File System Access API is supported, each file object also has a FileSystemFileHandle stored in the handle property, so you can optionally serialize the handle to disk.

const openFileOrFiles = async (multiple = false) => {
  // Feature detection. The API needs to be supported
  // and the app not run in an iframe.
  const supportsFileSystemAccess =
    "showOpenFilePicker" in window &&
    (() => {
      try {
        return window.self === window.top;
      } catch {
        return false;
      }
    })();
  // If the File System Access API is supported…
  if (supportsFileSystemAccess) {
    let fileOrFiles = undefined;
    try {
      // Show the file picker, optionally allowing multiple files.
      const handles = await showOpenFilePicker({ multiple });
      // Only one file is requested.
      if (!multiple) {
        // Add the `FileSystemFileHandle` as `.handle`.
        fileOrFiles = await handles[0].getFile();
        fileOrFiles.handle = handles[0];
      } else {
        fileOrFiles = await Promise.all(
          handles.map(async (handle) => {
            const file = await handle.getFile();
            // Add the `FileSystemFileHandle` as `.handle`.
            file.handle = handle;
            return file;
          })
        );
      }
    } catch (err) {
      // Fail silently if the user has simply canceled the dialog.
      if (err.name !== 'AbortError') {
        console.error(err.name, err.message);
      }
    }
    return fileOrFiles;
  }
  // Fallback if the File System Access API is not supported.
  return new Promise((resolve) => {
    // Append a new `<input type="file" multiple? />` and hide it.
    const input = document.createElement('input');
    input.style.display = 'none';
    input.type = 'file';
    document.body.append(input);
    if (multiple) {
      input.multiple = true;
    }
    // The `change` event fires when the user interacts with the dialog.
    input.addEventListener('change', () => {
      // Remove the `<input type="file" multiple? />` again from the DOM.
      input.remove();
      // If no files were selected, return.
      if (!input.files) {
        return;
      }
      // Return all files or just one file.
      resolve(multiple ? Array.from(input.files) : input.files[0]);
    });
    // Show the picker.
    if ('showPicker' in HTMLInputElement.prototype) {
      input.showPicker();
    } else {
      input.click();
    }
  });
};

Demo #

How to save a file

2022-10-10T00:00:00Z

The modern way #

Using the File System Access API's `showSaveFilePicker()` method #

To save a file, call showSaveFilePicker(), which returns a promise with FileSystemFileHandle. You can pass the desired file name to the method as { suggestedName: 'example.txt' }.

Browser support

Chrome 86, Supported 86
Firefox, Not supported
Edge 86, Supported 86
Safari, Not supported

Source

The classic way #

Using the `<a download>` element #

The <a download> element on a page allows the user to click it and download a file. The trick now consists of inserting the element invisibly into a page with JavaScript and clicking it programmatically.

Browser support

Chrome 15, Supported 15
Firefox 20, Supported 20
Edge 13, Supported 13
Safari 10.1, Supported 10.1

Source

Progressive enhancement #

The method below uses the File System Access API when it's supported and else falls back to the classic approach. In both cases the function saves the file, but in case of where the File System Access API is supported, the user will get a file save dialog where they can choose where the file should be saved.

const saveFile = async (blob, suggestedName) => {
  // Feature detection. The API needs to be supported
  // and the app not run in an iframe.
  const supportsFileSystemAccess =
    'showSaveFilePicker' in window &&
    (() => {
      try {
        return window.self === window.top;
      } catch {
        return false;
      }
    })();
  // If the File System Access API is supported…
  if (supportsFileSystemAccess) {
    try {
      // Show the file save dialog.
      const handle = await showSaveFilePicker({
        suggestedName,
      });
      // Write the blob to the file.
      const writable = await handle.createWritable();
      await writable.write(blob);
      await writable.close();
      return;
    } catch (err) {
      // Fail silently if the user has simply canceled the dialog.
      if (err.name !== 'AbortError') {
        console.error(err.name, err.message);
        return;
      }
    }
  }
  // Fallback if the File System Access API is not supported…
  // Create the blob URL.
  const blobURL = URL.createObjectURL(blob);
  // Create the `<a download>` element and append it invisibly.
  const a = document.createElement('a');
  a.href = blobURL;
  a.download = suggestedName;
  a.style.display = 'none';
  document.body.append(a);
  // Programmatically click the element.
  a.click();
  // Revoke the blob URL and remove the element.
  setTimeout(() => {
    URL.revokeObjectURL(blobURL);
    a.remove();
  }, 1000);
};

Demo #

How to create an app badge

2022-10-10T00:00:00Z

Badges are used to convey non-urgent information to the user. For example, they are used to indicate the status of an app, or the number of unread items. The classic way of creating an app badge is to add a number to the favicon. On modern browsers, after an app has been installed, there is a built-in way to add a badge to the app icon in the operating system's task bar.

The modern way #

Using the `navigator.setAppBadge()` method #

The navigator.setAppBadge() method sets a badge on the icon associated with the installed app. The method takes an optional single argument, which is an integer which will be used as the value of the badge. Setting the number to 0 clears the app badge. Not providing an argument results in a generic badge, commonly displayed as a colored dot.

Browser support

Chrome 81, Supported 81
Firefox, Not supported
Edge 81, Supported 81
Safari, Not supported

Source

The classic way #

Adding a number to the favicon #

If the app is not installed yet, you can add a number to the favicon. There are many ways of doing so, for example, by drawing the favicon dynamically to a canvas with the badge info added and displaying it as a Blob URL, or to craft an SVG image with the badge info as a data URL.

Progressive enhancement #

The snippet below uses a custom element <favicon-badge></favicon-badge> that lets the developer declaratively set a badge on the favicon specified via the src attribute by passing an integer to the badge attribute. When the user installs the application, the badge is "upgraded" to a native operating system badge.

import 'https://unpkg.com/favicon-badge@2.0.0/dist/FavIconBadge.js';

// DOM references.
const favicon = document.querySelector('favicon-badge');
const installButton = document.querySelector('button');

// Feature detection.
const supportsAppBadge = 'setAppBadge' in navigator;

let setAppBadge;

// For the demo simply set the badge between [0, 9].
let i = 0;
const getAppBadgeValue = () => {
  if (i > 9) {
    i = 0;
  }
  return i++;
};

// Set the badge on the favicon.
const setAppBadgeFavicon = (value) => {
  favicon.badge = value;
};

// Set the native operating system badge.
const setAppBadgeNative = (value) => {
  navigator.setAppBadge(value);
}

// If the app is installed and the Badging API is supported,
// set the badge on the native operating system. Else, fall
// back to the favicon.
if (
  matchMedia('(display-mode: standalone)').matches &&
  supportsAppBadge
) {
  setAppBadge = setAppBadgeNative;
} else {
  setAppBadge = setAppBadgeFavicon;
}

// Set a new badge every second.
setInterval(() => {
  setAppBadge(getAppBadgeValue());
}, 1000);

// If installation is supported…
if ('BeforeInstallPromptEvent' in window) {
  let installEvent = null;
  const onInstall = () => {
    // After installation, "upgrade" to the native operating system badge.
    installButton.disabled = true;
    installEvent = null;
    if (supportsAppBadge) {
      favicon.badge = false;
      setAppBadge = setAppBadgeNative;
    }
  };

  // …listen for the `beforeinstallprompt` event.
  window.addEventListener('beforeinstallprompt', (event) => {
    event.preventDefault();
    installEvent = event;
    installButton.disabled = false;
  });

  // Deal with installation.
  installButton.addEventListener('click', async () => {
    if (!installEvent) {
      return;
    }
    installEvent.prompt();
    const result = await installEvent.userChoice;
    if (result.outcome === 'accepted') {
      onInstall();
    }
  });

  // Listen for the `appinstalled` in case the user installs the app manually.
  window.addEventListener('appinstalled', (event) => {
    onInstall();
  });
}

Demo #

How to access contacts from the address book

2022-10-10T00:00:00Z

Sometimes you want to let users of your app select one of their contacts to message via an email or chat application, or help them discover which of their contacts have already joined a social platform.

The modern way #

Using the Contact Picker API #

To get contacts from the address book, you need to use the Contact Picker API, which allows users to select entries from their contact list and share limited details of the selected entries with your app. Several properties like name, email, tel, address, and icon are available. To find out about the concretely supported properties, call navigator.contacts.getProperties(). To allow the user to select multiple contacts, pass {multiple: true} as the second parameter of navigator.contacts.select().

Browser support

Chrome, Not supported
Firefox, Not supported
Edge, Not supported
Safari, Not supported

Source

The Contact Picker API is available in the Android version of Chrome from version 80.

The classic way #

Using a regular form #

The fallback method is to use a regular form that lets the user enter the contact's details.

Browser support

Chrome 1, Supported 1
Firefox 1, Supported 1
Edge 12, Supported 12
Safari ≤4, Supported ≤4

Source

Progressive enhancement #

If the Contact Picker API is supported, hide the static form fields and show a picker button instead.

const button = document.querySelector('button');
const name = document.querySelector('.name');
const address = document.querySelector('.address');
const email = document.querySelector('.email');
const tel = document.querySelector('.tel');
const pre = document.querySelector('pre');
const autofills = document.querySelectorAll('.autofill');

if ('contacts' in navigator) {
  button.hidden = false;
  for (const autofill of autofills) {
    autofill.parentElement.style.display = 'none';
  }
  address.parentElement.style.display = 'block';
  button.addEventListener('click', async () => {
    const props = ['name', 'email', 'tel', 'address'];
    const opts = { multiple: false };
    try {
      const [contact] = await navigator.contacts.select(props, opts);
      name.value = contact.name;
      address.value = contact.address;
      tel.value = contact.tel;
      email.value = contact.email;
    } catch (err) {
      pre.textContent = `${err.name}: ${err.message}`;
    }
  });
}

Demo #

MishiPay's PWA increases transactions 10 times and saves 2.5 years of queuing

2022-03-28T00:00:00Z

MishiPay empowers shoppers to scan and pay for their shopping with their smartphones, rather than wasting time queuing at the checkout. With MishiPay's Scan & Go technology, shoppers can use their own phone to scan the barcode on items and pay for them, then simply leave the store. Studies reveal that in-store queuing costs the global retail sector about $200 billion annually.

Our technology relies on device hardware capabilities such as GPS sensors and cameras that allow users to locate MishiPay-enabled stores, scan item barcodes within the physical store, and then pay using the digital payment method of their choice. The initial versions of our Scan & Go technology were platform-specific iOS and Android applications, and early adopters loved the technology. Read on to learn how switching to a PWA increased transactions by 10 times and saved 2.5 years of queuing!

10_×

Increased transactions

2.5 years

Queuing saved

Challenge #

Users find our technology extremely helpful when waiting in a queue or check-out line, as it allows them to skip the queue and have a smooth in-store experience. But the hassle of downloading an Android or iOS application made users not choose our technology despite the value. It was a growing challenge for MishiPay, and we needed to increase user adoption with a lower barrier of entry.

Solution #

Our efforts at building and launching the PWA helped us remove the installation hassle and encouraged new users to try our technology inside a physical store, skip the queue, and have a seamless shopping experience. Since the launch, we have seen a massive spike in user adoption with our PWA compared to our platform-specific applications.

Side-by-side comparison of directly launching the PWA (left, faster) vs. installing and launching the Android app (right, slower).

The majority of all transactions happen on the web.

Technical deep-dive #

Locating MishiPay enabled stores #

To enable this feature, we rely on the getCurrentPosition() API along with an IP-based fallback solution.

const geoOptions = {
  timeout: 10 * 1000,
  enableHighAccuracy: true,
  maximumAge: 0,
};

window.navigator.geolocation.getCurrentPosition(
  (position) => {
    const cords = position.coords;
    console.log(`Latitude :  ${cords.latitude}`);
    console.log(`Longitude :  ${cords.longitude}`);
  },
  (error) => {
    console.debug(`Error: ${error.code}:${error.message}`);
    /**
     * Invoke the IP based location services
     * to fetch the latitude and longitude of the user.
     */
  },
  geoOptions,
);

This approach worked well in the earlier versions of the app, but was later proven to be a huge pain point for MishiPay's users for the following reasons:

Location inaccuracies in the IP-based fallback solutions.
A growing listing of MishiPay-enabled stores per region requires users to scroll a list and identify the correct store.
Users accidentally occasionally choose the wrong store, causing the purchases to be recorded incorrectly.

To address these issues, we embedded unique geolocated QR codes on the in-store displays for each store. It paved the way for a faster onboarding experience. Users simply scan the geolocated QR codes printed on marketing material present in the stores to access the Scan & Go web application. This way, they can avoid typing in the web address mishipay.shop to access the service.

In-store scanning experience using the PWA.

Scanning products #

A core feature in the MishiPay app is the barcode scanning as this empowers our users to scan their own purchases and see the running total even before they would otherwise have reached a cash register.

To build a scanning experience on the web, we have identified three core layers.

Video stream #

With the help of the getUserMedia() method, we can access the user's rear view camera with the constraints listed below. Invoking the method automatically triggers a prompt for users to accept or deny access to their camera. Once we have access to the video stream, we can relay it to a video element as shown below:

/**
 * Video Stream Layer
 * https://developer.mozilla.org/docs/Web/API/MediaDevices/getUserMedia
 */
const canvasEle = document.getElementById('canvas');
const videoEle = document.getElementById('videoElement');
const canvasCtx = canvasEle.getContext('2d');
fetchVideoStream();
function fetchVideoStream() {
  let constraints = { video: { facingMode: 'environment' } };
  if (navigator.mediaDevices !== undefined) {
    navigator.mediaDevices
      .getUserMedia(constraints)
      .then((stream) => {
        videoEle.srcObject = stream;
        videoStream = stream;
        videoEle.play();
        // Initiate frame capture - Processing Layer.
      })
      .catch((error) => {
        console.debug(error);
        console.warn(`Failed to access the stream:${error.name}`);
      });
  } else {
    console.warn(`getUserMedia API not supported!!`);
  }
}

Processing layer #

For detecting a barcode in a given video stream, we need to periodically capture frames and transfer them to the decoder layer. To capture a frame, we simply draw the streams from VideoElement onto an HTMLCanvasElement using the drawImage() method of the Canvas API.

/**
 * Processing Layer - Frame Capture
 * https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Manipulating_video_using_canvas
 */
async function captureFrames() {
  if (videoEle.readyState === videoEle.HAVE_ENOUGH_DATA) {
    const canvasHeight = (canvasEle.height = videoEle.videoHeight);
    const canvasWidth = (canvasEle.width = videoEle.videoWidth);
    canvasCtx.drawImage(videoEle, 0, 0, canvasWidth, canvasHeight);
    // Transfer the `canvasEle` to the decoder for barcode detection.
    const result = await decodeBarcode(canvasEle);
  } else {
    console.log('Video feed not available yet');
  }
}

For advanced use cases, this layer also performs some pre-processing tasks such as cropping, rotating, or converting to grayscale. These tasks can be CPU-intensive and result in the application being unresponsive given that barcode scanning is a long-running operation. With the help of the OffscreenCanvas API, we can offload the CPU-intensive task to a web worker. On devices that support hardware graphics acceleration, WebGL API and its WebGL2RenderingContext can optimize gains on the CPU-intensive pre-processing tasks.

Decoder layer #

The final layer is the decoder layer which is responsible for decoding barcodes from the frames captured by the processing layer. Thanks to the Shape Detection API (which is not yet available on all browsers) the browser itself decodes the barcode from an ImageBitmapSource, which can be an img element, an SVG image element, a video element, a canvas element, a Blob object, an ImageData object, or an ImageBitmap object.

/**
 * Barcode Decoder with Shape Detection API
 * https://web.dev/shape-detection/
 */
async function decodeBarcode(canvas) {
  const formats = [
    'aztec',
    'code_128',
    'code_39',
    'code_93',
    'codabar',
    'data_matrix',
    'ean_13',
    'ean_8',
    'itf',
    'pdf417',
    'qr_code',
    'upc_a',
    'upc_e',
  ];
  const barcodeDetector = new window.BarcodeDetector({
    formats,
  });
  try {
    const barcodes = await barcodeDetector.detect(canvas);
    console.log(barcodes);
    return barcodes.length > 0 ? barcodes[0]['rawValue'] : undefined;
  } catch (e) {
    throw e;
  }
}

For devices that don't support the Shape Detection API yet, we need a fallback solution to decode the barcodes. The Shape Detection API exposes a getSupportedFormats() method which helps switch between the Shape Detection API and the fallback solution.

// Feature detection.
if (!('BarceodeDetector' in window)) {
  return;
}
// Check supported barcode formats.
BarcodeDetector.getSupportedFormats()
.then((supportedFormats) => {
  supportedFormats.forEach((format) => console.log(format));
});

Fallback solution #

Several open-source and enterprise scanning libraries are available that can be easily integrated with any web application to implement scanning. Here are some of the libraries that MishiPay recommend.

Comparison of open-source and commercial barcode scanning libraries
Library Name	Type	Wasm Solution	Barcode Formats
QuaggaJs	Open Source	No	1D
ZxingJs	Open Source	No	1D & 2D (Limited)
CodeCorp	Enterprise	Yes	1D & 2D
Scandit	Enterprise	Yes	1D & 2D

All the above libraries are full-fledged SDKs that compose all the layers discussed above. They also expose interfaces to support various scanning operations. Depending on the barcode formats and detection speed needed for the business case, a decision can be between Wasm and non-Wasm solutions. Despite the overhead of requiring an additional resource (Wasm) to decode the barcode, Wasm solutions outperform the non-Wasm solution in terms of accuracy.

Scandit was our primary choice. It supports all barcode formats required for our business use cases; it beats all the available open-source libraries in scanning speed.

Future of scanning #

Once the Shape Detection API is fully supported by all major browsers, we could potentially have a new HTML element <scanner> that has the capabilities required for a barcode scanner. Engineering at MishiPay believes there is a solid use case for the barcode scanning functionality to be a new HTML element due to the growing number of open source and licensed libraries that are enabling experiences such as Scan & Go and many others.

Conclusion #

App fatigue is an issue that developers face when their products enter the market. Users often want to understand the value that an application gives them before they download it. In a store, where MishiPay saves shoppers' time and improves their experience, it is counterintuitive to wait for a download before they can use an application. This is where our PWA helps. By eliminating the barrier to entry, we have increased our transactions by 10 times and enabled our users to save 2.5 years of waiting in the queue.

Acknowledgements #

This article was reviewed by Joe Medley.

PWAs on Oculus Quest 2

2022-01-10T00:00:00Z

The Oculus Quest 2 #

The Oculus Quest 2 is a virtual reality (VR) headset created by Oculus, a division of Meta. It is the successor to the company's previous headset, the Oculus Quest. The device is capable of running as both a standalone headset with an internal, Android-based operating system, and with Oculus-compatible VR software running on a desktop computer when connected over USB or Wi-Fi. It uses the Qualcomm Snapdragon XR2 system on a chip with 6 GB of RAM. The Quest 2's display is a singular fast-switch LCD panel with 1,832 × 1,920 pixels per eye resolution that runs at a refresh rate of up to 120 Hz.

The Oculus Browser #

Currently there are three browsers available for the Oculus Quest 2: Wolvic, a successor to Firefox Reality, and the built-in Oculus Browser. This article focuses on the latter. The Oculus website introduces the Oculus Browser as follows.

"Oculus Browser provides support for the latest web standards and other technologies to help you create VR experiences on the web. Today's 2D web sites work great in Oculus Browser because it's powered by the Chromium rendering engine. It's further optimized for Oculus headsets to get the best performance and to enable web developers take advantage of the full potential of VR with new APIs, like WebXR. Through WebXR, we're opening the doors to the next frontier of the web."

User agent #

The browser's user agent string at the time of writing is as follows.

Mozilla/5.0 (X11; Linux x86_64; Quest 2)
AppleWebKit/537.36 (KHTML, like Gecko)
OculusBrowser/18.1.0.2.46.337441587
SamsungBrowser/4.0
Chrome/95.0.4638.74
VR
Safari/537.36

As you can see, the current version 18.1.0.2.46.337441587 of the Oculus Browser is based on Chrome 95.0.4638.74, that is only one version behind the current stable version of Chrome, which is 96.0.4664.110. If the user switches to mobile mode, VR changes to Mobile VR.

User interface #

The browser's user interface (shown above) has the following features (top row from left to right):

Back button
Reload button
Site information
URL bar
Create bookmark button
Resize button with narrow, medium, and wide options as well as a zoom feature
Request mobile website button
Menu button with the following options:
- Enter private mode
- Close all tabs
- Settings
- Bookmarks
- Downloads
- History
- Clear browsing data

The bottom row includes the following features:

Close button
Minimize button
Three dots button with back, forward, and reload options

Refresh rate and device pixel ratio #

For Oculus Quest 2, Oculus Browser renders both 2D web page content and WebXR at a 90 Hz refresh rate. When watching fullscreen media, Oculus Browser optimizes the device refresh rate based on the frame rate of the video, for example, 24 fps. The Oculus Quest 2 has a device pixel ratio of 1.5 for crisp text.

PWAs in Oculus Browser and the Oculus Store #

On October 28, 2021, Jacob Rossi, Product Management Lead at Meta (Oculus), shared that PWAs were coming to Oculus Quest and Oculus Quest 2. In the following, I describe the PWA experience on Oculus and explain how to build, sideload, and test your PWA on the Oculus Quest 2.

Login state is shared between Oculus Browser and PWAs, allowing users to seamlessly switch between the two. Naturally, Facebook Login is supported out of the box. The Oculus Browser includes a password manager that allows users to store and share their passwords securely between the browser and installed app experiences.

PWA window sizes #

Browser windows and windows of installed PWAs can be freely resized by the user. The height can vary between 625 px and 1,200 px. The width can be set between 400 px and 2,000 px. The default dimensions are 1,000 × 625 px.

Interacting with PWAs #

PWAs can be controlled with the Oculus left and right controllers, Bluetooth mice and keyboards, and via hand tracking. Scrolling works via the thumb sticks on the Oculus controllers, or by pinching the thumb and the index finger and moving in the desired direction. To select something, the user can point and pinch.

Permissions for PWAs #

Permissions in Oculus Browser work pretty much the same way as in Chrome. The state is shared between apps running in the browser and installed PWAs, so users can switch between the two experiences without having to grant the same permissions again.

Albeit many permissions are implemented, not all features are supported. For example, while requesting the geolocation permission succeeds, the device never actually gets a location. Similarly, the various hardware APIs like WebHID, Web Bluetooth, etc. all pass feature detection, but don't actually show a picker that would let the user pair the Oculus with a hardware device. I suppose feature detectability of APIs will be refined once the browser matures.

Debugging PWAs via Chrome DevTools #

After enabling Developer Mode, debugging PWAs on Oculus Quest 2 works exactly as described in Remote debug Android devices.

On the Oculus device, browse to the desired site in Oculus Browser.
Launch Google Chrome on your computer and navigate to chrome://inspect/#devices.
Find the Oculus device in question, which will be followed by a set of Oculus Browser tabs currently open on the device.
Click inspect on the desired Oculus Browser tab.

App discovery #

People can use the browser itself or the Oculus Store to discover PWAs. Just like with any other browser, installed PWAs also work in Oculus Browser as websites running in a tab. When a user visits a site, the Oculus Browser will help them discover the app if (and only if) it is available in the Oculus Store. For users that already have the app installed, Oculus Browser will help them easily switch to the app if they desire.

Exemplary PWAs on the Oculus Quest 2 #

PWAs by Meta #

Multiple Meta divisions have created PWAs for the Oculus Quest 2, for example Instagram and Facebook. These PWAs run in standalone app windows that don't have a URL bar and that can be freely resized.

PWAs by other developers #

At the time of this writing, there is a small but growing number of PWAs for the Oculus Quest 2 on the Oculus Store. Spike lets users experience all the essential work tools like email, chat, calls, notes, tasks, and to-dos from their inbox in a virtual environment hub right in the Spike app.

Another example is Smartsheet, a dynamic workspace that provides project management, automated workflows, and rapid building of new solutions.

More PWAs like Slack, Dropbox, or Canva are coming, as teased in a video featuring Jacob Rossi that was released in the context of Facebook's Connect conference in 2021.

Creating PWAs for Oculus #

Meta outlined the required steps in their documentation. In general, PWAs that are installable in Chrome should oftentimes work out of the box on Oculus.

Web App Manifest requirements #

There are some important differences compared to Chrome's installability criteria and the Web App Manifest spec. For example, Oculus only supports left-to-right languages at the moment, whereas the Web App Manifest spec enforces no such constraints. Another example is start_url, which Chrome strictly requires for an app to be installable, but which on Oculus is optional. Oculus offers a command line tool that lets developers create PWAs for the Oculus Quest 2, which allows them to pass the missing (or override the existing) parameters in the Web App Manifest.

Name	Description
`name`	(Required) The name of the PWA. Currently Oculus only supports left-to-right languages for the name.
`display`	(Required) Either `"standalone"` or `"minimal-ui"`. Oculus currently doesn't support any other values.
`short_name`	(Required) A shorter version of the app name, if needed.
`scope`	(Optional) The URL or paths that should be considered as part of the app.
`start_url`	(Optional) The URL to show at app launch.

Oculus has a number of optional proprietary Web App Manifest fields that can be used to customize the PWA experience.

Name	Description
`ovr_package_name`	(Optional) Sets the package name of the APK generated for the PWA. This must be in reverse domain name notation, e.g., `"com.company.app.pwa"`. If not set, developers must provide a package name to the command line tool with the (then required) parameter `--package-name`.
`ovr_multi_tab_enabled`	(Optional) If `true`, this boolean field will give the PWA a tab bar similar to Oculus Browser. In multi-tab PWAs, internal links that target a new tab (`target="_new"` or `target="_blank"`) will open in new tabs within the PWA window. This differs from single-tab PWAs where such links would open to a Oculus Browser window. This feature is currently being standardized as Tabbed Application Mode.
`ovr_scope_extensions`	(Optional) Allows a PWA to include more web pages within the scope of the web application. It consists of a JSON dictionary containing extension URLs or wildcard patterns. This feature is currently being standardized as Scope Extensions for Web Apps.

Packaging PWAs with Bubblewrap CLI #

Bubblewrap is an open source set of libraries and a command line tool (CLI) for Node.js. Bubblewrap is developed by the Google Chrome team, to help developers generate, build, and sign an Android project that launches your PWA as a Trusted Web Activity (TWA).

Meta Quest Browser currently doesn't fully support TWA, but starting from version 1.18.0, Bubblewrap supports packaging PWAs for Meta Quest devices.

It can generate universal APK files that open a TWA on regular Android devices and the Meta Quest Browser on Meta Quest devices.

Assuming you have Node.js installed, Bubblewrap CLI can be installed with the following command:

npm i -g @bubblewrap/cli

When running Bubblewrap for the first time, it will offer to automatically download and install the required external dependencies—Java Development Kit (JDK) and Android SDK Build Tools.

To generate a Meta Quest compatible Android project that wraps your PWA, run the init command with the --metaquest flag and follow the wizard:

bubblewrap init --manifest="https://your.web.app/manifest.json" --metaquest

Once the project has been generated, build and sign it by running:

bubblewrap build

This will output a file called app-release-signed.apk. This file can be installed on the device or published to the Meta Quest Store, Google Play Store or any of the other Android app distribution platforms.

Packaging PWAs with Oculus Platform Utility #

Oculus Platform Utility is the official command line tool developed by Meta for publishing apps for Oculus Rift and Meta Quest devices.

It also allows to package PWAs for Meta Quest devices with the create-pwa command and publish them to the Meta Quest Store and App Lab.

Set the output file name via the -o parameter and the path to Android SDK via the --android-sdk parameter.

Point the tool at the live URL of the web app manifest via the --web-manifest-url parameter.

If you don't have a manifest on your live PWA or wish to override the live manifest, you can still generate an APK for your PWA using a local manifest file and the --manifest-content-file parameter.

To leave the manifest as pure as possible, use the --package-name parameter with a value in reverse domain name notation (for example, com.company.app.pwa), rather than adding the proprietary ovr_package_name field to the manifest.

ovr-platform-util create-pwa -o output.apk --android-sdk ~/bin/android-10 --manifest-content-file manifest.json --package-name com.company.app.pwa

Packaging PWAs with PWABuilder #

Using PWABuilder is in the authors' view the easiest and therefore recommended way to package PWAs for Meta Quest at the moment.

PWABuilder is an open source project developed by Microsoft, that allows developers to package and sign their PWAs for publishing to various stores, including Microsoft Store, Google Play Store, App Store, and Meta Quest Store.

Packaging PWAs with PWABuilder is as easy as entering the URL of a PWA, entering/editing the metadata for the app, and clicking the Generate button.

PWABuilder gives developers the choice of what tool under the hood to use for packaging PWAs for Meta Quest devices.

You can choose the Meta Quest option to use the Oculus Platform Utility.

You can choose the Android option to use the Bubblewrap and select the Meta Quest compatible checkbox.

Installing PWAs with ADB #

After creating the APK file, you can sideload it to the Meta Quest device using the ADB via USB or Wi-Fi:

adb install app-release-signed.apk

If you use the Bubblewrap CLI for packaging PWAs, it provides a convenient alias command to sideload the APK file:

bubblewrap install

Sideloaded apps appear in an Unknown Sources section in the app drawer.

App submission #

Uploading and submitting PWAs to the Oculus Store is covered in detail in the Oculus Developer Center docs.

Apart from submitting apps to the Oculus Store, developers can also distribute their apps via platforms like SideQuest directly to consumers safely and securely, without requiring store approval. This allows them to get an app directly to end users, even if it is early in development, experimental, or aimed at a unique audience.

Testing multi-tab apps #

To test multi-tab apps, I created a little test PWA that demonstrates the various link features: namely opening a new in-PWA tab, staying on the current tab, opening a new browser window, and opening in a WebView staying on the current tab. Create a locally installable copy of this app by running the commands below on your machine.

ovr-platform-util create-pwa -o test.apk --android-skd ~/bin/android-10 --web-manifest-url https://tomayac.github.io/oculus-pwa-test/manifest.json --package-name com.example.pwa
adb install test.apk

Here's a screencast of the test app.

An Oculus version of SVGcode #

To take the instructions for a spin, I created an Oculus version of my most recent PWA, SVGcode. You can download the resulting APK file output.apk from my Google Drive. If you want to investigate the package further, I have a decompiled version, too. Find the build instructions in package.json.

Using the app on Oculus works fine, including the ability to open and save files. The Oculus Browser doesn't support the File System Access API, but the fallback approach helps. The only thing that didn't function is pinch-zooming. My expectation was for it to work by pressing the trigger button on both controllers and then moving the controllers in opposed directions. Other than that, everything else was performant and responsive, as you can see in the embedded screencast.

Immersive 3D WebXR PWAs #

PWA support on Oculus Quest is not limited to flat 2D apps. Developers can build immersive 3D experiences for VR using the WebXR API.

Wondering how various prompts (PWA install, permission requests, notifications) are handled from within VR, if at all?

Here's a screencast of User Agent Prompts test from the Immersive Web Working Group's WebXR Tests.

As you can see, entering VR mode requires the user's permission. Permissions are asked once per origin. Requesting permissions leaves the immersive mode. Notifications are currently not supported.

Hand tracking #

You can use your hands to interact with PWAs in immersive mode thanks to the WebXR Hand Input API and Meta's AI-based hand-tracking system.

Here's a screencast of Hand Tracking Sample from the Immersive Web Working Group's WebXR Samples.

Augmented/Mixed Reality (Passthrough) #

As announced at Meta Connect 2022, Meta Quest Browser has added support for WebXR Augmented Reality (AR), also known as Mixed Reality (MR), on Meta Quest 2 and Meta Quest Pro devices.

Let's check a slightly modified A-Frame starter example with scaled-down models and hidden sky and plane for augmented reality.

A-Frame is an open source web framework for building 3D/VR/AR experiences entirely out of declarative, reusable custom HTML elements that are easy to read, understand, and copy-and-paste.

Here's a screencast of this demo on Meta Quest 2.

Meta Quest 2 has monochrome cameras, so the passthrough is in grayscale, while Meta Quest Pro has color cameras.

Conclusions #

PWAs on Oculus Quest 2 are a lot of fun and very promising. The endless virtual canvas that lets users scale their screen to whatever fits the current task best has a lot of potential to change the way we work in the future. While typing in VR with hand tracking is still in its infancy and, at least for me, doesn't work very reliably yet, it works well enough for entering URLs or typing short texts.

What I like the most about PWAs on the Oculus Quest 2 is that they are just regular PWAs that can be used unchanged in a browser tab or through a thin APK wrapper without any platform-specific APIs. Targeting multiple platforms with the same code has never been easier. Here's to PWAs in VR and AR on the web. The future is bright!

Acknowledgements #

Oculus Quest 2 photo by Maximilian Prandstätter on Flickr. Oculus Store images of Instagram, Facebook, Oculus Browser, and Spike apps as well as app discoverability illustration and hand tracking animation courtesy of Meta. Hero image by Arnau Marín i Puig. This post was reviewed by Joe Medley.

Designcember Calculator

2021-12-26T00:00:00Z

The challenge #

I'm a kid of the 1980s. A thing that was all the rage back when I was in high school were solar calculators. We were all given a TI-30X SOLAR by the school, and I have fond memories of us benchmarking our calculators against each other by calculating the factorial of 69, the highest number the TI-30X could handle. (The speed variance was very measurable, I have still no idea why.)

Now, almost 28 years later, I thought it would be a fun Designcember challenge to recreate the calculator in HTML, CSS, and JavaScript. Being not much of a designer, I did not start from scratch, but with a CodePen by Sassja Ceballos.

Make it installable #

While not a bad start, I decided to pump it up for full skeuomorphic awesomeness. The first step was to make it a PWA so it could be installed. I maintain a baseline PWA template on Glitch that I remix whenever I need a quick demo. Its service worker will not win you any coding award and it is definitely not production-ready, but it is sufficient to trigger Chromium's mini infobar so the app can be installed.

self.addEventListener('install', (event) => {
  self.skipWaiting();
});

self.addEventListener('activate', (event) => {
  self.clients.claim();
  event.waitUntil(
    (async () => {
      if ('navigationPreload' in self.registration) {
        await self.registration.navigationPreload.enable();
      }
    })(),
  );
});

self.addEventListener('fetch', (event) => {
  event.respondWith(
    (async () => {
      try {
        const response = await event.preloadResponse;
        if (response) {
          return response;
        }
        return fetch(event.request);
      } catch {
        return new Response('Offline');
      }
    })(),
  );
});

Blending with mobile #

Now that the app is installable, the next step is to make it blend in with the operating system apps as much as possible. On mobile, I can do this by setting the display mode to fullscreen in the Web App Manifest.

{
  "display": "fullscreen"
}

On devices with a camera hole or notch, tweaking the viewport so that the content covers the whole screen makes the app look gorgeous.

<meta name="viewport" content="initial-scale=1, viewport-fit=cover" />

Blending with desktop #

On desktop, there is a cool feature I can use: Window Controls Overlay, which allows me to put content in the title bar of the app window. The first step is to override the display mode fallback sequence so it tries to use window-controls-overlay first when it is available.

{
  "display_override": ["window-controls-overlay"]
}

This makes the title bar effectively go away and the content moves up into the title bar area as if the title bar were not there. My idea is to move the skeuomorphic solar cell up into the title bar and the rest of the calculator UI down accordingly, which I can do with some CSS that uses the titlebar-area-* environment variables. You will notice that all the selectors carry a wco class, which will be relevant a couple of paragraphs down.

#calc_solar_cell.wco {
  position: fixed;
  left: calc(0.25rem + env(titlebar-area-x, 0));
  top: calc(0.75rem + env(titlebar-area-y, 0));
  width: calc(env(titlebar-area-width, 100%) - 0.5rem);
  height: calc(env(titlebar-area-height, 33px) - 0.5rem);
}

#calc_display_surface.wco {
  margin-top: calc(env(titlebar-area-height, 33px) - 0.5rem);
}

Next, I need to decide which elements to make draggable, since the title bar that I would usually use for dragging is not available. In the style of a classic widget, I can even make the entire calculator draggable by applying (-webkit-)app-region: drag, apart from the buttons, which get (-webkit-)app-region: no-drag so they cannot be used to drag.

#calc_inside.wco,
#calc_solar_cell.wco {
  -webkit-app-region: drag;
  app-region: drag;
}

button {
  -webkit-app-region: no-drag;
  app-region: no-drag;
}

The final step is to make the app reactive to window controls overlay changes. In a true progressive enhancement approach, I only load the code for this feature when the browser supports it.

if ('windowControlsOverlay' in navigator) {
  import('/wco.js');
}

Whenever the window controls overlay geometry changes, I modify the app to make it look as natural as possible. It is a good idea to debounce this event, since it can be triggered frequently when the user resizes the window. Namely, I apply the wco class to some elements, so my CSS from above kicks in, and I also change the theme color. I can detect if the window controls overlay is visible by checking the navigator.windowControlsOverlay.visible property.

const meta = document.querySelector('meta[name="theme-color"]');
const nodes = document.querySelectorAll(
  '#calc_display_surface, #calc_solar_cell, #calc_outside, #calc_inside',
);

const toggleWCO = () => {
  if (!navigator.windowControlsOverlay.visible) {
    meta.content = '';
  } else {
    meta.content = '#385975';
  }
  nodes.forEach((node) => {
    node.classList.toggle('wco', navigator.windowControlsOverlay.visible);
  });
};

const debounce = (func, wait) => {
  let timeout;
  return function executedFunction(...args) {
    const later = () => {
      clearTimeout(timeout);
      func(...args);
    };
    clearTimeout(timeout);
    timeout = setTimeout(later, wait);
  };
};

navigator.windowControlsOverlay.ongeometrychange = debounce((e) => {
  toggleWCO();
}, 250);

toggleWCO();

Now with all this in place, I get a calculator widget that feels almost like the classic Winamp with one of the oldschool Winamp themes. I can now freely place the calculator on my desktop and activate the window controls feature by clicking the chevron in the upper right corner.

An actually working solar cell #

For the ultimate geekery, I of course needed to make the solar cell actually work. The calculator should only be functioning if there is enough light. The way I modeled this is through setting the CSS opacity of the digits on the display via a CSS variable --opacity that I control via JavaScript.

:root {
  --opacity: 0.75;
}

#calc_expression,
#calc_result {
  opacity: var(--opacity);
}

To detect if enough light is available for the calculator to work, I use the AmbientLightSensor API. For this API to be available, I needed to set the #enable-generic-sensor-extra-classes flag in about:flags and request the 'ambient-light-sensor' permission. As before, I use progressive enhancement to only load the relevant code when the API is supported.

if ('AmbientLightSensor' in window) {
  import('/als.js');
}

The sensor returns the ambient light in lux units whenever a new reading is available. Based on a table of values of typical light situations, I came up with a very simple formula to convert the lux value to a value between 0 and 1 that I programmatically assign to the --opacity variable.

const luxToOpacity = (lux) => {
  if (lux > 250) {
    return 1;
  }
  return lux / 250;
};

const sensor = new window.AmbientLightSensor();
sensor.onreading = () => {
  console.log('Current light level:', sensor.illuminance);
  document.documentElement.style.setProperty(
    '--opacity',
    luxToOpacity(sensor.illuminance),
  );
};
sensor.onerror = (event) => {
  console.log(event.error.name, event.error.message);
};

(async () => {
  const {state} = await navigator.permissions.query({
    name: 'ambient-light-sensor',
  });
  if (state === 'granted') {
    sensor.start();
  }
})();

In the video below you can see how the calculator starts working once I turn the room light up enough. And there you have it: a skeuomorphic solar calculator that actually works. My good old time-tested TI-30X SOLAR has come a long way indeed.

Demo #

Be sure to play with the Designcember Calculator demo and check out the source code on Glitch. (To install the app, you need to open it in its own window. The embedded version below will not trigger the mini infobar.)

Happy Designcember!

SVGcode: a PWA to convert raster images to SVG vector graphics

2021-11-19T00:00:00Z

(If you prefer watching over reading, this article is also available as a video.)

From raster to vector #

Have you ever scaled an image and the result was pixelated and unsatisfactory? If so, you have probably dealt with a raster image format such as WebP, PNG, or JPG.

Scaling up a raster image makes it look pixelated.

In contrast, vector graphics are images that are defined by points in a coordinate system. These points are connected by lines and curves to form polygons and other shapes. Vector graphics have an advantage over raster graphics in that they may be scaled up or down to any resolution without pixelation.

Scaling up a vector image with no loss of quality.

Introducing SVGcode #

I have built a PWA called SVGcode that can help you convert raster images to vectors. Credit where credit is due: I didn't invent this. With SVGcode, I just stand on the shoulders of a command line tool called Potrace by Peter Selinger that I have converted to Web Assembly, so it can be used in a Web app.

The SVGcode app.

Using SVGcode #

First, I want to show you how to use the app. I start with the teaser image for Chrome Dev Summit that I downloaded from the ChromiumDev Twitter channel. This is a PNG raster image that I then drag onto the SVGcode app. When I drop the file, the app traces the image color by color, until a vectorized version of the input appears. I can now zoom into the image, and as you can see, the edges stay sharp. But zooming in on the Chrome logo, you can see that the tracing wasn't perfect, and especially the outlines of the logo look a bit speckled. I can improve the result by de-speckling the tracing by suppressing speckles of up to, say, five pixels.

Converting a dropped image to SVG.

Posterization in SVGcode #

An important step for vectorization, especially for photographic images, is posterizing the input image to reduce the number of colors. SVGcode allows me to do this per color channel, and see the resulting SVG as I make changes. When I'm happy with the result, I can save the SVG to my hard disk and use it wherever I like.

Posterizing an image to reduce the number of colors.

APIs used in SVGcode #

Now that you have seen what the app is capable of, let me show you some of the APIs that help make the magic happen.

Progressive Web App #

SVGcode is an installable Progressive Web App and therefore fully offline enabled. The app is based on the Vanilla JS template for Vite.js and uses the popular Vite plugin PWA, which creates a service worker that uses Workbox.js under the hood. Workbox is a set of libraries that can power a production-ready service worker for Progressive Web Apps, This pattern may not necessarily work for all apps, but for SVGcode's use case it's great.

Window Controls Overlay #

To maximize the available screen real estate, SVGcode uses Window Controls Overlay customization by moving its main menu up into the titlebar area. You can see this get activated at the end of the install flow.

Installing SVGcode and activating the Window Controls Overlay customization.

File System Access API #

To open input image files and save the resulting SVGs, I use the File System Access API. This allows me to keep a reference to previously opened files and to continue where I left off, even after an app reload. Whenever an image gets saved, it is optimized via the svgo library, which may take a moment, depending on the complexity of the SVG. Showing the file save dialog requires a user gesture. It is therefore important to obtain the file handle before the SVG optimization happens, so the user gesture is not invalidated by the time the optimized SVG is ready.

try {
  let svg = svgOutput.innerHTML;
  let handle = null;
  // To not consume the user gesture obtain the handle before preparing the
  // blob, which may take longer.
  if (supported) {
    handle = await showSaveFilePicker({
      types: [{description: 'SVG file', accept: {'image/svg+xml': ['.svg']}}],
    });
  }
  showToast(i18n.t('optimizingSVG'), Infinity);
  svg = await optimizeSVG(svg);
  showToast(i18n.t('savedSVG'));
  const blob = new Blob([svg], {type: 'image/svg+xml'});
  await fileSave(blob, {description: 'SVG file'}, handle);
} catch (err) {
  console.error(err.name, err.message);
  showToast(err.message);
}

Drag an drop #

For opening an input image, I can either use the file open feature, or, as you have seen above, just drag and drop an image file onto the app. The file open feature is pretty straightforward, more interesting is the drag and drop case. What's particularly nice about this is that you can get a file system handle from the data transfer item via the getAsFileSystemHandle() method. As mentioned before, I can persist this handle, so it's ready when the app gets reloaded.

document.addEventListener('drop', async (event) => {
  event.preventDefault();
  dropContainer.classList.remove('dropenter');
  const item = event.dataTransfer.items[0];
  if (item.kind === 'file') {
    inputImage.addEventListener(
      'load',
      () => {
        URL.revokeObjectURL(blobURL);
      },
      {once: true},
    );
    const handle = await item.getAsFileSystemHandle();
    if (handle.kind !== 'file') {
      return;
    }
    const file = await handle.getFile();
    const blobURL = URL.createObjectURL(file);
    inputImage.src = blobURL;
    await set(FILE_HANDLE, handle);
  }
});

For more details, check out the article on the File System Access API and, if you're interested, study the SVGcode source code in src/js/filesystem.js.

Async Clipboard API #

SVGcode is also fully integrated with the operating system's clipboard via the Async Clipboard API. You can paste images from the operating system's file explorer into the app either by clicking the paste image button or by pressing command or control plus v on your keyboard.

Pasting an image from the file explorer into SVGcode.

The Async Clipboard API has recently gained the ability to deal with SVG images as well, so you can also copy an SVG image and paste it into another application for further processing.

Copying an image from SVGcode into SVGOMG.

copyButton.addEventListener('click', async () => {
  let svg = svgOutput.innerHTML;
  showToast(i18n.t('optimizingSVG'), Infinity);
  svg = await optimizeSVG(svg);
  const textBlob = new Blob([svg], {type: 'text/plain'});
  const svgBlob = new Blob([svg], {type: 'image/svg+xml'});
  navigator.clipboard.write([
    new ClipboardItem({
      [svgBlob.type]: svgBlob,
      [textBlob.type]: textBlob,
    }),
  ]);
  showToast(i18n.t('copiedSVG'));
});

To learn more, read the Async Clipboard article, or see the file src/js/clipboard.js.

File Handling #

One of my favorite features of SVGcode is how well it blends in with the operating system. As an installed PWA, it can become a file handler, or even the default file handler, for image files. This means that when I'm in the Finder on my macOS machine, I can right-click an image and open it with SVGcode. This feature is called File Handling and works based on the file_handlers property in the Web App Manifest and the launch queue, which allows the app to consume the passed file.

Opening a file from the desktop with installed SVGcode app.

window.launchQueue.setConsumer(async (launchParams) => {
  if (!launchParams.files.length) {
    return;
  }
  for (const handle of launchParams.files) {
    const file = await handle.getFile();
    if (file.type.startsWith('image/')) {
      const blobURL = URL.createObjectURL(file);
      inputImage.addEventListener(
        'load',
        () => {
          URL.revokeObjectURL(blobURL);
        },
        {once: true},
      );
      inputImage.src = blobURL;
      await set(FILE_HANDLE, handle);
      return;
    }
  }
});

For more information, see Let installed web applications be file handlers, and view the source code in src/js/filehandling.js.

Another example of blending in with the operating system is the app's share feature. Assuming I want to make edits to an SVG created with SVGcode, one way to deal with this would be to save the file, launch the SVG editing app, and then open the SVG file from there. A smoother flow, though, is to use the Web Share API, which allows for files to be shared directly. So if the SVG editing app is a share target, it can directly receive the file without deviation.

shareSVGButton.addEventListener('click', async () => {
  let svg = svgOutput.innerHTML;
  svg = await optimizeSVG(svg);
  const suggestedFileName =
    getSuggestedFileName(await get(FILE_HANDLE)) || 'Untitled.svg';
  const file = new File([svg], suggestedFileName, { type: 'image/svg+xml' });
  const data = {
    files: [file],
  };
  if (navigator.canShare(data)) {
    try {
      await navigator.share(data);
    } catch (err) {
      if (err.name !== 'AbortError') {
        console.error(err.name, err.message);
      }
    }
  }
});

Sharing an SVG image to Gmail.

The other way round, SVGcode can also act as a share target and receive files from other apps. To make this work, the app needs to let the operating system know via the Web Share Target API what types of data it can accept. This happens via a dedicated field in the Web App Manifest.

{
  "share_target": {
    "action": "https://svgco.de/share-target/",
    "method": "POST",
    "enctype": "multipart/form-data",
    "params": {
      "files": [
        {
          "name": "image",
          "accept": ["image/jpeg", "image/png", "image/webp", "image/gif"]
        }
      ]
    }
  }
}

The action route does not actually exist, but is handled purely in the service worker's fetch handler, which then passes on received files for actual processing in the app.

self.addEventListener('fetch', (fetchEvent) => {
  if (
    fetchEvent.request.url.endsWith('/share-target/') &&
    fetchEvent.request.method === 'POST'
  ) {
    return fetchEvent.respondWith(
      (async () => {
        const formData = await fetchEvent.request.formData();
        const image = formData.get('image');
        const keys = await caches.keys();
        const mediaCache = await caches.open(
          keys.filter((key) => key.startsWith('media'))[0],
        );
        await mediaCache.put('shared-image', new Response(image));
        return Response.redirect('./?share-target', 303);
      })(),
    );
  }
});

Sharing a screenshot to SVGcode.

Conclusion #

Alright, this was a quick tour through some of the advanced app features in SVGcode. I hope this app can become an essential tool for your image processing needs alongside other amazing apps like Squoosh or SVGOMG.

SVGcode is available at svgco.de. See what I did there? You can review its source code on GitHub. Note that since Potrace is GPL-licensed, so is SVGcode. And with that, happy vectorizing! I hope SVGcode will be useful, and some of its features can inspire your next app.

Acknowledgements #

This article was reviewed by Joe Medley.

JavaScript eventing deep dive

2021-09-10T00:00:00Z

Event.stopPropagation() and Event.preventDefault() #

JavaScript event handling is often straightforward. This is especially true when dealing with a simple (relatively flat) HTML structure. Things get a bit more involved though when events are traveling (or propagating) through a hierarchy of elements. This is typically when developers reach for stopPropagation() and/or preventDefault() to solve the problems they're experiencing. If you've ever thought to yourself "I'll just try preventDefault() and if that doesn't work I'll try stopPropagation() and if that doesn't work, I'll try both," then this article is for you! I will explain exactly what each method does, when to use which one, and provide you with a variety of working examples for you to explore. My goal is to end your confusion once and for all.

Before we dive too deeply though, it's important to briefly touch on the two kinds of event handling possible in JavaScript (in all modern browsers that is—Internet Explorer prior to version 9 did not support event capturing at all).

Eventing styles (capturing and bubbling) #

All modern browsers support event capturing, but it is very rarely used by developers. Interestingly, it was the only form of eventing that Netscape originally supported. Netscape's biggest rival, Microsoft Internet Explorer, did not support event capturing at all, but rather, only supported another style of eventing called event bubbling. When the W3C was formed, they found merit in both styles of eventing and declared that browsers should support both, via a third parameter to the addEventListener method. Originally, that parameter was just a boolean, but all modern browsers support an options object as the third parameter, which you can use to specify (among other things) if you want to use event capturing or not:

someElement.addEventListener('click', myClickHandler, { capture: true | false });

Note that the options object is optional, as is its capture property. If either is omitted, the default value for capture is false, meaning event bubbling will be used.

Event capturing #

What does it mean if your event handler is "listening in the capturing phase?" To understand this, we need to know how events originate and how they travel. The following is true of all events, even if you, as the developer, don't leverage it, care about it, or think about it.

All events begin at the window and first go through the capturing phase. This means that when an event is dispatched, it starts the window and travels "downwards" towards its target element first. This happens even if you are only listening in the bubbling phase. Consider the following example markup and JavaScript:

<html>
  <body>
    <div id="A">
      <div id="B">
        <div id="C"></div>
      </div>
    </div>
  </body>
</html>

document.getElementById('C').addEventListener(
  'click',
  function (e) {
    console.log('#C was clicked');
  },
  true,
);

When a user clicks on element #C, an event, originating at the window, is dispatched. This event will then propagate through its descendants as follows:

window => document => <html> => <body> => and so on, until it reaches the target.

It does not matter if nothing is listening for a click event at the window or document or <html> element or <body> element (or any other element on the way to its target). An event still originates at the window and begins its journey as just described.

In our example, the click event will then propagate (this is an important word as it will tie directly into how the stopPropagation() method works and will be explained later in this document) from the window to its target element (in this case, #C) by way of every element between the window and #C.

This means that the click event will begin at window and the browser will ask the following questions:

"Is anything listening for a click event on the window in the capturing phase?" If so, the appropriate event handlers will fire. In our example, nothing is, so no handlers will fire.

Next, the event will propagate to the document and the browser will ask: "Is anything listening for a click event on the document in the capturing phase?" If so, the appropriate event handlers will fire.

Next, the event will propagate to the <html> element and the browser will ask: "Is anything listening for a click on the <html> element in the capturing phase?" If so, the appropriate event handlers will fire.

Next, the event will propagate to the <body> element and the browser will ask: "Is anything listening for a click event on the <body> element in the capturing phase?" If so, the appropriate event handlers will fire.

Next, the event will propagate to the #A element. Again, the browser will ask: "Is anything listening for a click event on #A in the capturing phase and if so, the appropriate event handlers will fire.

Next, the event will propagate to the #B element (and the same question will be asked).

Finally, the event will reach its target and the browser will ask: "Is anything listening for a click event on the #C element in the capturing phase?" The answer this time is "yes!" This brief period of time when the event is at the target, is known as the "target phase." At this point, the event handler will fire, the browser will console.log "#C was clicked" and then we're done, right? Wrong! We're not done at all. The process continues, but now it changes to the bubbling phase.

Event bubbling #

The browser will ask:

"Is anything listening for a click event on #C in the bubbling phase?" Pay close attention here. It is completely possible to listen for clicks (or any event type) in both the capturing and bubbling phases. And if you had wired up event handlers in both phases (e.g. by calling .addEventListener() twice, once with capture = true and once with capture = false), then yes, both event handlers would absolutely fire for the same element. But it's also important to note that they fire in different phases (one in the capturing phase and one in the bubbling phase).

Next, the event will propagate (more commonly stated as "bubble" because it seems as though the event is traveling "up" the DOM tree) to its parent element, #B, and the browser will ask: "Is anything listening for click events on #B in the bubbling phase?" In our example, nothing is, so no handlers will fire.

Next, the event will bubble to #A and the browser will ask: "Is anything listening for click events on #A in the bubbling phase?"

Next, the event will bubble to <body>: "Is anything listening for click events on the <body> element in the bubbling phase?"

Next, the <html> element: "Is anything listening for click events on the <html> element in the bubbling phase?

Next, the document: "Is anything listening for click events on the document in the bubbling phase?"

Finally, the window: "Is anything listening for click events on the window in the bubbling phase?"

Phew! That was a long journey, and our event is probably very tired by now, but believe it or not, that is the journey every event goes through! Most of the time, this is never noticed because developers are typically only interested in one event phase or the other (and it is usually the bubbling phase).

It's worth spending some time playing around with event capturing and event bubbling and logging some notes to the console as handlers fire. It's very insightful to see the path that an event takes. Here is an example that listens to every element in both phases.

<html>
  <body>
    <div id="A">
      <div id="B">
        <div id="C"></div>
      </div>
    </div>
  </body>
</html>

document.addEventListener(
  'click',
  function (e) {
    console.log('click on document in capturing phase');
  },
  true,
);
// document.documentElement == <html>
document.documentElement.addEventListener(
  'click',
  function (e) {
    console.log('click on <html> in capturing phase');
  },
  true,
);
document.body.addEventListener(
  'click',
  function (e) {
    console.log('click on <body> in capturing phase');
  },
  true,
);
document.getElementById('A').addEventListener(
  'click',
  function (e) {
    console.log('click on #A in capturing phase');
  },
  true,
);
document.getElementById('B').addEventListener(
  'click',
  function (e) {
    console.log('click on #B in capturing phase');
  },
  true,
);
document.getElementById('C').addEventListener(
  'click',
  function (e) {
    console.log('click on #C in capturing phase');
  },
  true,
);

document.addEventListener(
  'click',
  function (e) {
    console.log('click on document in bubbling phase');
  },
  false,
);
// document.documentElement == <html>
document.documentElement.addEventListener(
  'click',
  function (e) {
    console.log('click on <html> in bubbling phase');
  },
  false,
);
document.body.addEventListener(
  'click',
  function (e) {
    console.log('click on <body> in bubbling phase');
  },
  false,
);
document.getElementById('A').addEventListener(
  'click',
  function (e) {
    console.log('click on #A in bubbling phase');
  },
  false,
);
document.getElementById('B').addEventListener(
  'click',
  function (e) {
    console.log('click on #B in bubbling phase');
  },
  false,
);
document.getElementById('C').addEventListener(
  'click',
  function (e) {
    console.log('click on #C in bubbling phase');
  },
  false,
);

The console output will depend on which element you click. If you were to click on the "deepest" element in the DOM tree (the #C element), you will see every single one of these event handlers fire. With a bit of CSS styling to make it more obvious which element is which, here is the console output #C element (with a screenshot as well):

"click on document in capturing phase"
"click on <html> in capturing phase"
"click on <body> in capturing phase"
"click on #A in capturing phase"
"click on #B in capturing phase"
"click on #C in capturing phase"
"click on #C in bubbling phase"
"click on #B in bubbling phase"
"click on #A in bubbling phase"
"click on <body> in bubbling phase"
"click on <html> in bubbling phase"
"click on document in bubbling phase"

You can interactively play with this in the live demo below. Click on the #C element and observe the console output.

`event.stopPropagation()` #

With an understanding of where events originate and how they travel (i.e. propagate) through the DOM in both the capturing phase and the bubbling phase, we can now turn our attention to event.stopPropagation().

The stopPropagation() method can be called on (most) native DOM events. I say "most" because there are a few on which calling this method won't do anything (because the event doesn't propagate to begin with). Events like focus, blur, load, scroll, and a few others fall into this category. You can call stopPropagation() but nothing interesting will happen, since these events don't propagate.

But what does `stopPropagation` do? #

It does, pretty much, just what it says. When you call it, the event will, from that point, cease propagating to any elements it would otherwise travel to. This is true of both directions (capturing and bubbling). So if you call stopPropagation() anywhere in the capturing phase, the event will never make it to the target phase or bubbling phase. If you call it in the bubbling phase, it will have already gone through the capturing phase, but it will cease "bubbling up" from the point at which you called it.

Returning to our same example markup, what do you think would happen, if we called stopPropagation() in the capturing phase at the #B element?

It would result in the following output:

"click on document in capturing phase"
"click on <html> in capturing phase"
"click on <body> in capturing phase"
"click on #A in capturing phase"
"click on #B in capturing phase"

You can interactively play with this in the live demo below. Click on the #C element in the live demo and observe the console output.

How about stopping propagation at #A in the bubbling phase? That would result in the following output:

"click on document in capturing phase"
"click on <html> in capturing phase"
"click on <body> in capturing phase"
"click on #A in capturing phase"
"click on #B in capturing phase"
"click on #C in capturing phase"
"click on #C in bubbling phase"
"click on #B in bubbling phase"
"click on #A in bubbling phase"

You can interactively play with this in the live demo below. Click on the #C element in the live demo and observe the console output.

One more, just for fun. What happens if we call stopPropagation() in the target phase for #C? Recall that the "target phase" is the name given to the period of time when the event is at its target. It would result in the following output:

"click on document in capturing phase"
"click on <html> in capturing phase"
"click on <body> in capturing phase"
"click on #A in capturing phase"
"click on #B in capturing phase"
"click on #C in capturing phase"

Note that the event handler for #C in which we log "click on #C in the capturing phase" still executes, but the one in which we log "click on #C in the bubbling phase" does not. This should make perfect sense. We called stopPropagation() from the former, so that is the point at which the event's propagation will cease.

You can interactively play with this in the live demo below. Click on the #C element in the live demo and observe the console output.

In any of these live demos, I encourage you to play around. Try clicking on the #A element only or the body element only. Try to predict what will happen and then observe if you are correct. At this point, you should be able to predict pretty accurately.

`event.stopImmediatePropagation()` #

What is this strange, and not oft-used method? It's similar to stopPropagation, but rather than stopping an event from traveling to descendents (capturing) or ancestors (bubbling), this method only applies when you have more than one event handler wired up to a single element. Since addEventListener() supports a multicast style of eventing, it's completely possible to wire up an event handler to a single element more than once. When this happens, (in most browsers), event handlers are executed in the order they were wired up. Calling stopImmediatePropagation() prevents any subsequent handlers from firing. Consider the following example:

<html>
  <body>
    <div id="A">I am the #A element</div>
  </body>
</html>

document.getElementById('A').addEventListener(
  'click',
  function (e) {
    console.log('When #A is clicked, I shall run first!');
  },
  false,
);

document.getElementById('A').addEventListener(
  'click',
  function (e) {
    console.log('When #A is clicked, I shall run second!');
    e.stopImmediatePropagation();
  },
  false,
);

document.getElementById('A').addEventListener(
  'click',
  function (e) {
    console.log('When #A is clicked, I would have run third, if not for stopImmediatePropagation');
  },
  false,
);

The above example will result in the following console output:

"When #A is clicked, I shall run first!"
"When #A is clicked, I shall run second!"

Note that the third event handler never runs due to the fact that the second event handler calls e.stopImmediatePropagation(). If we had instead called e.stopPropagation(), the third handler would still run.

`event.preventDefault()` #

If stopPropagation() prevents an event from traveling "downwards" (capturing) or "upwards" (bubbling), what then, does preventDefault() do? It sounds like it does something similar. Does it?

Not really. While the two are often confused, they actually don't have much to do with each other. When you see preventDefault(), in your head, add the word "action." Think "prevent the default action."

And what is the default action you may ask? Unfortunately, the answer to that isn't quite as clear because it's highly dependent on the element + event combination in question. And to make matters even more confusing, sometimes there is no default action at all!

Let's begin with a very simple example to understand. What do you expect to happen when you click a link on a web page? Obviously, you expect the browser to navigate to the URL specified by that link. In this case, the element is an anchor tag and the event is a click event. That combination (<a> + click) has a "default action" of navigating to the link's href. What if you wanted to prevent the browser from performing that default action? That is, suppose you want to prevent the browser from navigating to the URL specified by the <a> element's href attribute? This is what preventDefault() will do for you. Consider this example:

<a id="avett" href="https://www.theavettbrothers.com/welcome">The Avett Brothers</a>

document.getElementById('avett').addEventListener(
  'click',
  function (e) {
    e.preventDefault();
    console.log('Maybe we should just play some of their music right here instead?');
  },
  false,
);

You can interactively play with this in the live demo below. Click the link The Avett Brothers and observe the console output (and the fact that you are not redirected to the Avett Brothers website).

Normally, clicking the link labelled The Avett Brothers would result in browsing to www.theavettbrothers.com. In this case though, we've wired up a click event handler to the <a> element and specified that the default action should be prevented. Thus, when a user clicks this link, they won't be navigated anywhere, and instead the console will simply log "Maybe we should just play some of their music right here instead?"

What other element/event combinations allow you to prevent the default action? I cannot possibly list them all, and sometimes you have to just experiment to see. But briefly, here are a few:

<form> element + "submit" event: preventDefault() for this combination will prevent a form from submitting. This is useful if you want to perform validation and should something fail, you can conditionally call preventDefault to stop the form from submitting.
<a> element + "click" event: preventDefault() for this combination prevents the browser from navigating to the URL specified in the <a> element's href attribute.
document + "mousewheel" event: preventDefault() for this combination prevents page scrolling with the mousewheel (scrolling with keyboard would still work though).
↜ This requires calling addEventListener() with { passive: false }.
document + "keydown" event: preventDefault() for this combination is lethal. It renders the page largely useless, preventing keyboard scrolling, tabbing, and keyboard highlighting.
document + "mousedown" event: preventDefault() for this combination will prevent text highlighting with the mouse and any other "default" action that one would invoke with a mouse down.
<input> element + "keypress" event: preventDefault() for this combination will prevent characters typed by the user from reaching the input element (but don't do this; there is rarely, if ever, a valid reason for it).
document + "contextmenu" event: preventDefault() for this combination prevents the native browser context menu from appearing when a user right-clicks or long-presses (or any other way in which a context menu might appear).

This is not an exhaustive list by any means, but hopefully it gives you a good idea of how preventDefault() can be used.

A fun practical joke? #

What happens if you stopPropagation() and preventDefault() in the capturing phase, starting at the document? Hilarity ensues! The following code snippet will render any web page just about completely useless:

function preventEverything(e) {
  e.preventDefault();
  e.stopPropagation();
  e.stopImmediatePropagation();
}

document.addEventListener('click', preventEverything, true);
document.addEventListener('keydown', preventEverything, true);
document.addEventListener('mousedown', preventEverything, true);
document.addEventListener('contextmenu', preventEverything, true);
document.addEventListener('mousewheel', preventEverything, { capture: true, passive: false });

I don't really know why you'd ever want to do this (except maybe to play a joke on someone), but it is useful to think about what's happening here, and realize why it creates the situation it does.

All events originate at window, so in this snippet, we're stopping, dead in their tracks, all click, keydown, mousedown, contextmenu, and mousewheel events from ever getting to any elements that might be listening for them. We also call stopImmediatePropagation so that any handlers wired up to the document after this one, will be thwarted as well.

Note that stopPropagation() and stopImmediatePropagation() aren't (at least not mostly) what render the page useless. They simply prevent events from getting where they would otherwise go.

But we also call preventDefault(), which you'll recall prevents the default action. So any default actions (like mousewheel scroll, keyboard scroll or highlight or tabbing, link clicking, context menu display, etc.) are all prevented, thus leaving the page in a fairly useless state.

Live demos #

To explore all the examples from this article again in one place, check out the embedded demo below.

Acknowledgements #

Hero image by Tom Wilson on Unsplash.

User preference media features client hints headers

2021-08-02T00:00:00Z

CSS media queries, and specifically user preference media features like prefers-color-scheme or prefers-reduced-motion, have a potentially significant impact on the amount of CSS that needs to be delivered by a page, and on the experience the user is going to have when the page loads.

Focusing on prefers-color-scheme—but highlighting that the reasoning applies to other user preference media features as well—it is a best practice to not load CSS for the particular non-matching color scheme in the critical rendering path, and instead to initially only load the currently relevant CSS. One way of doing so is via <link media>.

However, high-traffic sites like Google Search that wish to honor user preference media features like prefers-color-scheme and that inline CSS for performance reasons, need to know about the preferred color scheme (or other user preference media features respectively) ideally at request time, so that the initial HTML payload already has the right CSS inlined.

Additionally, and specifically for prefers-color-scheme, sites by all means want to avoid a flash of inaccurate color theme.

The Sec-CH-Prefers-Color-Scheme and the Sec-CH-Prefers-Reduced-Motion client hint headers are the first of a series of user preference media features client hints headers that aims to solve this issue.

Background on client hints #

HTTP Client Hints defines an Accept-CH response header that servers can use to advertise their use of request headers for proactive content negotiation, colloquially referred to as client hints. The User Preference Media Features Client Hints Headers proposal defines a set of client hints aimed at conveying user preference media features. These client hints are named after the corresponding user preference media feature that they report on. For example, the currently preferred color scheme as per prefers-color-scheme is reported via the aptly named Sec-CH-Prefers-Color-Scheme client hint.

Background on critical client hints #

The client hints proposed in User Preference Media Features Client Hints Headers will presumably most commonly be used as Critical Client Hints. Critical Client Hints are Client Hints which meaningfully change the resulting resource. Such a resource should be fetched consistently across page loads (including the initial page load) to avoid jarring user-visible switches.

Client hint syntax #

User preference media features consist of a name (like prefers-reduced-motion) and allowed values (like no-preference or reduce). Each client hint header field is represented as Structured Headers for HTTP object containing an item whose value is a string. For example, to convey that the user prefers a dark theme and reduced motion, the client hints look like in the example below.

GET / HTTP/2
Host: example.com
Sec-CH-Prefers-Color-Scheme: "dark"
Sec-CH-Prefers-Reduced-Motion: "reduce"

The CSS equivalent of the information conveyed in the above client hints is @media (prefers-color-scheme: dark) {} and @media (prefers-reduced-motion: reduce) {} respectively.

Complete list of the client hints #

The list of the client hints is modeled after the user preference media features in Media Queries Level 5.

Client Hint	Allowed Values	Corresponding User Preference Media Feature
`Sec-CH-Prefers-Reduced-Motion`	`no-preference`, `reduce`	`prefers-reduced-motion`
`Sec-CH-Prefers-Reduced-Transparency`	`no-preference`, `reduce`	`prefers-reduced-transparency`
`Sec-CH-Prefers-Contrast`	`no-preference`, `less`, `more`, `custom`	`prefers-contrast`
`Sec-CH-Forced-Colors`	`active`, `none`	`forced-colors`
`Sec-CH-Prefers-Color-Scheme`	`light`, `dark`	`prefers-color-scheme`
`Sec-CH-Prefers-Reduced-Data`	`no-preference`, `reduce`	`prefers-reduced-data`

Browser support #

The Sec-CH-Prefers-Color-Scheme client hint header is supported on Chromium 93. The Sec-CH-Prefers-Reduced-Motion client hint header is supported on Chromium 108. Other vendors' feedback, namely WebKit's and Mozilla's, is pending.

Demo of `Sec-CH-Prefers-Color-Scheme` #

Try the demo in Chromium 93 and notice how the inlined CSS changes according to the user's preferred color scheme.

Demo of `Sec-CH-Prefers-Reduced-Motion` #

Try the demo in Chromium 108 and notice how the inlined CSS changes according to the user's motion preferences.

How it works #

The client makes an initial request to the server.
```
GET / HTTP/2
Host: example.com
```
The server responds, telling the client via Accept-CH that it accepts the Sec-CH-Prefers-Color-Scheme and the Sec-CH-Prefers-Contrast Client Hints, out of which as per Critical-CH it considers Sec-CH-Prefers-Color-Scheme a Critical Client Hint that it also varies the response on as conveyed by Vary.
```
HTTP/2 200 OK
Content-Type: text/html
Accept-CH: Sec-CH-Prefers-Color-Scheme, Sec-CH-Prefers-Contrast
Vary: Sec-CH-Prefers-Color-Scheme
Critical-CH: Sec-CH-Prefers-Color-Scheme
```
The client then retries the request, telling the server via Sec-CH-Prefers-Color-Scheme that it has a user preference for dark-schemed content.
```
GET / HTTP/2
Host: example.com
Sec-CH-Prefers-Color-Scheme: "dark"
```
The server can then tailor the response to the client's preferences accordingly and, for example, inline the CSS responsible for the dark theme into the response body.

Node.js example #

The Node.js example below, written for the popular Express.js framework, shows how dealing with the Sec-CH-Prefers-Color-Scheme client hint header could look like in practice. This code is what actually powers the demo above.

app.get("/", (req, res) => {
  // Tell the client the server accepts the `Sec-CH-Prefers-Color-Scheme` client hint…
  res.set("Accept-CH", "Sec-CH-Prefers-Color-Scheme");
  // …and that the server's response will vary based on its value…
  res.set("Vary", "Sec-CH-Prefers-Color-Scheme");
  // …and that the server considers this client hint a _critical_ client hint.
  res.set("Critical-CH", "Sec-CH-Prefers-Color-Scheme");
  // Read the user's preferred color scheme from the headers…
  const prefersColorScheme = req.get("sec-ch-prefers-color-scheme");
  // …and send the adequate HTML response with the right CSS inlined.
  res.send(getHTML(prefersColorScheme));
});

Privacy and security considerations #

The Chromium team designed and implemented User Preference Media Features Client Hints Headers using the core principles defined in Controlling Access to Powerful Web Platform Features, including user control, transparency, and ergonomics.

The Security Considerations of HTTP Client Hints and the Security Considerations of Client Hint Reliability likewise apply to this proposal.

References #

Acknowledgements #

Many thanks for valuable feedback and advice from Yoav Weiss. Hero image by Tdadamemd on Wikimedia Commons.

Excalidraw and Fugu: Improving Core User Journeys

2021-05-18T00:00:00Z

How I came to Excalidraw #

I want to start with a story. On January 1st, 2020, Christopher Chedeau, a software engineer at Facebook, tweeted about a small drawing app he had started to work on. With this tool, you could draw boxes and arrows that feel cartoony and hand-drawn. The next day, you could also draw ellipses and text, as well as select objects and move them around. On January 3, the app had gotten its name, Excalidraw, and, like with every good side project, buying the domain name was one of Christopher's first acts. By now, you could use colors and export the whole drawing as a PNG.

On January 15, Christopher put out a blog post that drew a lot of attention on Twitter, including mine. The post started with some impressive stats:

12K unique active users
1.5K stars on GitHub
26 contributors

For a project that started a mere two weeks ago, that's not bad at all. But the thing that truly spiked my interest was further down in the post. Christopher wrote that he tried something new this time: giving everyone who landed a pull request unconditional commit access. The same day of reading the blog post, I had a pull request up that added File System Access API support to Excalidraw, fixing a feature request that someone had filed.

My pull request was merged a day later and from thereon, I had full commit access. Needless to say, I didn't abuse my power. And nor did anybody else from the 149 contributors so far.

Today, Excalidraw is a full-fledged installable progressive web app with offline support, a stunning dark mode, and yes, the ability to open and save files thanks to the File System Access API.

Lipis on why he dedicates so much of his time to Excalidraw #

So this marks the end of my "how I came to Excalidraw" story, but before I dive into some of Excalidraw's amazing features, I have the pleasure to introduce Panayiotis. Panayiotis Lipiridis, on the Internet simply known as lipis, is the most prolific contributor to Excalidraw. I asked lipis what motivates him to dedicate so much of his time to Excalidraw:

Like everyone else I learned about this project from Christopher's tweet. My first contribution was to add the Open Color library, the colors that are still part of Excalidraw today. As the project grew and we had quite many requests, my next big contribution was to build a backend for storing drawings so users could share them. But what really drives me to contribute is that whoever tried Excalidraw is looking to find excuses to use it again.

I fully agree with lipis. Whoever tried Excalidraw is looking to find excuses to use it again.

Excalidraw in action #

I want to show you now how you can use Excalidraw in practice. I'm not a great artist, but the Google I/O logo is simple enough, so let me give it a try. A box is the "i", a line can be the slash, and the "o" is a circle. I hold down shift, so I get a perfect circle. Let me move the slash a little, so it looks better. Now some color for the "i" and the "o". Blue is good. Maybe a different fill style? All solid, or cross-hatch? Nah, hachure looks great. It's not perfect, but that's the idea of Excalidraw, so let me save it.

I click the save icon and enter a file name in the file save dialog. In Chrome, a browser that supports the File System Access API, this is not a download, but a true save operation, where I can choose the location and name of the file, and where, if I make edits, I can just save them to the same file.

Let me change the logo and make the "i" red. If I now click save again, my modification is saved to the same file as before. As a proof, let me clear the canvas and reopen the file. As you can see, the modified red-blue logo is there again.

Working with files #

On browsers that currently don't support the File System Access API, each save operation is a download, so when I make changes, I end up with multiple files with an incrementing number in the filename that fill up my Downloads folder. But despite this downside, I can still save the file.

Opening files #

So what's the secret? How can opening and saving work on different browsers that may or may not support the File System Access API? Opening a file in Excalidraw happens in a function called loadFromJSON)(), which in turn calls a function called fileOpen().

export const loadFromJSON = async (localAppState: AppState) => {
  const blob = await fileOpen({
    description: 'Excalidraw files',
    extensions: ['.json', '.excalidraw', '.png', '.svg'],
    mimeTypes: ['application/json', 'image/png', 'image/svg+xml'],
  });
  return loadFromBlob(blob, localAppState);
};

The fileOpen() function that comes from a small library I wrote called browser-fs-access that we use in Excalidraw. This library provides file system access through the File System Access API with a legacy fallback, so it can be used in any browser.

Let me first show you the implementation for when the API is supported. After negotiating the accepted MIME types and file extensions, the central piece is calling the File System Access API's function showOpenFilePicker(). This function returns an array of files or a single file, dependent on whether multiple files are selected. All that's left then is to put the file handle on the file object, so it can be retrieved again.

export default async (options = {}) => {
  const accept = {};
  // Not shown: deal with extensions and MIME types.
  const handleOrHandles = await window.showOpenFilePicker({
    types: [
      {
        description: options.description || '',
        accept: accept,
      },
    ],
    multiple: options.multiple || false,
  });
  const files = await Promise.all(handleOrHandles.map(getFileWithHandle));
  if (options.multiple) return files;
  return files[0];
  const getFileWithHandle = async (handle) => {
    const file = await handle.getFile();
    file.handle = handle;
    return file;
  };
};

The fallback implementation relies on an input element of type "file". After the negotiation of the to-be-accepted MIME types and extensions, the next step is to programmatically click the input element so the file open dialog shows. On change, that is, when the user has selected one or multiple files, the promise resolves.

export default async (options = {}) => {
  return new Promise((resolve) => {
    const input = document.createElement('input');
    input.type = 'file';
    const accept = [
      ...(options.mimeTypes ? options.mimeTypes : []),
      options.extensions ? options.extensions : [],
    ].join();
    input.multiple = options.multiple || false;
    input.accept = accept || '*/*';
    input.addEventListener('change', () => {
      resolve(input.multiple ? Array.from(input.files) : input.files[0]);
    });
    input.click();
  });
};

Saving files #

Now to saving. In Excalidraw, saving happens in a function called saveAsJSON(). It first serializes the Excalidraw elements array to JSON, converts the JSON to a blob, and then calls a function called fileSave(). This function is likewise provided by the browser-fs-access library.

export const saveAsJSON = async (
  elements: readonly ExcalidrawElement[],
  appState: AppState,
) => {
  const serialized = serializeAsJSON(elements, appState);
  const blob = new Blob([serialized], {
    type: 'application/vnd.excalidraw+json',
  });
  const fileHandle = await fileSave(
    blob,
    {
      fileName: appState.name,
      description: 'Excalidraw file',
      extensions: ['.excalidraw'],
    },
    appState.fileHandle,
  );
  return { fileHandle };
};

Again let me first look at the implementation for browsers with File System Access API support. The first couple of lines look a little involved, but all they do is negotiate the MIME types and file extensions. When I have saved before and already have a file handle, no save dialog needs to be shown. But if this is the first save, a file dialog gets displayed and the app gets a file handle back for future use. The rest is then just writing to the file, which happens through a writable stream.

export default async (blob, options = {}, handle = null) => {
  options.fileName = options.fileName || 'Untitled';
  const accept = {};
  // Not shown: deal with extensions and MIME types.
  handle =
    handle ||
    (await window.showSaveFilePicker({
      suggestedName: options.fileName,
      types: [
        {
          description: options.description || '',
          accept: accept,
        },
      ],
    }));
  const writable = await handle.createWritable();
  await writable.write(blob);
  await writable.close();
  return handle;
};

The "save as" feature #

If I decide to ignore an already existing file handle, I can implement a "save as" feature to create a new file based on an existing file. To show this, let me open an existing file, make some modification, and then not overwrite the existing file, but create a new file by using the save-as feature. This leaves the original file intact.

The implementation for browsers that don't support the File System Access API is short, since all it does is create an anchor element with a download attribute whose value is the desired filename and a blob URL as its href attribute value.

export default async (blob, options = {}) => {
  const a = document.createElement('a');
  a.download = options.fileName || 'Untitled';
  a.href = URL.createObjectURL(blob);
  a.addEventListener('click', () => {
    setTimeout(() => URL.revokeObjectURL(a.href), 30 * 1000);
  });
  a.click();
};

The anchor element then gets clicked programmatically. To prevent memory leaks, the blob URL needs to be revoked after use. As this is just a download, no file save dialog gets shown ever, and all files land in the default Downloads folder.

Drag and drop #

One of my favorite system integrations on desktop is drag and drop. In Excalidraw, when I drop an .excalidraw file onto the application, it opens right away and I can start editing. On browsers that support the File System Access API, I can then even immediately save my changes. No need to go through a file save dialog since the required file handle has been obtained from the drag and drop operation.

The secret for making this happen is by calling getAsFileSystemHandle() on the data transfer item when the File System Access API is supported. I then pass this file handle to loadFromBlob(), which you may remember from a couple paragraphs above. So many things you can do with files: opening, saving, over-saving, dragging, dropping. My colleague Pete and I have documented all these tricks and more in our article so you can catch up in case all this went a little too fast.

const file = event.dataTransfer?.files[0];
if (file?.type === 'application/json' || file?.name.endsWith('.excalidraw')) {
  this.setState({ isLoading: true });
  // Provided by browser-fs-access.
  if (supported) {
    try {
      const item = event.dataTransfer.items[0];
      file as any.handle = await item as any
        .getAsFileSystemHandle();
    } catch (error) {
      console.warn(error.name, error.message);
    }
  }
  loadFromBlob(file, this.state).then(({ elements, appState }) =>
    // Load from blob
  ).catch((error) => {
    this.setState({ isLoading: false, errorMessage: error.message });
  });
}

Another system integration currently on Android, ChromeOS, and Windows is through the Web Share Target API. Here I am in the Files app in my Downloads folder. I can see two files, one of them with the non-descript name untitled and a timestamp. To check what it contains, I click on the three dots, then share, and one of the options that appears is Excalidraw. When I tap the icon, I can then see that the file just contains the I/O logo again.

Lipis on the deprecated Electron version #

One thing you can do with files that I haven't talked about yet is doubleclick them. What typically happens when you doubleclick a file is that the app that's associated with the file's MIME type opens. For example for .docx this would be Microsoft Word.

Excalidraw used to have an Electron version of the app that supported such file type associations, so when you double-clicked an .excalidraw file, the Excalidraw Electron app would open. Lipis, whom you have already met before, was both the creator and the deprecator of Excalidraw Electron. I asked him why he felt it was possible to deprecate the Electron version:

People have been asking for an Electron app since the beginning, mainly because they wanted to open files by double-clicking. We also intended to put the app in app stores. In parallel, someone suggested creating a PWA instead, so we just did both. Luckily we were introduced to Project Fugu APIs like file system access, clipboard access, file handling, and more. With a sole click you can install the app on your desktop or mobile, without the extra weight of Electron. It was an easy decision to deprecate the Electron version, concentrate just on the web app, and make it the best-possible PWA. On top, we're now able to publish PWAs to the Play Store and the Microsoft Store! That's huge!

One could say Excalidraw for Electron was not deprecated because Electron is bad, not at all, but because the web has become good enough. I like this!

File handling #

When I say "the web has become good enough", it's because of features like the upcoming File Handling.

This is a regular macOS Big Sur installation. Now check out what happens when I right-click an Excalidraw file. I can choose to open it with Excalidraw, the installed PWA. Of course double-clicking would work, too, it's just less dramatic to demonstrate in a screencast.

So how does this work? The first step is to make the file types my application can handle known to the operating system. I do this in a new field called file_handlers in the web app manifest. Its value is an array of objects with an action and an accept property. The action determines the URL path the operating system launches your app at and the accept object are key value pairs of MIME types and the associated file extensions.

{
  "name": "Excalidraw",
  "description": "Excalidraw is a whiteboard tool...",
  "start_url": "/",
  "display": "standalone",
  "theme_color": "#000000",
  "background_color": "#ffffff",
  "file_handlers": [
    {
      "action": "/",
      "accept": {
        "application/vnd.excalidraw+json": [".excalidraw"]
      }
    }
  ]
}

The next step is to handle the file when the application launches. This happens in the launchQueue interface where I need to set a consumer by calling, well, setConsumer(). The parameter to this function is an asynchronous function that receives the launchParams. This launchParams object has a field called files that gets me an array of file handles to work with. I only care for the first one and from this file handle I get a blob that I then pass to our old friend loadFromBlob().

if ('launchQueue' in window && 'LaunchParams' in window) {
  window as any.launchQueue
    .setConsumer(async (launchParams: { files: any[] }) => {
      if (!launchParams.files.length) return;
      const fileHandle = launchParams.files[0];
      const blob: Blob = await fileHandle.getFile();
      blob.handle = fileHandle;
      loadFromBlob(blob, this.state).then(({ elements, appState }) =>
        // Initialize app state.
      ).catch((error) => {
        this.setState({ isLoading: false, errorMessage: error.message });
      });
    });
}

Again, if this went too fast, you can read more about the File Handling API in my article. You can enable file handling by setting the experimental web platform features flag. It's scheduled to land in Chrome later this year.

Clipboard integration #

Another cool feature of Excalidraw is the clipboard integration. I can copy my entire drawing or just parts of it into the clipboard, maybe adding a watermark if I feel like, and then paste it into another app. This is a web version of the Windows 95 Paint app by the way.

The way this works is surprisingly simple. All I need is the canvas as a blob, which I then write onto the clipboard by passing a one-element array with a ClipboardItem with the blob to the navigator.clipboard.write() function. For more information on what you can do with the clipboard API, See Jason's and my article.

export const copyCanvasToClipboardAsPng = async (canvas: HTMLCanvasElement) => {
  const blob = await canvasToBlob(canvas);
  await navigator.clipboard.write([
    new window.ClipboardItem({
      'image/png': blob,
    }),
  ]);
};

export const canvasToBlob = async (canvas: HTMLCanvasElement): Promise<Blob> => {
  return new Promise((resolve, reject) => {
    try {
      canvas.toBlob((blob) => {
        if (!blob) {
          return reject(new CanvasError(t('canvasError.canvasTooBig'), 'CANVAS_POSSIBLY_TOO_BIG'));
        }
        resolve(blob);
      });
    } catch (error) {
      reject(error);
    }
  });
};

Collaborating with others #

Did you know that Excalidraw also has a collaborative mode? Different people can work together on the same document. To start a new session, I click on the live collaboration button and then start a session. I can share the session URL with my collaborators easily thanks to the Web Share API that Excalidraw has integrated.

Live collaboration #

I have simulated a collaboration session locally by working on the Google I/O logo on my Pixelbook, my Pixel 3a phone, and my iPad Pro. You can see that changes I make on one device are reflected on all other devices.

I can even see all cursors move around. The Pixelbook's cursor moves steadily, since it's controlled by a trackpad, but the Pixel 3a phone's cursor and the iPad Pro's tablet cursor jump around, since I control these devices by tapping with my finger.

Seeing collaborator statuses #

To improve the realtime collaboration experience, there is even an idle detection system running. The cursor of the iPad Pro shows a green dot when I use it. The dot turns black when I switch to a different browser tab or app. And when I'm in the Excalidraw app, but just not doing anything, the cursor shows me as idle, symbolized by the three zZZs.

Avid readers of our publications might be inclined to think that idle detection is realized through the Idle Detection API, an early stage proposal that's been worked on in the context of Project Fugu. Spoiler alert: it's not. While we had an implementation based on this API in Excalidraw, in the end, we decided to go for a more traditional approach based on measuring pointer movement and page visibility.

We filed feedback on why the Idle Detection API wasn't solving the use case we had. All Project Fugu APIs are being developed in the open, so everyone can chime in and have their voice heard!

Lipis on what is holding back Excalidraw #

Talking of which, I asked lipis one last question regarding what he thinks is missing from the web platform that holds back Excalidraw:

The File System Access API is great, but you know what? Most files that I care about these days live in my Dropbox or Google Drive, not on my hard disk. I wish the File System Access API would include an abstraction layer for remote file systems providers like Dropbox or Google to integrate with and that developers could code against. Users could then relax and know their files are safe with the cloud provider they trust.

I fully agree with lipis, I live in the cloud, too. Here's hoping that this will be implemented soon.

Tabbed application mode #

Wow! We have seen a lot of really great API integrations in Excalidraw. File system, file handling, clipboard, web share, and web share target. But here is one more thing. Up until now, I could only ever edit one document at a given time. Not anymore. Please enjoy for the first time an early version of tabbed application mode in Excalidraw. This is how it looks.

I have an existing file open in the installed Excalidraw PWA that's running in standalone mode. Now I open a new tab in the standalone window. This is not a regular browser tab, but a PWA tab. In this new tab I can then open a secondary file, and work on them independently from the same app window.

Tabbed application mode is in its early stages and not everything is set in stone. If you're interested, be sure to read up on the current status of this feature in my article.

Closing #

To stay in the loop on this and other features, be sure to watch our Fugu API tracker. We're super excited to push the web forward and allow you to do more on the platform. Here's to an ever improving Excalidraw, and here's to all the amazing applications that you will build. Go start creating at excalidraw.com.

I can't wait to see some of the APIs that I have shown today pop up in your apps. My name is Tom, you can find me as @tomayac on Twitter and the internet in general. Thank you very much for watching, and enjoy the rest of Google I/O.

Customize the window controls overlay of your PWA's title bar

2021-04-22T00:00:00Z

If you remember my article Make your PWA feel more like an app, you may recall how I mentioned customizing the title bar of your app as a strategy for creating a more app-like experience. Here is an example of how this can look showing the macOS Podcasts app.

A custom title bar makes your PWA feel more like a platform-specific app.

Now you may be tempted to object by saying that Podcasts is a platform-specific macOS app that does not run in a browser and therefore can do what it wants without having to play by the browser's rules. True, but the good news is that the Window Controls Overlay feature, which is the topic of this very article, soon lets you create similar user interfaces for your PWA.

Window Controls Overlay components #

Window Controls Overlay consists of four sub-features:

The "window-controls-overlay" value for the "display_override" field in the web app manifest.
The CSS environment variables titlebar-area-x, titlebar-area-y, titlebar-area-width, and titlebar-area-height.
The standardization of the previously proprietary CSS property -webkit-app-region as the app-region property to define draggable regions in web content.
A mechanism to query for and work around the window controls region via the windowControlsOverlay member of window.navigator.

What is Window Controls Overlay #

The title bar area refers to the space to the left or right of the window controls (that is, the buttons to minimize, maximize, close, etc.) and often contains the title of the application. Window Controls Overlay lets progressive web applications (PWAs) provide a more app-like feel by swapping the existing full-width title bar for a small overlay containing the window controls. This allows developers to place custom content in what was previously the browser-controlled title bar area.

Current status #

Step	Status
1. Create explainer	Complete
2. Create initial draft of specification	Complete
3. Gather feedback & iterate on design	In progress
4. Origin trial	Complete
5. Launch	Complete (in Chromium 104)

How to use Window Controls Overlay #

Adding `window-controls-overlay` to the web app manifest #

A progressive web app can opt-in to the window controls overlay by adding "window-controls-overlay" as the primary "display_override" member in the web app manifest:

{
  "display_override": ["window-controls-overlay"]
}

The window controls overlay will be visible only when all of the following conditions are satisfied:

The app is not opened in the browser, but in a separate PWA window.
The manifest includes "display_override": ["window-controls-overlay"]. (Other values are allowed thereafter.)
The PWA is running on a desktop operating system.
The current origin matches the origin for which the PWA was installed.

The result of this is an empty title bar area with the regular window controls on the left or the right, depending on the operating system.

An empty title bar ready for custom content.

Moving content into the title bar #

Now that there is space in the title bar, you can move something there. For this article, I built a Wikimedia Featured Content PWA. A useful feature for this app may be a search for words in the article titles. The HTML for the search feature looks like this:

<div class="search">
  <img src="logo.svg" alt="Wikimedia logo." width="32" height="32" />
  <label>
    <input type="search" />
    Search for words in articles
  </label>
</div>

To move this div up into the title bar, some CSS is needed:

.search {
  /* Make sure the `div` stays there, even when scrolling. */
  position: fixed;
  /**
   * Gradient, because why not. Endless opportunities.
   * The gradient ends in `#36c`, which happens to be the app's
   * `<meta name="theme-color" content="#36c">`.
   */
  background-image: linear-gradient(90deg, #36c, #131313, 33%, #36c);
  /* Use the environment variable for the left anchoring with a fallback. */
  left: env(titlebar-area-x, 0);
  /* Use the environment variable for the top anchoring with a fallback. */
  top: env(titlebar-area-y, 0);
  /* Use the environment variable for setting the width with a fallback. */
  width: env(titlebar-area-width, 100%);
  /* Use the environment variable for setting the height with a fallback. */
  height: env(titlebar-area-height, 33px);
}

You can see the effect of this code in the screenshot below. The title bar is fully responsive. When you resize the PWA window, the title bar reacts as if it were composed of regular HTML content, which, in fact, it is.

The new title bar is active and responsive.

Determining which parts of the title bar are draggable #

While the screenshot above suggests that you are done, you are not done quite yet. The PWA window is no longer draggable (apart from a very small area), since the window controls buttons are not drag areas, and the rest of the title bar consists of the search widget. Fix this using the app-region CSS property with a value of drag. In the concrete case, it is fine to make everything besides the input element draggable.

/* The entire search `div` is draggable… */
.search {
  -webkit-app-region: drag;
  app-region: drag;
}

/* …except for the `input`. */
input {
  -webkit-app-region: no-drag;
  app-region: no-drag;
}

With this CSS in place, the user can drag the app window as usual by dragging the div, the img, or the label. Only the input element is interactive so the search query can be entered.

Feature detection #

Support for Window Controls Overlay can be detected by testing for the existence of windowControlsOverlay:

if ('windowControlsOverlay' in navigator) {
  // Window Controls Overlay is supported.
}

Querying the window controls region with `windowControlsOverlay` #

The code so far has one problem: on some platforms the window controls are on the right, on others they are on the left. To make matters worse, the "three dots" Chrome menu will change position, too, based on the platform. This means that the linear gradient background image needs to be dynamically adapted to run from #131313→maroon or maroon→#131313→maroon, so that it blends in with the title bar's maroon background color which is determined by <meta name="theme-color" content="maroon">. This can be achieved by querying the getTitlebarAreaRect() API on the navigator.windowControlsOverlay property.

if ('windowControlsOverlay' in navigator) {
  const { x } = navigator.windowControlsOverlay.getTitlebarAreaRect();
  // Window controls are on the right (like on Windows).
  // Chrome menu is left of the window controls.
  // [ windowControlsOverlay___________________ […] [_] [■] [X] ]
  if (x === 0) {
    div.classList.add('search-controls-right');
  }
  // Window controls are on the left (like on macOS).
  // Chrome menu is right of the window controls overlay.
  // [ [X] [_] [■] ___________________windowControlsOverlay [⋮] ]
  else {
    div.classList.add('search-controls-left');
  }
} else {
  // When running in a non-supporting browser tab.
  div.classList.add('search-controls-right');
}

Rather than having the background image in the .search class CSS rules directly (as before), the modified code now uses two classes that the code above sets dynamically.

/* For macOS: */
.search-controls-left {
  background-image: linear-gradient(90deg, #36c, 45%, #131313, 90%, #36c);
}

/* For Windows: */
.search-controls-right {
  background-image: linear-gradient(90deg, #36c, #131313, 33%, #36c);
}

Determining if the window controls overlay is visible #

The window controls overlay will not be visible in the title bar area in all circumstances. While it will naturally not be there on browsers that do not support the Window Controls Overlay feature, it will also not be there when the PWA in question runs in a tab. To detect this situation, you can query the visible property of the windowControlsOverlay:

if (navigator.windowControlsOverlay.visible) {
  // The window controls overlay is visible in the title bar area.
}

Alternatively, you can also use the display-mode media query in JavaScript and/or CSS:

// Create the query list.
const mediaQueryList = window.matchMedia('(display-mode: window-controls-overlay)');

// Define a callback function for the event listener.
function handleDisplayModeChange(mql) {
  // React on display mode changes.
}

// Run the display mode change handler once.
handleOrientationChange(mediaQueryList);

// Add the callback function as a listener to the query list.
mediaQueryList.addEventListener('change', handleDisplayModeChange);

@media (display-mode: window-controls-overlay) { 
  /* React on display mode changes. */ 
}

Being notified of geometry changes #

Querying the window controls overlay area with getTitlebarAreaRect() can suffice for one-off things like setting the correct background image based on where the window controls are, but in other cases, more fine-grained control is necessary. For example, a possible use case could be to adapt the window controls overlay based on the available space and to add a joke right in the window control overlay when there is enough space.

Title bar controls adapted to a narrow window.

You can be notified of geometry changes by subscribing to navigator.windowControlsOverlay.ongeometrychange or by setting up an event listener for the geometrychange event. This event will only fire when the window controls overlay is visible, that is, when navigator.windowControlsOverlay.visible is true.

const debounce = (func, wait) => {
  let timeout;
  return function executedFunction(...args) {
    const later = () => {
      clearTimeout(timeout);
      func(...args);
    };
    clearTimeout(timeout);
    timeout = setTimeout(later, wait);
  };
};

if ('windowControlsOverlay' in navigator) {
  navigator.windowControlsOverlay.ongeometrychange = debounce((e) => {
    span.hidden = e.titlebarAreaRect.width < 800;
  }, 250);
}

Rather than assigning a function to ongeometrychange, you can also add an event listener to windowControlsOverlay as below. You can read about the difference between the two on MDN.

navigator.windowControlsOverlay.addEventListener(
  'geometrychange',
  debounce((e) => {
    span.hidden = e.titlebarAreaRect.width < 800;
  }, 250),
);

Compatibility when running in a tab and on non-supporting browsers #

There are two possible cases to consider:

The case where an app is running in a browser that supports Window Controls Overlay, but where the app is used in a browser tab.
The case where an app is running in a browser that does not support Window Controls Overlay.

In both cases, by default the HTML built for the window controls overlay will display inline like regular HTML content and the env() variables' fallback values will kick in for the positioning. On supporting browsers, you can also decide to not display the HTML designated for the window controls overlay by checking the overlay's visible property, and if it reports false, then hiding that HTML content.

Controls meant for the title bar can be easily displayed in the body on older browsers.

As a reminder, non-supporting browsers will either not consider the "display_override" web app manifest property at all, or not recognize the "window-controls-overlay" and thus use the next possible value according to the fallback chain, for example, "standalone".

Controls meant for the title bar can be easily displayed in the body on older browsers.

UI considerations #

While it may be tempting, creating a classic dropdown menu in the Window Controls Overlay area is not recommended. Doing so would violate the design guidelines on macOS, a platform on which users expect menu bars (both system-provided ones and custom ones) at the top of the screen.

If your app provides a fullscreen experience, carefully consider whether it makes sense for your Window Controls Overlay to be part of the fullscreen view. Potentially you may want to rearrange your layout when the onfullscreenchange event fires.

Demo #

I have created a demo that you can play with in different supporting and non-supporting browsers and in the installed and non-installed state. For the actual Window Controls Overlay experience, you need to install the app and set the flag. You can see two screenshots of what to expect below. The source code for the app is available on Glitch.

The demo app is available for experimentation.

The search feature in the window controls overlay is fully functional:

A search feature using the Window Controls Overlay.

Security considerations #

The Chromium team designed and implemented the Window Controls Overlay API using the core principles defined in Controlling Access to Powerful Web Platform Features, including user control, transparency, and ergonomics.

Spoofing #

Giving sites partial control of the title bar leaves room for developers to spoof content in what was previously a trusted, browser-controlled region. Currently, in Chromium browsers, standalone mode includes a title bar which on initial launch displays the title of the webpage on the left, and the origin of the page on the right (followed by the "settings and more" button and the window controls). After a few seconds, the origin text disappears. If the browser is set to a right-to-left (RTL) language, this layout is flipped such that the origin text is on the left. This opens the window controls overlay to spoof the origin if there is insufficient padding between the origin and the right edge of the overlay. For example, the origin "evil.ltd" could be appended with a trusted site "google.com", leading users to believe that the source is trustworthy. The plan is to keep this origin text so that users know what the origin of the app is and can ensure that it matches their expectations. For RTL configured browsers, there must be enough padding to the right of the origin text to prevent a malicious website from appending the unsafe origin with a trusted origin.

Fingerprinting #

Enabling the window controls overlay and draggable regions do not pose considerable privacy concerns other than feature detection. However, due to differing sizes and positions of the window control buttons across operating systems, the navigator.windowControlsOverlay.getTitlebarAreaRect() method returns a DOMRect whose position and dimensions reveal information about the operating system upon which the browser is running. Currently, developers can already discover the OS from the user agent string, but due to fingerprinting concerns, there is discussion about freezing the UA string and unifying OS versions. There is an ongoing effort within the browser community to understand how frequently the size of the window controls overlay changes across platforms, as the current assumption is that these are fairly stable across OS versions and thus would not be useful for observing minor OS versions. Although this is a potential fingerprinting issue, it only applies to installed PWAs that use the custom title bar feature and does not apply to general browser usage. Additionally, the navigator.windowControlsOverlay API will not be available to iframes embedded inside of a PWA.

Navigating to a different origin within a PWA will cause it to fall back to the normal standalone title bar, even if it meets the above criteria and is launched with the window controls overlay. This is to accommodate the black bar that appears on navigation to a different origin. After navigating back to the original origin, the window controls overlay will be used again.

A black bar is shown when the user navigates to a different origin.

Feedback #

The Chromium team wants to hear about your experiences with the Window Controls Overlay API.

Tell us about the API design #

Is there something about the API that doesn't work like you expected? Or are there missing methods or properties that you need to implement your idea? Have a question or comment on the security model? File a spec issue on the corresponding GitHub repo, or add your thoughts to an existing issue.

Report a problem with the implementation #

Did you find a bug with Chromium's implementation? Or is the implementation different from the spec? File a bug at new.crbug.com. Be sure to include as much detail as you can, simple instructions for reproducing, and enter UI>Browser>WebAppInstalls in the Components box. Glitch works great for sharing quick and easy repros.

Show support for the API #

Are you planning to use the Window Controls Overlay API? Your public support helps the Chromium team to prioritize features and shows other browser vendors how critical it is to support them.

Send a Tweet to @ChromiumDev with the #WindowControlsOverlay hashtag and let us know where and how you're using it.

Helpful links #

Acknowledgements #

Window Controls Overlay was implemented and specified by Amanda Baker from the Microsoft Edge team. This article was reviewed by Joe Medley and Kenneth Rohde Christiansen. Hero image by Sigmund on Unsplash.

Breaking down barriers using the DataTransfer API

2021-04-21T00:00:00Z

You might have heard about the DataTransfer API, which is part of the HTML5 Drag and Drop API and Clipboard events. It can be used to transfer data between source and receiving targets.

Browser support

Chrome 3, Supported 3
Firefox 3.5, Supported 3.5
Edge 12, Supported 12
Safari 4, Supported 4

Source

The drag-drop and copy-paste interactions are often used for interactions within a page to transfer simple text from A to B. But what is oftentimes overlooked is the ability to use these same interactions to go beyond the browser window.

Both the browser's built-in drag-and-drop and the copy-paste interactions can communicate with other applications, web or otherwise, and are not tied to any origin. The API supports multiple data entries with different behaviors based on where data is transferred to. Your web application can send and receive the transferred data when listening to incoming events.

This capability can change the way we think about sharing and interoperability in web applications on desktop. Transferring data between applications doesn't need to rely on tightly coupled integrations anymore. Instead you can give users full control to transfer data to wherever they like.

An example of interactions that are possible with the DataTransfer API. (Video does not include sound.)

Transferring data #

To get started, you'll need to implement drag-drop or copy-paste. The examples below show drag-drop interactions, but the process for copy-paste is similar. If you are unfamiliar with the Drag and Drop API, there's a great article explaining HTML5 Drag and Drop, which explains the ins and outs.

By providing MIME-type keyed data, you are able to freely interact with external applications. Most WYSIWYG editors, text editors, and browsers respond to the "primitive" mime-types used in the example below.

document.querySelector('#dragSource')
.addEventListener('dragstart', (event) => {
  event.dataTransfer.setData('text/plain', 'Foo bar');
  event.dataTransfer.setData('text/html', '<h1>Foo bar</h1>');
  event.dataTransfer.setData('text/uri-list', 'https://example.com');
});

Notice the event.dataTransfer property. This returns an instance of DataTransfer. As you'll see, this object is sometimes returned by properties with other names.

Receiving the data transfer works almost the same as providing it. Listen to the receiving events (drop, or paste) and read the keys. When dragging over an element, the browser only has access to the type keys of the data. The data itself can only be accessed after a drop.

document.querySelector('#dropTarget')
.addEventListener('dragover', (event) => {
  console.log(event.dataTransfer.types);
  // Without this, the drop event won't fire.
  event.preventDefault();
});

document.querySelector('#dropTarget')
.addEventListener('drop', (event) => {
  // Log all the transferred data items to the console.
  for (let type of event.dataTransfer.types) {
    console.log({ type, data: event.dataTransfer.getData(type) });
  }
  event.preventDefault();
});

Three MIME-types are widely supported across applications:

text/html: Renders the HTML payload in contentEditable elements and rich text (WYSIWYG) editors like Google Docs, Microsoft Word, and others.
text/plain: Sets the value of input elements, content of code editors, and the fallback from text/html.
text/uri-list: Navigates to the URL when dropping on the URL bar or browser page. A URL shortcut will be created when dropping on a directory or the desktop.

The widespread adoption of text/html by WYSIWYG editors makes it very useful. As in HTML documents, you can embed resources by using Data URLs or publicly accessible URLs. This works well with exporting visuals (for example from a canvas) to editors like Google Docs.

const redPixel = 'data:image/gif;base64,R0lGODdhAQABAPAAAP8AAAAAACwAAAAAAQABAAACAkQBADs=';
const html = '<img src="' + redPixel + '" width="100" height="100" alt="" />';
event.dataTransfer.setData('text/html', html);

Transfer using copy and paste #

Using the DataTransfer API with copy-paste interactions is shown below. Notice that the DataTransfer object is returned by a property called clipboardData for clipboard events.

// Listen to copy-paste events on the document.
document.addEventListener('copy', (event) => {
  const copySource = document.querySelector('#copySource');
  // Only copy when the `activeElement` (i.e., focused element) is,
  // or is within, the `copySource` element.
  if (copySource.contains(document.activeElement)) {
    event.clipboardData.setData('text/plain', 'Foo bar');
    event.preventDefault();
  }
});

document.addEventListener('paste', (event) => {
  const pasteTarget = document.querySelector('#pasteTarget');
  if (pasteTarget.contains(document.activeElement)) {
    const data = event.clipboardData.getData('text/plain');
    console.log(data);
  }
});

Custom data formats #

You are not limited to the primitive MIME types, but can use any key to identify the transferred data. This can be useful for cross-browser interactions within your application. As shown below, you can transfer more complex data using the JSON.stringify() and JSON.parse() functions.

document.querySelector('#dragSource')
.addEventListener('dragstart', (event) => {
  const data = { foo: 'bar' };
  event.dataTransfer.setData('my-custom-type', JSON.stringify(data));
});

document.querySelector('#dropTarget')
.addEventListener('dragover', (event) => {
  // Only allow dropping when our custom data is available.
  if (event.dataTransfer.types.includes('my-custom-type')) {
    event.preventDefault();
  }
});

document.querySelector('#dropTarget')
.addEventListener('drop', (event) => {
  if (event.dataTransfer.types.includes('my-custom-type')) {
    event.preventDefault();
    const dataString = event.dataTransfer.getData('my-custom-type');
    const data = JSON.parse(dataString);
    console.log(data);
  }
});

Connecting the web #

While custom formats are great for communication between applications you have in your control, it also limits the user when transferring data to applications that aren't using your format. If you want to connect with third-party applications across the web, you need a universal data format.

The JSON-LD (Linked Data) standard is a great candidate for this. It is lightweight and easy to read from and write to in JavaScript. Schema.org contains many predefined types that can be used, and custom schema definitions are an option as well.

const data = {
  '@context': 'https://schema.org',
  '@type': 'ImageObject',
  contentLocation: 'Venice, Italy',
  contentUrl: 'venice.jpg',
  datePublished: '2010-08-08',
  description: 'I took this picture during our honey moon.',
  name: 'Canal in Venice',
};
event.dataTransfer.setData('application/ld+json', JSON.stringify(data));

When using the Schema.org types, you can start with the generic Thing type, or use something closer to your use case like Event, Person, MediaObject, Place, or even highly-specific types like MedicalEntity if need be. When you use TypeScript, you can use the interface definitions from the schema-dts type definitions.

By transmitting and receiving JSON-LD data, you will support a more connected and open web. With applications speaking the same language, you can create deep integrations with external applications. There's no need for complicated API integrations; all the information that's needed is included in the transferred data.

Think of all the possibilities for transferring data between any (web) application with no restrictions: sharing events from a calendar to your favorite ToDo app, attaching virtual files to emails, sharing contacts. That would be great, right? This starts with you! 🙌

Concerns #

While the DataTransfer API is available today, there are some things to be aware of before integrating.

Browser compatibility #

Desktop browsers all have great support for the technique described above, while mobile devices do not. The technique has been tested on all major browsers (Chrome, Edge, Firefox, Safari) and operating systems (Android, ChromeOS, iOS, macOS, Ubuntu Linux, and Windows), but unfortunately Android and iOS didn't pass the test. While browsers continue to develop, for now the technique is limited to desktop browsers only.

Discoverability #

Drag-drop and copy-paste are system-level interactions when working on a desktop computer, with roots back to the first GUIs more than 40 years ago. Think about how many times you have used these interactions for organizing files. This isn't yet very common on the web.

You will need to educate users about this new interaction, and come up with UX patterns to make this recognizable, especially for people whose experience with computers so far has been confined to mobile devices.

Accessibility #

Drag-drop is not a very accessible interaction, but the DataTransfer API works with copy-paste, too. Make sure you listen to copy-paste events. It doesn't take much extra work, and your users will be grateful to you for adding it.

Security and privacy #

There are some security and privacy considerations you should be aware of when using the technique.

Clipboard data is available to other applications on the user's device.
Web applications you are dragging over have access to the type keys, not the data. The data only becomes available on drop or paste.
The received data should be treated like any other user input; sanitize and validate before using.

Getting started with the Transmat helper library #

Are you excited about using the DataTransfer API in your application? Consider taking a look at the Transmat library on GitHub. This open-source library aligns browser differences, provides JSON-LD utilities, contains an observer to respond to transfer events for highlighting drop-areas, and lets you integrate the data transfer operations among existing drag and drop implementations.

import { Transmat, TransmatObserver, addListeners } from 'transmat';

// Send data on drag/copy.
addListeners(myElement, 'transmit', (event) => {
  const transmat = new Transmat(event);
  transmat.setData({
    'text/plain': 'Foobar',
    'application/json': { foo: 'bar' },
  });
});

// Receive data on drop/paste.
addListeners(myElement, 'receive', (event) => {
  const transmat = new Transmat(event);
  if (transmat.hasType('application/json') && transmat.accept()) {
    const data = JSON.parse(transmat.getData('application/json'));
  }
});

// Observe transfer events and highlight drop areas.
const obs = new TransmatObserver((entries) => {
  for (const entry of entries) {
    const transmat = new Transmat(entry.event);
    if (transmat.hasMimeType('application/json')) {
      entry.target.classList.toggle('drag-over', entry.isTarget);
      entry.target.classList.toggle('drag-active', entry.isActive);
    }
  }
});
obs.observe(myElement);

Acknowledgements #

Hero image by Luba Ertel on Unsplash.

Mainline Menswear implements PWA and sees a 55% conversion rate uplift

2021-04-20T00:00:00Z

Mainline is an online clothing retailer that offers the biggest designer brand names in fashion. The UK-based company entrusts its team of in-house experts, blended strategically with key partners, to provide a frictionless shopping experience for all. With market presence in over 100 countries via seven custom-built territorial websites and an app, Mainline will continue to ensure the ecommerce offering is rivalling the competition.

Challenge #

Mainline Menswear's goal was to complement the current mobile optimized website with progressive features that would adhere to their 'mobile first' vision, focusing on mobile-friendly design and functionality with a growing smartphone market in mind.

Solution #

The objective was to build and launch a PWA that complemented the original mobile friendly version of the Mainline Menswear website, and then compare the stats to their hybrid mobile app, which is currently available on Android and iOS.

Once the app launched and was being used by a small section of Mainline Menswear users, they were able to determine the difference in key stats between PWA, app, and Web.

The approach Mainline took when converting their website to a PWA was to make sure that the framework they selected for their website (Nuxt.js, utilizing Vue.js) would be future-proof and enable them to take advantage of fast moving web technology.

Results #

139_%

More pages per session in PWA vs. web.

161_%

Longer session durations in PWA vs. web.

10_%

Lower bounce rate in PWA vs. web

12.5_%

Higher average order value in PWA vs. web

55_%

Higher conversion rate in PWA vs. web.

243_%

Higher revenue per session in PWA vs. web.

Technical deep dive #

Mainline Menswear is using the Nuxt.js framework to bundle and render their site, which is a single page application (SPA).

Generating a service worker file #

For generating the service worker, Mainline Menswear added configuration through a custom implementation of the nuxt/pwa Workbox module.

The reason they forked the nuxt/pwa module was to allow the team to add more customizations to the service worker file that they weren't able to or had issues with when using the standard version. One such optimization was around the offline functionality of the site like, for example, serving a default offline page and gathering analytics while offline.

Anatomy of the web app manifest #

The team generated a manifest with icons for different mobile app icon sizes and other web app details like name, description and theme_color:

{
  "name": "Mainline Menswear",
  "short_name": "MMW",
  "description": "Shop mens designer clothes with Mainline Menswear. Famous brands including Hugo Boss, Adidas, and Emporio Armani.",
  "icons": [
    {
      "src": "/_nuxt/icons/icon_512.c2336e.png",
      "sizes": "512x512",
      "type": "image/png"
    }
  ],
  "theme_color": "#107cbb"
}

The web app, once installed, can be launched from the home screen without the browser getting in the way. This is achieved by adding the display parameter in the web app manifest file:

{
  "display": "standalone"
}

Last but not the least, the company is now able to easily track how many users are visiting their web app from the home screen by simply appending a utm_source parameter in the start_url field of the manifest:

{
  "start_url": "/?utm_source=pwa"
}

Runtime caching for faster navigations #

Caching for web apps is a must for page speed optimization and for providing a better user experience for returning users.

For caching on the web, there are quite a few different approaches. The team is using a mix of the HTTP cache and the Cache API for caching assets on the client side.

The Cache API gives Mainline Menswear finer control over the cached assets, allowing them to apply complex strategies to each file type. While all this sounds complicated and hard to set up and maintain, Workbox provides them with an easy way of declaring such complex strategies and eases the pain of maintenance.

Caching CSS and JS #

For CSS and JS files, the team chose to cache them and serve them over the cache using the StaleWhileRevalidate Workbox strategy. This strategy allows them to serve all Nuxt CSS and JS files fast, which significantly increases their site's performance. At the same time, the files are being updated in the background to the latest version for the next visit:

/* sw.js */
workbox.routing.registerRoute(
  /\/_nuxt\/.*(?:js|css)$/,
  new workbox.strategies.StaleWhileRevalidate({
    cacheName: 'css_js',
  }),
  'GET',
);

Caching Google fonts #

The strategy for caching Google Fonts depends on two file types:

The stylesheet that contains the @font-face declarations.
The underlying font files (requested within the stylesheet mentioned above).

// Cache the Google Fonts stylesheets with a stale-while-revalidate strategy.
workbox.routing.registerRoute(
  /https:\/\/fonts\.googleapis\.com\/*/,
  new workbox.strategies.StaleWhileRevalidate({
    cacheName: 'google_fonts_stylesheets',
  }),
  'GET',
);

// Cache the underlying font files with a cache-first strategy for 1 year.
workbox.routing.registerRoute(
  /https:\/\/fonts\.gstatic\.com\/*/,
  new workbox.strategies.CacheFirst({
    cacheName: 'google_fonts_webfonts',
    plugins: [
      new workbox.cacheableResponse.CacheableResponsePlugin({
        statuses: [0, 200],
      }),
      new workbox.expiration.ExpirationPlugin({
        maxAgeSeconds: 60 * 60 * 24 * 365, // 1 year
        maxEntries: 30,
      }),
    ],
  }),
  'GET',
);

Caching images #

For images, Mainline Menswear decided to go with two strategies. The first strategy applies to all images coming from their CDN, which are usually product images. Their pages are image-heavy so they are conscious of not taking too much of their users' device storage. So through Workbox, they added a strategy that is caching images coming only from their CDN with a maximum of 60 images using the ExpirationPlugin.

The 61st (newest) image requested, replaces the 1st (oldest) image so that no more than 60 product images are cached at any point in time.

workbox.routing.registerRoute(
  ({ url, request }) =>
    url.origin === 'https://mainline-menswear-res.cloudinary.com' &&
    request.destination === 'image',
  new workbox.strategies.StaleWhileRevalidate({
    cacheName: 'product_images',
    plugins: [
      new workbox.expiration.ExpirationPlugin({
        // Only cache 60 images.
        maxEntries: 60,
        purgeOnQuotaError: true,
      }),
    ],
  }),
);

The second image strategy handles the rest of the images being requested by the origin. These images tend to be very few and small across the whole origin, but to be on the safe side, the number of these cached images is also limited to 60.

workbox.routing.registerRoute(
  /\.(?:png|gif|jpg|jpeg|svg|webp)$/,
  new workbox.strategies.StaleWhileRevalidate({
    cacheName: 'images',
    plugins: [
      new workbox.expiration.ExpirationPlugin({
        // Only cache 60 images.
        maxEntries: 60,
        purgeOnQuotaError: true,
      }),
    ],
  }),
);

Providing offline functionality #

The offline page is precached right after the service worker is installed and activated. They do this by creating a list of all offline dependencies: the offline HTML file and an offline SVG icon.

const OFFLINE_HTML = '/offline/offline.html';
const PRECACHE = [
  { url: OFFLINE_HTML, revision: '70f044fda3e9647a98f084763ae2c32a' },
  { url: '/offline/offline.svg', revision: 'efe016c546d7ba9f20aefc0afa9fc74a' },
];

The precache list is then fed into Workbox which takes care of all the heavy lifting of adding the URLs to the cache, checking for any revision mismatch, updating, and serving the precached files with a CacheFirst strategy.

workbox.precaching.precacheAndRoute(PRECACHE);

Handling offline navigations #

Once the service worker activates and the offline page is precached, it is then used to respond to offline navigation requests by the user. While Mainline Menswear's web app is an SPA, the offline page shows only after the page reloads, the user closes and reopens the browser tab, or when the web app is launched from the home screen while offline.

To achieve this, Mainline Menswear provided a fallback to failed NavigationRoute requests with the precached offline page:

const htmlHandler = new workbox.strategies.NetworkOnly();
const navigationRoute = new workbox.routing.NavigationRoute(({ event }) => {
    const request = event.request;
    // A NavigationRoute matches navigation requests in the browser, i.e. requests for HTML
    return htmlHandler.handle({ event, request }).catch(() => caches.match(OFFLINE_HTML, {
        ignoreSearch: true
    }));
});
workbox.routing.registerRoute(navigationRoute);

Demo #

Offline page example as seen on www.mainlinemenswear.co.uk.

Reporting successful installs #

Apart from the home screen launch tracking (with "start_url": "/?utm_source=pwa" in the web application manifest), the web app also reports successful app installs by listening to the appinstalled event on window:

window.addEventListener('appinstalled', (evt) => {
  ga('send', 'event', 'Install', 'Success');
});

Adding PWA capabilities to your website will further enhance your customers experience of shopping with you, and will be quicker to market than a [platform-specific] app. Andy Hoyle, Head of Development

Conclusion #

To learn more about progressive web apps and how to build them, head to the Progressive Web Apps section on web.dev.

To read more Progressive Web Apps case studies, browse to the case studies section.

What are mini apps?

2021-03-03T00:00:00Z

Building blocks and compatibility #

Mini apps are small (commonly 2-4 MB) apps that require a super app to run. What they have in common, independent of the super app, is that they are built with ("dialects" of) the web technologies HTML, CSS, and JavaScript. The runtime of a mini app is a WebView in the super app, not the underlying operating system, which makes mini apps cross platform. The same mini app can run in the same super app, no matter if the super app runs on Android, iOS, or another OS. However, not all mini apps can run in all super apps, more on this later.

Discovery #

Mini apps are often discovered ad-hoc via branded 2D barcodes, which solves an important offline-to-online challenge, for example, getting from a physical restaurant menu to a payment mini app, or from a physical e-scooter to a rental mini app. The image below shows an example of such a branded 2D barcode for WeChat's demo mini app. When the code is scanned with the WeChat super app, the mini app launches directly. Other super apps will typically not be able to recognize the barcode.

Scanning this 2d barcode with the WeChat app launches a demo mini app.

Mini apps can also be discovered through regular in-app search in the super app, be shared in chat messages, or be part of a news item in a news feed. Some super apps have the notion of verified accounts that can contain mini apps in their profiles. Mini apps can be highlighted when they are physically geographically close, like the mini app of a business in front of which the user stands, or virtually close, like when the user gets directions on a map shown in the super app. Frequently used mini apps are available in an app drawer that in many super apps can be accessed through a swipe down gesture or through a special section in the super app menu.

The user experience #

All super apps have more or less the same user interface for mini apps. A themeable top bar with the mini app's name, and, in the upper corner of the screen, a close button on the far right preceded by an action menu that provides access to common features like sharing the app, adding it to a favorites list or the home screen, reporting abusive apps, providing feedback, and settings. The screenshot below shows a shopping mini app running in the context of the Alipay super app with the action menu opened.

Opened action menu of a shopping mini app running in the Alipay super app.

UI paradigms #

Usually there is a bottom tab bar for the mini app's main navigation. Most super app providers offer components that help developers quickly implement common UI paradigms, such as carousels, accordions, progress bars, spinners, switches, maps, and so on. This also helps make the user experience between different mini apps consistent, which is encouraged by WeChat's Mini Program Design Guidelines. This is similar to what Apple incentivizes with its Apple Human Interface Guidelines, and Google with its Design for Android recommendations.

Douyin's slider (carousel) component with various options.

Serving #

Rather than being served piece by piece as separate resources, mini apps are served as encrypted packaged apps, that is, as archives that contain all resources in just one file. Unlike regular web apps, they are also not served from the particular origin of the mini app creator, but from the super app provider directly. They can still access APIs from the servers of the mini app creator, but the core resources (commonly referred to as the app shell), must be served from the super app provider. Mini apps have to declare the origins they request additional data from.

Caching, updates, and deep-linking #

Mini apps are kept in the cache of the super app, so the next time the user launches a cached mini app, it loads almost instantly. If there is an update, a new app package is loaded. The version number can be part of the launch URI (see Discovery), so the super app knows early on if the locally cached version is still current. The launch URI also optionally contains the desired page of the mini app, so deep-linking into specific pages of mini apps is possible. Via a sitemap, mini apps can declare which of their pages should be indexable by the super app provider's mini app crawler.

Mini apps are cached as encrypted packaged apps.

Security and permissions #

Mini apps are reviewed by the super app provider, which means users perceive them as more secure than web apps. They need to declare their potentially required permissions beforehand in a manifest or mini app configuration file, which, for some providers, also requires explanations for why each permission is needed. Mini apps can of course still lie, but they would have a hard time justifying why they are, for example, trying to access motion sensors without a reason that is apparent to the user. The incentive to fingerprint the user is notably a lot lower compared to the web, since the user is typically already logged in to the super app anyway (see Identity, payement, and social graph).

Whenever a mini app performs an operation that requires a special permission, a prompt is shown to the user that, if enforced by the platform, also includes the usage justification, as stated by the developer. The screenshot below shows the Douyin demo mini app as it asks the user for permission to share their location. In some super apps, there is also an imperative API that mini apps can leverage to request permissions without immediately using them, or to only check the status of a permission. This may even include an API to open the central super app permission settings, which corresponds to Chrome's Site Settings. Mini apps also have to declare beforehand the origins of all servers that they potentially will request data from.

The Douyin demo mini app asking for the geolocation permission.

Access to powerful features #

The hosting super app offers access to powerful APIs via a JavaScript bridge that gets injected into the WebView offered by the super app (see Building blocks and compatibility). This JavaScript bridge provides hooks into the operating system's APIs. For example, a mini app JavaScript function like getConnectedWifi()—the capability of a mini app to obtain the name of the currently active Wi-Fi network—under the hood is facilitated through Android's getConnectionInfo() API or iOS' CNCopyCurrentNetworkInfo() API. Other examples of powerful device APIs exposed in common super apps are Bluetooth, NFC, iBeacon, GPS, system clipboard, orientation sensors, battery information, calendar access, phonebook access, screen brightness control, file system access, vibration hardware for physical feedback, camera and microphone access, screen recording and screenshot creation, network status, UDP sockets, barcode scanning, device memory information, and more.

The WeChat demo mini app setting the device's screen brightness to the maximum.

Access to cloud services #

Many super apps also provide "serverless" access to cloud services of the super app provider that, apart from raw cloud computing and cloud storage, frequently also include higher-level tasks like text translation, object detection or content classification in images, speech recognition, or other Machine Learning tasks. Mini apps can be monetized with ads, which are commonly made available by super apps providers. Super app platforms usually also provide cloud analytics data, so mini app developers can better understand how users interact with their apps.

A very important feature of mini apps is the identity and social graph information that is shared from the super app. Super apps like Douyin or WeChat started as social networking sites in the broadest sense, where users have a (sometimes even government-verified) identity, a friend or follower network, and frequently also payment data stored. For example, a shopping mini app can (or sometimes even must) process any payments directly through the payment APIs of the super app and, upon user consent, can obtain user data like their shipping address, phone number, and full name, all without ever having to force the user to painfully fill out forms. Below, you can see the Walmart mini app running in WeChat, opened for the very first time, greeting me with a familiar face.

The Walmart mini app with a personalized "Me" view on the first visit.

Mini apps can get highly popular by letting people share their achievements like highscores in a game and challenge their contacts through status updates. The mini app is then only a tap away, so users can enter into competition without any friction and the mini app thereby grow its reach.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Other mini app runtime environments

2021-03-03T00:00:00Z

More than just on mobile #

In countries like China, mini apps have taken the market by storm. Apart from mobile devices, where mini apps are omnipresent and which are their natural habitat, mini apps have started to conquer other runtime environments like cars and the classic desktop.

Mini apps in cars #

In July 2020 the German car maker BMW Group announced a collaboration with Tencent branded as WeScenario, which, according to Tencent, will be rolled out to "30 leading auto companies in the world, and [bring the WeScenario] ecosystem of social, content and services to more than 110 mainstream automobile models".

Landing page of Tencent WeScenario (Source: BMW).

Mini apps on the desktop #

Mini apps in WeChat Desktop #

Using the WeChat desktop client available for macOS and Windows), it is possible to run WeChat mini apps on the desktop. (Make sure to not load the macOS version from the App Store if you are doing research and want the full experience, since it is more limited.)

To test it on macOS, share a mini app from a mobile device with yourself via the "File Transfer" account. This will result in a message that you can then open on the desktop client. In most cases, the mini app will then be directly clickable and runnable. In other cases, you have to forward the chat history to yourself again from a mobile device.

Sharing a mini app with oneself in the WeChat macOS desktop client.

On Windows, the workaround to share mini apps with oneself is not necessary, since there is a dedicated mini apps panel that shows the user's recently used mini apps and also includes an app search where new mini apps can be discovered.

The mini app panel in the WeChat Windows client.

The mini app search in the WeChat Windows client.

WeChat mini apps on the desktop naturally integrate with the operating system. On both macOS and Windows, they get their own entry in the multitasking bar and have their own taskbar icon. While on macOS, there is an option to be kept in the Dock, the icon disappears the moment the WeChat client app gets closed. On Windows, mini app icons can be pinned to the taskbar, but cannot be launched. On macOS, the title of the app is always "WeChat" and not the actual title of the app, whereas the title is displayed correctly on Windows.

The Starbucks app is a mini app and can be multitasked to like any regular macOS app.

Mini apps on macOS have WeChat as their title.

Most mini apps are not optimized for desktop yet and run in a fixed, non-resizable window that includes the well-known UI affordances and permission prompts as on mobile (see The user experience).

The Starbucks mini app running on macOS asking for the user profile permission.

The Starbucks mini app running on macOS in a fixed, non-resizable window.

Responsive mini apps that are optimized for the desktop (apart from for mobile) can be displayed in a wider window that on macOS is currently still fixed, but that on Windows is flexibly resizable.

The WeChat components demo app in a responsive app window.

The WeChat components demo app in a narrow app window.

The WeChat components demo app in a wide app window.

Mini app permission settings on macOS can be changed via the context menu. On Windows, this does not seem to be possible and the location reported by the demo app appears to be the coarse location that Windows allows apps to obtain without asking for permission.

WeChat mini app settings on macOS.

Mini apps in the 360 Secure Browser #

360 Secure Browser (360 安全浏览器) is a web browser developed by the company Qihoo. Apart from iOS and Android, the browser is also available for Windows, macOS, and Linux. On Windows, it is capable of running special 360 mini apps. The developer documentation and the API are comparable to those of other vendors; however, 360 does not offer dedicated DevTools. Instead, developers need to create their mini apps in an IDE of their own choosing and can then test them in the browser using a special development mode. Debugging happens through Chrome Dev Tools. A demo app is available for getting started.

Debugging a 360 mini app using Chrome Dev Tools.

360 mini apps can run in fullscreen mode and do appear as separate entries in the multitasking bar. Via the context menu, a home screen icon can be added that allows for mini apps to be launched from the desktop.

360 mini app running in fullscreen mode.

Web-based mini apps #

There are some mini app platforms that are web-based, but that depend on the presence of a special WebView to unlock their full potential.

LINE #

LINE is an app for instant communications on electronic devices such as smartphones, tablet computers, and personal computers. In addition, LINE is a platform providing various services including digital wallet, news stream, video on demand, and digital comic distribution. The service is a subsidiary of Korean internet search engine company, Naver Corporation.

Since LINE mini apps is essentially just a regular web app (see sample app) that pulls in the LINE Front-end Framework (LIFF), it can also be accessed outside of the main LINE app through special permanent links (example). However, not all APIs are available in such cases. Examples of not available in the browser APIs include the liff.scanCode() method for reading QR codes or Bluetooth-related APIs like liff.bluetooth.getAvailability(). To get a feel for the platform, you can test the LINE Playground app in the browser and the LINE app if you have a LINE account.

The LINE Playground demo app running on an iOS device.

The LINE Playground demo app running in the web browser.

Google Spot #

The Google Spot Platform allows developers to set up a Spot on Google Pay—a digital storefront that they can create, brand, and host however they choose. It is discoverable both online as well as through physical barcodes. Users can easily share a "Spot" (as the app calls it) on their favorite messaging app or find it on Google Pay. A Spot is built using HTML and JavaScript, so existing investments into mobile websites or PWAs can be easily transformed into a Spot by "adding a few lines of JavaScript" according to the announcement post.

The Eat.fit mini app running in the Google Pay super app (Source: Google).

Snap Minis #

Snap Inc. is an American camera and social media company most known for its chat app Snapchat. Snap has announced Snap Minis, bite-size utilities made for friends. They are built with HTML5, so they are easy to develop. Plus, they work for all Snapchatters, on any kind of device, with no installation required.

The Atom Tickets mini app running in Snapchat (Source: Snap).

VK Mini Apps #

The Russian social networking platform VK runs a mini app program that allows developers to build mini apps that tie in deeply with the social network. VK mini apps work on both VK's platform-specific mobile apps, as well as on the desktop website. Apart from several of the brand's other platforms like Mail.ru, VK mini apps are also integrated in the Atom browser.

The Все аптеки mini app running in VK (Source: VK).

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Concluding thoughts about mini apps from a web developer

2021-03-03T00:00:00Z

Where does this leave us? #

Writing and researching mini apps has been quite a ride, but one that I do not regret. On the one hand, the success and the popularity of mini apps have proven their creators right about their approach. On the other hand, though, this success is geographically concentrated in regions where the few popular super apps are dominant, at least at the time of writing. What is undoubtedly true is that the ecosystem is highly fascinating and well worth a look. This collection of articles has provided deep-dives into many of the aspects that make a difference when using and creating mini apps. From the DevTools experience to the mark-up, styling, and scripting approaches, over to the component model, and finally to the overall architecture; mini apps provide learning and inspiration opportunities for app developers, and even so for those who purely aim for the web.

My initial experiments with building a web application the mini app way were successful. Future work will show to what extent this model is performant and flexible enough to cater for the many shapes that web apps can take. My current ad-hoc approach can be formalized by packaging up the relevant pieces of code in a dedicated library, mini-app.js if you will. What is interesting is that this kind of programming goes back all the way to frameset. Just that today it is about applications and not documents.

I see great potential for improvement with the entire web development experience by taking inspiration from the various mini apps DevTools. From the easy (remote) on-device testing feature to the packaging and building experience; the integration of the IDE with the DevTools environment offers a lot of starting points for making developers' lives easier.

Closing thoughts #

From a features point of view, the web is becoming more and more powerful with each release of essentially any browser. The ever-growing list of capabilities makes use cases possible on the web that were unthinkable a mere year ago. At the same time, the need for mini apps standardization shows that developers are not willing or able to build the same mini app for each super app. On the horizon maybe there is a desire for an abstraction layer on the browser level that allows for mini apps to run on the web, while noting that the web is not immune from fragmentation, especially when looking at different browser vendors and what they choose to implement and what not. Concluding, I am looking forward to seeing where all this is headed. Thinking outside of the box and taking input and inspiration from outside of one's own bubble can definitely help when building a better future on the web.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Mini app components

2021-03-03T00:00:00Z

Web components #

Web components started with the promise of letting developers piece them together and build great apps on top of them. Examples of such atomic components are GitHub's time-elements, Stefan Judis' web-vitals-element, or, shameless plug, Google's dark mode toggle. When it comes to complete design systems, however, I have observed that people prefer to rely on a coherent set of components from the same vendor. An incomplete list of examples includes SAP's UI5 Web Components, the Polymer Elements, Vaadin's elements, Microsoft's FAST, the Material Web Components, arguably the AMP components, and many more. Due to a number of factors out of scope for this article, a lot of developers, however, have also flocked to frameworks like React, Vue.js, Ember.js, etc. Rather than giving the developer the freedom to choose from any of these options (or, dependent on your viewpoint, forcing them to make a technology choice), super app providers universally supply a set of components that developers must use.

Components in mini apps #

You can think of these components like any of the component libraries mentioned above. To get an overview of the available components, you can browse WeChat's component library, ByteDance's components, Alipay's components, Baidu's, and Quick App components.

Earlier I showed that while, for example, WeChat's <image> is a web component under the hood, not all of these components are technically web components. Some components, like <map> and <video>, are rendered as OS-built-in components that get layered over the WebView. For the developer, this implementation detail is not revealed, they are programmed like any other component.

As always, the details vary, but the overall programming concepts are the same across all super app providers. An important concept is data binding, as shown before in Markup languages. Generally, components are grouped by function, so finding the right one for the job is easier. Below is an example from Alipay's categorization, which is similar to the component grouping of other vendors.

View containers
- view
- swiper
- scroll-view
- cover-view
- cover-image
- movable-view
- movable-area
Basic content
- text
- icon
- progress
- rich-text
Form components
- button
- form
- label
- input
- textarea
- radio
- radio-group
- checkbox
- checkbox-group
- switch
- slider
- picker-view
- picker
Navigation
- navigator
Media components
- image
- video
Canvas
- canvas
Map
- map
Open components
- web-view
- lifestyle
- contact-button
Accessibility
- aria-component

Below, you can see Alipay's <image> used in an a:for directive (see List rendering) that loops over an image data array provided in index.js.

/* index.js */
Page({
  data: {
    array: [
      {
        mode: "scaleToFill",
        text: "scaleToFill",
      },
      {
        mode: "aspectFit",
        text: "aspectFit",
      },
    ],
    src: "https://images.example.com/sample.png",
  },
  imageError(e) {
    console.log("image", e.detail.errMsg);
  },
  onTap(e) {
    console.log("image tap", e);
  },
  imageLoad(e) {
    console.log("image", e);
  },
});

<!-- index.axml -->
<view class="page">
  <view class="page-section" a:for="{{array}}" a:for-item="item">
    <view class="page-section-demo" onTap="onTap">
      <image
        class="image"
        mode="{{item.mode}}"
        onTap="onTap"
        onError="imageError"
        onLoad="imageLoad"
        src="{{src}}"
        lazy-load="true"
        default-source="https://images.example.com/loading.png"
      />
    </view>
  </view>
</view>

Note the data binding of the item.mode to the mode attribute, the src to the src attribute, and the three event handlers onTap, onError, and onLoad to the functions of the same name. As shown before, the <image> tag internally gets converted into a <div> with a placeholder of the image's final dimensions, optional lazy loading, a default source, etc.

The available configuration options of the component are all listed in the documentation. An embedded-in-the-docs component preview with simulator makes the code immediately tangible.

Alipay component documentation with embedded component preview.

Alipay component preview popped out into its own tab.

Each component also has a QR code that can be scanned with the Alipay app that opens the component example in a self-contained minimal example.

Preview of the Alipay <image> component on a real device after following a QR code link from the docs.

Developers can jump from the documentation straight into Alipay DevTools IDE by leveraging a proprietary URI scheme antdevtool-tiny://. This allows the documentation to link directly into a to-be-imported mini app project, so developers can get started with the component immediately.

Custom components #

Apart from using the vendor-provided components, developers can also create custom components. The concept exists for WeChat, ByteDance, Alipay, and Baidu, as well as Quick App. For example, a Baidu custom component consists of four files that must reside in the same folder: custom.swan, custom.css, custom.js, and custom.json.

The file custom.json denotes the folder contents as a custom component.

{
  "component": true
}

The file custom.swan contains the markup and custom.css the CSS.

<view class="name" bindtap="tap">{{name}} {{age}}</view>

.name {
  color: red;
}

The file custom.js contains the logic. The component lifecycle functions are attached(), detached(), created(), and ready(). The component can additionally also react on page lifecycle events, namely show() and hide().

Component({
  properties: {
    name: {
      type: String,
      value: "swan",
    },
  },
  data: {
    age: 1,
  },
  methods: {
    tap: function () {},
  },
  lifetimes: {
    attached: function () {},
    detached: function () {},
    created: function () {},
    ready: function () {},
  },
  pageLifetimes: {
    show: function () {},
    hide: function () {},
  },
});

The custom component can then be imported in index.json, the key of the import determines the name (here: "custom") that the custom component can then be used with in index.swan.

{
  "usingComponents": {
    "custom": "/components/custom/custom"
  }
}

<view>
  <custom name="swanapp"></custom>
</view>

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Mini app DevTools

2021-03-03T00:00:00Z

The developer experience #

Now that I have covered mini apps per se, I want to focus on the developer experience for the various super app platforms. Mini app development on all platforms happens in IDEs that are provided for free by the super app platforms. While there are more, I want to focus on the four most popular ones, and a fifth for Quick App for comparison.

Mini app IDEs #

Like the super apps, the majority of the IDEs are available only in Chinese. You actually want to make sure that you install the Chinese version and not a sometimes available English (or overseas) version, since it might not be up-to-date. If you are a macOS developer, be aware that not all IDEs are signed, which means macOS refuses to run the installer. You can, at your own risk, bypass this as outlined by Apple help.

Mini app starter projects #

To get started quickly with mini app development, all super app providers offer demo apps that can be downloaded and tested immediately, and that are sometimes also integrated in the "New Project" wizards of the various IDEs.

Development flow #

After launching the IDE and loading or creating a (demo) mini app, the first step is always to log in. Usually you just need to scan a QR code with the super app (where you are already logged in) that is generated by the IDE. Very rarely do you have to enter a password. Once you are logged in, the IDE knows your identity and lets you start programming, debugging, testing, and submitting your app for review. In the following, you can see screenshots of the five IDEs mentioned in the paragraph above.

WeChat DevTools with simulator, code editor, and debugger.

Alipay DevTools with code editor, simulator, and debugger.

Baidu DevTools with simulator, code editor, and debugger.

ByteDance DevTools with simulator, code editor, and debugger.

Quick App DevTools with code editor, simulator, and debugger.

As you can see, the fundamental components of all IDEs are very similar. You always have a code editor based on the Monaco Editor, the same project that also powers VS Code. In all IDEs, there is a debugger based on the Chrome DevTools frontend with some modifications, more on those later (see Debugger). The IDEs per se are implemented either as NW.js or as Electron apps, the simulators in the IDEs are realized as an NW.js <webview> tag or Electron <webview> tag, which in turn are based on a Chromium <webview> tag. If you are interested in the IDE internals, you can oftentimes simply inspect them with Chrome DevTools with the keyboard shortcut Control+Alt+I (or Command+Option+I on Mac).

Inspecting Baidu DevTools with Chrome DevTools reveals that the simulator is realized as an Electron <webview> tag.

Simulator and real device testing and debugging #

The simulator is comparable with what you might know from Chrome DevTools' device mode. You can simulate different Android and iOS devices, change the scale and device orientation, but also simulate various network states, memory pressure, a barcode reading event, unexpected termination, and dark mode.

While the built-in simulator suffices to get a rough feeling for how the app behaves, on-device testing, like with regular web apps, is irreplaceable. Testing an in-development mini app is just a QR code scan away. For example, in ByteDance DevTools, scanning a QR code dynamically generated by the IDE with a real device leads to a cloud-hosted version of the mini app that can then immediately be tested on the device. The way this works for ByteDance is that the URL behind the QR code (example) redirects to a hosted page (example), that contains links with special URI schemes like, for example, snssdk1128://, to preview the mini app on the various ByteDance super apps like Douyin or Toutiao (here is an example). Other super app providers do not go through an intermediate page, but open the preview directly.

ByteDance DevTools showing a QR code that the user can scan with the Douyin app for immediate on-device testing.

Intermediate ByteDance landing page for previewing a mini app (opened on a desktop browser to show the flow).

An even more compelling feature is cloud-based preview remote debugging. After simply scanning a special likewise IDE-generated QR code, the mini app opens on the physical device, with a Chrome DevTools window running on the computer for remote debugging.

Wirelessly remote-debugging a mini app on a real device with ByteDance DevTools.

Debugger #

Elements debugging #

The mini app debugging experience is very familiar to anyone who has ever worked with Chrome DevTools. There are some important differences, though, that make the workflow tailored to mini apps. Instead of the Chrome DevTools' Elements panel, mini app IDEs have a customized panel that is tailored to their particular dialect of HTML. For example, in the case of WeChat, the panel is called Wxml, which stands for WeiXin Markup Language. In Baidu DevTools, it's called Swan Element. ByteDance DevTools calls it Bxml. Alipay names it AXML, and Quick App references the panel simply as UX. I will dive into these markup languages later.

Inspecting an <image> element with WeChat DevTools.

Custom elements under the hood #

Inspecting the WebView on a real device via about://inspect/#devices reveals that WeChat DevTools was deliberately hiding the truth. Where WeChat DevTools showed an <image>, the actual thing I am looking at is a custom element called <wx-image> with a <div> as its only child. It is interesting to note that this custom element does not use Shadow DOM. More on these components later.

Inspecting an <image> element with WeChat DevTools reveals that it is actually a <wx-image> custom element.

CSS debugging #

Another difference is the new length unit rpx for responsive pixel in the various dialects of CSS (more on this unit later). WeChat DevTools uses device-independent CSS length units to make developing for different device sizes more intuitive.

Inspecting the padding specified in responsive pixels (200rpx 0) of a view with WeChat DevTools.

Performance auditing #

Performance is front and center for mini apps, so it is no surprise that WeChat DevTools and some other DevTools have an integrated Lighthouse-inspired auditing tool. The focus areas of the audits are Total, Performance, Experience, and Best Practice. The view of the IDE can be customized. In the screenshot below I have temporarily hidden the code editor to have more screen real estate for the audit tool.

The built-in Audit tool in WeChat DevTools showing Total, Performance, Experience, and Best Practice.

API mocking #

Rather than requiring the developer to set up a separate service, mocking API responses is directly part of WeChat DevTools. Via an easy-to-use interface the developer can set up API endpoints and the desired mock responses.

WeChat DevTools' integrated API response mocking feature.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Applying the mini app programming principles to an example project

2021-03-03T00:00:00Z

The app domain #

To show the mini app way of programming applied to a web app, I needed a small but complete enough app idea. High-intensity interval training (HIIT) is a cardiovascular exercise strategy of alternating sets of short periods of intense anaerobic exercise with less intense recovery periods. Many HIIT trainings use HIIT timers, for example, this 30 minute online session from The Body Coach TV YouTube channel.

Active period.

Resting period.

HIIT Time example app #

For this chapter, I have built a basic example of such a HIIT timer application aptly named "HIIT Time" that lets the user define and manage various timers, always consisting of a high and a low intensity interval, and then select one of them for a training session. It is a responsive app with a navbar, a tabbar, and three pages:

Workout: The active page during a workout. It lets the user select one of the timers and features three progress rings: the number of sets, the active period, and the resting period.
Timers: Manages existing timers and lets the user create new ones.
Preferences: Allows toggling sound effects and speech output and selecting language and theme.

The following screenshots give an impression of the application.

HIIT Time "Workout" tab in portrait mode.

HIIT Time "Workout" tab in landscape mode.

HIIT Time timer management.

App structure #

As outlined above, the app consists of a navbar, a tabbar, and three pages, arranged in a grid. Navbar and tabbar are realized as iframes with a <div> container in between them with three more iframes for the pages, out of which one is always visible and dependent on the active selection in the tabbar. A final iframe pointing to about:blank serves for dynamically created in-app pages, which are needed for modifying existing timers or creating new ones. I call this pattern multi-page single-page app (MPSPA).

The app consists of six iframes.

Components-based lit-html markup #

The structure of each page is realized as lit-html scaffold that gets dynamically evaluated at runtime. For a background on lit-html, it is an efficient, expressive, extensible HTML templating library for JavaScript. By using it directly in the HTML files, the mental programming model is directly output-oriented. As a programmer, you write a template of what the final output will look like, and lit-html then fills the gaps dynamically based on your data and hooks up the event listeners. The app makes use of third-party custom elements like Shoelace's <sl-progress-ring> or a self-implemented custom element called <human-duration>. Since custom elements have a declarative API (for example, the percentage attribute of the progress ring), they work well together with lit-html, as you can see in the listing below.

<div>
  <button class="start" @click="${eventHandlers.start}" type="button">
    ${strings.START}
  </button>
  <button class="pause" @click="${eventHandlers.pause}" type="button">
    ${strings.PAUSE}
  </button>
  <button class="reset" @click="${eventHandlers.reset}" type="button">
    ${strings.RESET}
  </button>
</div>

<div class="progress-rings">
  <sl-progress-ring
    class="sets"
    percentage="${Math.floor(data.sets/data.activeTimer.sets*100)}"
  >
    <div class="progress-ring-caption">
      <span>${strings.SETS}</span>
      <span>${data.sets}</span>
    </div>
  </sl-progress-ring>
</div>

Rendered section of the page corresponding to the mark-up above.

Programming model #

Each page has a corresponding Page class that fills the lit-html markup with life by providing implementations of the event handlers and providing the data for each page. This class also supports lifecycle methods like onShow(), onHide(), onLoad(), and onUnload(). Pages have access to a data store that serves for sharing optionally persisted per-page state and global state. All strings are centrally managed, so internationalization is built in. Routing is handled by the browser essentially for free, since all the app does is toggle iframe visibility and for dynamically created pages change the src attribute of the placeholder iframe. The example below shows the code for closing a dynamically created page.

import Page from '../page.js';

const page = new Page({
  eventHandlers: {
    back: (e) => {
      e.preventDefault();
      window.top.history.back();
    },
  },
});

Navigation happens from iframe to iframe.

Styling #

Styling of pages happens per-page in its own scoped CSS file. This means elements can usually just be directly addressed by their element names, since no clashes with other pages can occur. Global styles are added to each page, so central settings like the font-family or the box-sizing do not need to be declared repeatedly. This is also where the themes and dark mode options are defined. The listing below shows the rules for the Preferences page that lays out the various form elements on a grid.

main {
  max-width: 600px;
}

form {
  display: grid;
  grid-template-columns: auto 1fr;
  grid-gap: 0.5rem;
  margin-block-end: 1rem;
}

label {
  text-align: end;
  grid-column: 1 / 2;
}

input,
select {
  grid-column: 2 / 3;
}

Every page is its own world. Styling happens directly with the element names.

Screen wake lock #

During a workout, the screen should not turn off. On browsers that support it, HIIT Time realizes this through a screen wake lock. The snippet below shows how it is done.

if ('wakeLock' in navigator) {
  const requestWakeLock = async () => {
    try {
      page.shared.wakeLock = await navigator.wakeLock.request('screen');
      page.shared.wakeLock.addEventListener('release', () => {
        // Nothing.
      });
    } catch (err) {
      console.error(`${err.name}, ${err.message}`);
    }
  };
  // Request a screen wake lock…
  await requestWakeLock();
  // …and re-request it when the page becomes visible.
  document.addEventListener('visibilitychange', async () => {
    if (
      page.shared.wakeLock !== null &&
      document.visibilityState === 'visible'
    ) {
      await requestWakeLock();
    }
  });
}

Testing the application #

The HIIT Time application is available on GitHub. You can play with the demo in a new window, or right in the iframe embed below, which simulates a mobile device.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Mini app markup, styling, and scripting

2021-03-03T00:00:00Z

Markup languages #

As outlined before, rather than with plain HTML, mini apps are written with dialects of HTML. If you have ever dealt with Vue.js text interpolation and directives, you will feel immediately at home, but similar concepts existed way before that in XML Transformations (XSLT). Below, you can see code samples from WeChat's WXML, but the concept is the same for all mini apps platforms, namely Alipay's AXML, Baidu's Swan Element, ByteDance's TTML (despite the DevTools calling it Bxml), and Quick App's HTML. Just like with Vue.js, the underlying mini app programming concept is the model-view-viewmodel (MVVM).

Data binding #

Data binding corresponds to Vue.js' text interpolation.

<!-- wxml -->
<view>{{message}}</view>

// page.js
Page({
  data: {
    message: "Hello World!",
  },
});

List rendering #

List rendering works like Vue.js v-for directive.

<!-- wxml -->
<view wx:for="{{array}}">{{item}}</view>

// page.js
Page({
  data: {
    array: [1, 2, 3, 4, 5],
  },
});

Conditional rendering #

Conditional rendering works like Vue.js' v-if directive.

<!-- wxml -->
<view wx:if="{{view == 'one'}}">One</view>
<view wx:elif="{{view == 'two'}}">Two</view>
<view wx:else="{{view == 'three'}}">Three</view>

// page.js
Page({
  data: {
    view: "three",
  },
});

Templates #

Rather than requiring the imperative cloning of the content of an HTML template, WXML templates can be used declaratively via the is attribute that links to a template definition.

<!-- wxml -->
<template name="person">
  <view>
    First Name: {{firstName}}, Last Name: {{lastName}}
  </view>
</template>

<template is="person" data="{{...personA}}"></template>
<template is="person" data="{{...personB}}"></template>
<template is="person" data="{{...personC}}"></template>

// page.js
Page({
  data: {
    personA: { firstName: "Alice", lastName: "Foo" },
    personB: { firstName: "Bob", lastName: "Bar" },
    personC: { firstName: "Charly", lastName: "Baz" },
  },
});

Styling #

Styling happens with dialects of CSS. WeChat's is named WXSS. For Alipay, theirs is called ACSS, Baidu's simply CSS, and for ByteDance, their dialect is referred to as TTSS. What they have in common is that they extend CSS with responsive pixels. When writing regular CSS, developers need to convert all pixel units to adapt to different mobile device screens with different widths and pixel ratios. TTSS supports the rpx unit as its underlying layer, which means the mini app takes over the job from the developer and converts the units on their behalf, based on a specified screen width of 750rpx. For example, on a Pixel 3a phone with a screen width of 393px (and a device pixel ratio of 2.75), responsive 200rpx become 104px on the real device when inspected with Chrome DevTools (393px / 750rpx * 200rpx ≈ 104px). In Android, the same concept is called density-independent pixel.

Inspecting the actual padding on a Pixel 3a device with Chrome DevTools.

/* app.wxss */
.container {
  height: 100%;
  display: flex;
  flex-direction: column;
  align-items: center;
  justify-content: space-between;
  padding: 200rpx 0; /* ← responsive pixels */
  box-sizing: border-box;
}

Since components (see later) do not use shadow DOM, styles declared on a page reach into all components. The common stylesheet file organization is to have one root stylesheet for global styles, and individual per-page stylesheets specific to each page of the mini app. Styles can be imported with an @import rule that behaves like the @import CSS at-rule. Like in HTML, styles can also be declared inline, including dynamic text interpolation (see before).

<view style="color:{{color}};" />

Scripting #

Mini apps support a "safe subset" of JavaScript that includes support for modules using varying syntaxes that remind of CommonJS or RequireJS. JavaScript code cannot be executed via eval() and no functions can be created with new Function(). The scripting execution context is V8 or JavaScriptCore on devices, and V8 or NW.js in the simulator. Coding with ES6 or newer syntax is usually possible, since the particular DevTools automatically transpile the code to ES5 if the build target is an operating system with an older WebView implementation (see later). The documentation of the super app providers explicitly mentions that their scripting languages are not to be confused with and are distinct from JavaScript. This statement, however, refers mostly just to the way modules work, that is, that they do not support standard ES Modules yet.

As mentioned before, the mini app programming concept is the model-view-viewmodel (MVVM). The logic layer and the view layer run on different threads, which means the user interface does not get blocked by long-running operations. In web terms, you can think of scripts running in a Web Worker.

WeChat's scripting language is called WXS, Alipay's SJS, ByteDance's likewise SJS. Baidu speaks of JS when referencing theirs. These scripts need to be included using a special kind of tag, for example, <wxs> in WeChat. In contrast, Quick App uses regular <script> tags and the ES6 JS syntax.

<wxs module="m1">
  var msg = "hello world";
  module.exports.message = msg;
</wxs>

<view>{{m1.message}}</view>

Modules can also be loaded via a src attribute, or imported via require().

// /pages/tools.wxs
var foo = "'hello world' from tools.wxs";
var bar = function (d) {
  return d;
};
module.exports = {
  FOO: foo,
  bar: bar,
};
module.exports.msg = "some msg";

<!-- page/index/index.wxml -->
<wxs src="./../tools.wxs" module="tools" />
<view>{{tools.msg}}</view>
<view>{{tools.bar(tools.FOO)}}</view>

// /pages/logic.wxs
var tools = require("./tools.wxs");

console.log(tools.FOO);
console.log(tools.bar("logic.wxs"));
console.log(tools.msg);

JavaScript bridge API #

The JavaScript bridge that connects mini apps with the operating system makes it possible to use OS capabilities (see Access to powerful features. It also offers a number of convenience methods. For an overview, you can check out the different APIs of WeChat, Alipay, Baidu, ByteDance, and Quick App.

Feature detection is straightforward, since all platforms provide a (literally called like this) canIUse() method whose name seems inspired by the website caniuse.com. For example, ByteDance's tt.canIUse() allows for support checks for APIs, methods, parameters, options, components, and attributes.

// Testing if the `<swiper>` component is supported.
tt.canIUse("swiper");
// Testing if a particular field is supported.
tt.canIUse("request.success.data");

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Mini app open source projects

2021-03-03T00:00:00Z

kbone #

The kbone project (open source on GitHub) implements an adapter that simulates a browser environment in the adaptation layer, so that code written for the web can run without changes in a mini app. Several starter templates (among them Vue, React, and Preact) exist that make the onboarding experience for web developers coming from these frameworks easier.

A new project can be created with the kbone-cli tool. A wizard asks what framework to initiate the project with. The code snippet below shows the Preact demo. In the code snippet below, the mp command builds the mini app, the web command builds the web app, and build creates the production web app.

npx kbone-cli init my-app
cd my-app
npm run mp
npm run web
npm run build

The code snippet below shows a simple counter component that then gets isomorphically rendered in a mini app and a web app. The overhead of the mini app is significant, purely judging from the DOM structure.

import { h, Component } from "preact";
import "./index.css";

class Counter extends Component {
  state = { count: 1 };

  sub = () => {
    this.setState((prevState) => {
      return { count: --prevState.count };
    });
  };

  add = () => {
    this.setState((prevState) => {
      return { count: ++prevState.count };
    });
  };

  clickHandle = () => {
    if ("undefined" != typeof wx && wx.getSystemInfoSync) {
      wx.navigateTo({
        url: "../log/index?id=1",
      });
    } else {
      location.href = "log.html";
    }
  };

  render({}, { count }) {
    return (
      <div>
        <button onClick={this.sub}>-</button>
        <span>{count}</span>
        <button onClick={this.add}>+</button>
        <div onClick={this.clickHandle}>跳转</div>
      </div>
    );
  }
}

export default Counter;

The Preact kbone template demo app opened in WeChat DevTools. Note the complex DOM structure and how the plus and minus buttons are actually not <button> elements.

The Preact kbone template demo app opened in the web browser. Note the DOM structure.

kbone-ui #

The kbone-ui project (open source on GitHub) is a UI framework that facilitates both mini app development as well as Vue.js development with kbone. The kbone-ui components emulate the look and feel of WeChat's built-in mini app components (also see Components above). A demo that runs directly in the browser lets you explore the available components.

The kbone-ui demo showing form-related components.

WeUI #

WeUI is a set of basic style libraries consistent with WeChat's default visual experience. The official WeChat design team has tailored designs for WeChat internal web pages and WeChat mini apps to make users' perception of use more uniform. It includes components such as button, cell, dialog, progress, toast, article, actionsheet, and icon. There are different versions of WeUI available like weui-wxss for WeChat mini apps styled with WXSS (see Styling above), weui.js for web apps, and react-weui for WeChat React components.

The WeUI demo app showing switches.

Omi #

Omi is a self-proclaimed frontend cross-frameworks framework (open source on GitHub. It merges Web Components, JSX, Virtual DOM, functional style, observer or Proxy into one framework with tiny size and high performance. Its aim is to let developers write components once and use them everywhere, such as Omi, React, Preact, Vue.js, or Angular. Writing Omi components is very intuitive and free of almost all boilerplate.

import { render, WeElement, define } from "omi";

define("my-counter", class extends WeElement {
  data = {
    count: 1,
  };

  static css = `
    span{
        color: red;
    }`;

  sub = () => {
    this.data.count--;
    this.update();
  };

  add = () => {
    this.data.count++;
    this.update();
  };

  render() {
    return (
      <div>
        <button onClick={this.sub}>-</button>
        <span>{this.data.count}</span>
        <button onClick={this.add}>+</button>
      </div>
    );
  }
});

render(<my-counter />, "body");

Omiu #

Omiu is a cross framework UI component library (open source on GitHub) developed based on Omi, which outputs custom elements of standard web components.

The Omiu demo app showing switches.

WePY #

WePY is a framework that allows mini apps to support componentized development. Through pre-compilation, developers can choose their favorite development style to develop mini apps. The detailed optimization of the framework and the introduction of Promises and async functions all make the development of mini app projects easier and more efficient. At the same time, WePY is also a growing framework, which has largely absorbed some optimized frontend tools and framework design concepts and ideas, mostly from Vue.js.

<style lang="less">
  @color: #4d926f;
  .num {
    color: @color;
  }
</style>

<template>
  <div class="container">
    <div class="num" @tap="num++">{{num}}</div>
    <custom-component></custom-component>
    <vendor-component></vendor-component>
    <div>{{text}}</div>
    <input v-model="text" />
  </div>
</template>

<config>
  { usingComponents: { customComponent: '@/components/customComponent', vendorComponent:
  'module:vendorComponent' } }
</config>

<script>
  import wepy from "@wepy/core";

  wepy.page({
    data: {
      num: 0,
      text: "Hello World",
    },
  });
</script>

WePY "getting started" documentation.

vConsole #

The vConsole project provides a lightweight, extendable frontend developer tool for mobile web pages. It offers a DevTools-like debugger that can be injected directly into web apps and mini apps. A demo showcases the opportunities. The vConsole includes tabs for logs, system, network, elements, and storage.

vConsole demo app showing the elements explorer.

weweb #

The weweb project (open source on GitHub) is the underlying frontend framework of the Hera mini app framework that claims to be compatible with the syntax of WeChat mini apps, so you can write web applications in the way of mini apps. The documentation promises that if you already have a mini app, you can run it in the browser thanks to weweb. In my experiments, this did not work reliably for current mini apps, most probably because the project has not seen updates recently leading its compiler to miss changes in the WeChat platform.

Hera mini app framework documentation showing the list of supported WeChat APIs.

The weweb demo mini app compiled with weweb to run in the browser.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Programming the mini app way

2021-03-03T00:00:00Z

What has worked well for mini apps #

In this chapter, I want to look at lessons I learned from researching mini apps from a web developer's point of view, or answer the question what does it mean to develop the mini app way.

Components #

Rather than reinvent the wheel and make developers build yet another implementation of common UI paradigms like tabs, accordions, carousels, etc., mini apps just ship with a default selection of components that is extensible in case you need more. On the web, there are likewise many options, some of which I have listed in the chapter on mini app components. In an ideal world, component libraries on the web were built in a way that you could mix them freely. In practice, too many times, there is a certain lock-in regarding a design system you need to buy in to when you use a component, or the component library is distributed in a way that it is all or nothing, but no individual components can be easily added to a project. There are, however, atomic components that you can use in isolation, or libraries like generic-components that are unstyled on purpose. Finding an using those seems like a good idea.

Model-view-viewmodel #

The model–view–viewmodel (MVVM) architectural pattern—that facilitates the separation of the development of the graphical user interface (the view) via a markup language from the development of the back-end logic (the model)—means the view is not dependent on any specific model platform. While there are some documented disadvantages of the pattern, in general it works really well for applications of the complexity of mini apps. It can shine especially with rich templating libraries (see next chapter).

Page-wise thinking #

Debugging mini apps shows that they are essentially multi-page applications (MPA). This has many advantages, like, for example, it allows for trivial routing and conflict-free per-page styling. People have successfully applied MPA architectures to Progressive Web Apps. Thinking in pages also helps manage resources like each page's CSS and JavaScript files, and other assets like images and videos. Most importantly, building this way means you get route-based code splitting for free if you do not load anything else. In that case, each page by definition strictly only loads what it needs to function.

Build process #

Mini apps have no visible build process. On the web, modern build tools like Snowpack leverage JavaScript's built-in module system (known as ESM) to avoid unnecessary work and stay fast no matter how big a project grows. While it is early days for technologies like Web Bundles, it is something that can be easily added to the build process.

Powerful capabilities #

The web platform has gained many new capabilities recently. Access to devices via Bluetooth, USB, HID, serial, and NFC is all possible now. Where mini apps run in WebViews and depend on a JavaScript bridge, on the web, these powerful capabilities are available directly, so you do not program against an API provided by the JavaScript bridge, but against the browser API without an intermediate actor.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Project structure, lifecycle, and bundling

2021-03-03T00:00:00Z

Mini app project structure #

As before with the markup languages, the styling languages, and the components; with the mini app project structure, too, the details like the file extensions or the default names vary. The overall idea, though, is the same for all super app providers. The project structure always consists of:

A root file app.js that initializes the mini app.
A configuration file app.json that roughly corresponds to a web app manifest.
An optional common style sheet file app.css with shared default styles.
A project.config.json file that contains build information.

All the pages are stored in individual subfolders in a pages folder. Each page subfolder contains a CSS file, a JS file, an HTML file, and an optional configuration JSON file. All files must be named like their containing folder, apart from the file extensions. Like that, the mini app just needs a pointer to the directory in the app.json file (the manifest-like file), and can find all subresources dynamically. From the perspective of a web developer, mini apps are thus multi page apps.

├── app.js               # Initialization logic
├── app.json             # Common configuration
├── app.css              # Common style sheet
├── project.config.json  # Project configuration
└── pages                # List of pages
   ├── index               # Home page
   │   ├── index.css         # Page style sheet
   │   ├── index.js          # Page logic
   │   ├── index.json        # Page configuration
   │   └── index.html        # Page markup
   └── other               # Other page
       ├── other.css         # Page style sheet
       ├── other.js          # Page logic
       ├── other.json        # Page configuration
       └── other.html        # Page markup

Mini app lifecycle #

A mini app must be registered with the super app by calling the globally defined App() method. Referring to the project structure outlined before, this happens in app.js. The mini app lifecycle essentially consists of four events: launch, show, hide, and error. Handlers for these events can be passed to the App() method in the form of a configuration object, which can also contain a globalData property that holds data that should be globally available across all pages.

/* app.js */
App({
  onLaunch(options) {
    // Do something when the app is launched initially.
  },
  onShow(options) {
    // Do something when the app is shown.
  },
  onHide() {
    // Do something when the app is hidden.
  },
  onError(msg) {
    console.log(msg);
  },
  globalData: "I am global data",
});

As usual, the individual details vary, but the concept is the same for WeChat, Alipay, Baidu, ByteDance, and also Quick App.

Page lifecycle #

Similar to the app lifecycle, the page lifecycle, too, has lifecycle events that the developer can listen for and react upon. These core events are load, show, ready, hide, and unload. Some platforms offer additional events like pulldownrefresh. Setting up the event handlers happens in the Page() method that is defined for each page. For the index or the other pages from the project structure before, this would happen in index.js or other.js respectively.

/* index.js */
Page({
  data: {
    text: "This is page data.",
  },
  onLoad: function (options) {
    // Do something when the page is initially loaded.
  },
  onShow: function () {
    // Do something when the page is shown.
  },
  onReady: function () {
    // Do something when the page is ready.
  },
  onHide: function () {
    // Do something when the page is hidden.
  },
  onUnload: function () {
    // Do something when the page is closed.
  },
  onPullDownRefresh: function () {
    // Do something when the user pulls down to refresh.
  },
  onReachBottom: function () {
    // Do something when the user scrolls to the bottom.
  },
  onShareAppMessage: function () {
    // Do something when the user shares the page.
  },
  onPageScroll: function () {
    // Do something when the user scrolls the page.
  },
  onResize: function () {
    // Do something when the user resizes the page.
  },
  onTabItemTap(item) {
    // Do something when the user taps the page's tab.
  },
  customData: {
    foo: "bar",
  },
});

The build process #

The build process of mini apps is abstracted away from the developer. Under the hood, it is using industry tools like the Babel compiler for transpilation and minification and the postcss CSS transformer. The build experience is comparable to that of Next.js or create-react-app, where developers, if they explicitly choose not to, never touch the build parameters. The resulting processed files are finally signed, encrypted, and packaged in one or several (sub)packages that then get uploaded to the servers of the super app providers. Subpackages are meant for lazy loading, so a mini app does not have to be downloaded all at once. The packaging details are meant to be private and are not documented, but some package formats like WeChat's wxapkg format have been reverse-engineered.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Mini app standardization

2021-03-03T00:00:00Z

Mini app popularity #

Mini apps have seen tremendous growth. WeChat mini apps as of June 2020 have reached 830 million active users, Alipay mini apps 401 million active users as of April 2019, and Baidu mini apps in the same month 115 million active users. Effectively companies have traded building apps for the two operating systems iOS and Android and additionally the web for building apps for three or more super apps platforms. The differences between each super app platform may not be as big as the differences between Android, iOS, and the web, but nevertheless they exist. Where on Android, iOS, and the web, we have seen cross-platform approaches like Flutter, Ionic, and React Native (for Web) gain popularity, in the mini apps ecosystem, we can see an effort led by the MiniApps Ecosystem Community Group with members from, among others, Alibaba, Baidu, ByteDance, Huawei, Intel, Xiaomi, China Mobile, Facebook, and Google to standardize aspects of mini apps.

Publications #

Notable publications of the group so far include a whitepaper, a Comparison of APIs in MiniApps, W3C specs, and PWAs, and specifications and explainers on the following aspects:

URI Scheme: spec, explainer
Lifecycle: spec, explainer
Manifest: spec, explainer
Packaging: spec, explainer
An exploration of widget requirements

W3C member and group participant Fuqiao Xue (W3C) has further published a Comparison of MiniApps and web apps on his own behalf, that is, not as an official group publication, but nonetheless worth the read.

The MiniApp Standardization White Paper.

Formal launch of the W3C MiniApps Working Group #

On January 19, 2021, the MiniApps Working Group was formally launched in the W3C. The group uses the spelling and capitalization "MiniApps" to distinguish the standardization effort from the technology. You can read the group's charter to get a feel for the planned work. Leaders of the group introduced the effort as follows:

Currently, there are many variants of MiniApps developed by different vendors with different APIs. To enhance the interoperability between MiniApp platforms, mainstream MiniApp vendors including Alibaba, Baidu, Huawei, and Xiaomi have been working together in the W3C Chinese Web Interest Group since May 2019 and published a MiniApp Standardization White Paper in September 2019 as the initial standardization exploration for MiniApp technologies. As more global companies get interested in joining the MiniApp related discussion, the MiniApps Ecosystem Community Group launched during TPAC 2019 so that the global Web community can join the discussion.

Based on extensive standardization requirements, W3C today announced the formal establishment of the MiniApps Working Group, dedicated to in-depth exploration and coordination of the diverse MiniApp ecosystem with W3C members and the public, and enhancing the interoperability of different MiniApp platforms, thereby maximizing the integration of MiniApps and the Web, reducing technical fragmentation and the learning cost of developers.

"Maximizing the integration of MiniApps and the Web" in particular sounds very interesting. As a curious member of the group, I look forward to seeing where this endeavor is headed.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Mini apps and super apps

2021-03-03T00:00:00Z

Welcome to mini apps #

When you look at applications on your phone, you probably have specific apps for specific tasks. You might have a banking app. You might have an app for buying public transit tickets. Likely you have an app for getting directions, and many more specialized apps. This post introduces you to the concept of a different kind of apps—mini apps—sometimes also called mini programs or applets. You will first learn about the background of various mini app platforms and their developer experience, and then focus on things the web can learn from mini apps. But before learning about mini apps, you first need to learn about super apps.

What are super apps? #

Super apps serve as hosts to other apps that run within them: the so-called mini apps. Popular super apps are WeChat (微信) by Tencent, Alipay (支付宝) by Ant Group (an affiliate company of the Chinese Alibaba Group), the app of the search engine Baidu (百度), as well as ByteDance's Douyin (抖音), which you might know as TikTok (蒂克托克). The first three are commonly also referred to as BAT, derived from B(aidu)A(libaba)T(encent). Super apps have taken the Chinese market by storm, which is why a lot of the examples in this article are Chinese.

The WeChat super app showing recently launched mini apps.

A few words about super app platforms #

WeChat aims to make itself a one-stop shop to meet almost any need users might have in their daily lives. Alipay builds its platforms on top of its payment system, focusing on retail and financial services, including credit, loan, insurance, installment, and local life services. Baidu strives to transform its search engine from solely connecting people, services, and information into information-as-a-service through mini programs for travel, retail, ads, payment, and more. Last but not least Douyin wants to boost itself as a hub for social e-commerce and transform to more of an entertainment and shopping platform.

Installing super apps #

Super apps are available on multiple operating systems. Note that the versions available in the official app stores may not always contain all features or be available in all locales. The links below point to links that work universally, but that may require loading from untrusted sources, so download and install the apps at your own risk. You typically need to create an account, which involves revealing your phone number. You might want to consider getting a burner phone. Be advised that many super apps only allow you to create a so-called overseas account, which does not have all features of a domestic account.

WeChat: iOS, Android, macOS, Windows
Baidu: iOS, Android
Alipay: iOS, Android
Douyin: iOS (CN-only), Android

Using Google Translate in camera mode to live-translate a Chinese mini app.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

What are H5 and QuickApp?

2021-03-03T00:00:00Z

What mini apps are not #

Before I go into more detail on the developer experience of mini apps, I want to briefly mention and set apart two technologies that come up in the context of mini apps, H5 and Quick App.

H5 #

H5 apps (or pages) are commonly seen as the predecessor of mini apps. What people mean by H5 is essentially a well-designed mobile web app (or page) that can be shared easily on chat applications. H5 is a reference to the HTML5 umbrella of technologies that includes responsive design, snappy CSS animations, multimedia content, etc. The Chinese Wikipedia actually redirects from H5 to HTML5. A good example of a representative H5 page experience is the WeChat H5 boilerplate project's demo.

Quick App #

Quick App is an industry alliance consisting of the following members:

While the technology of Quick App is comparable to "regular" mini apps (see Building blocks and compatibility), the discovery of Quick App is different. They are meant to be listed in stores, which come pre-installed on devices of the manufacturers in the alliance, but can also be shared by means of a deep link (see the Quick App showcase). They do not run in the context of a super app, but launch as seemingly self-contained full screen applications that feel deeply integrated into device. What happens in the background is that they are opened in a full screen view rendered by the operating system that provides the JavaScript bridge.

Acknowledgements #

This article was reviewed by Joe Medley, Kayce Basques, Milica Mihajlija, Alan Kent, and Keith Gu.

Tabbed application mode for PWAs

2021-02-25T00:00:00Z

In the world of computing, the desktop metaphor is an interface metaphor that is a set of unifying concepts used by graphical user interfaces (GUI) to help users interact more easily with the computer. In keeping with the desktop metaphor, GUI tabs are modeled after traditional card tabs inserted in books, paper files, or card indexes. A tabbed document interface (TDI) or tab is a graphical control element that allows multiple documents or panels to be contained within a single window, using tabs as a navigational widget for switching between sets of documents.

Progressive Web Apps can run in various display modes determined by the display property in the web app manifest. Examples are fullscreen, standalone, minimal-ui, and browser. These display modes follow a well-defined fallback chain ("fullscreen" → "standalone" → "minimal-ui" → "browser"). If a browser does not support a given mode, it falls back to the next display mode in the chain. Via the "display_override" property, developers can specify their own fallback chain if they need to.

What is tabbed application mode #

Something that has been missing from the platform so far is a way to let PWA developers offer their users a tabbed document interface, for example, to enable editing different files in the same PWA window. Tabbed application mode closes this gap.

Suggested use cases for tabbed application mode #

Examples of sites that may use tabbed application mode include:

Productivity apps that let the user edit more than one document (or file) at the same time.
Communication apps that let the user have conversations in different rooms per tab.
Reading apps that open article links in new in-app tabs.

Differences to developer-built tabs #

Having documents in separate browser tabs comes with resource isolation for free, which is not possible using the web today. Developer-built tabs would not scale acceptably to hundreds of tabs like browser tabs do. Browser affordances such as navigation history, "Copy this page URL", "Cast this tab" or "Open this page in a web browser" would be applied to the developer-built tabbed interface page, but not the currently selected document page.

Differences to `"display": "browser"` #

The current "display": "browser" already has a specific meaning:

Opens the web application using the platform-specific convention for opening hyperlinks in the user agent (e.g., in a browser tab or a new window).

While browsers can do whatever they want regarding UI, it would clearly be a pretty big subversion of developer expectations if "display": "browser" suddenly meant "run in a separate application-specific window with no browser affordances, but a tabbed document interface".

Setting "display": "browser" is effectively the way you opt out of being put into an application window.

Current status #

Step	Status
1. Create explainer	Completed
2. Create initial draft of specification	Not started
3. Gather feedback & iterate on design	In progress
4. Origin trial	Not started
5. Launch	Not started

Using tabbed application mode #

To use tabbed application mode, developers need to opt their apps in by setting a specific "display_override" mode value in the web app manifest.

{
  "display": "standalone",
  "display_override": ["tabbed"]
}

Next, the property "tab_strip" can optionally be used to fine-tune the tab behavior. It has two allowed sub-properties, "home_tab" and "new_tab_button". If the "tab_strip" property is not present, the particular properties' "auto" values are used. The browser determines what values to use for "auto".

Home tab #

The home tab is a pinned tab that, if enabled for an app, should always be present when the app is open. This tab should never navigate. Links clicked from this tab should open in a new app tab. Apps can choose to customize the URL this tab is locked to and the icon displayed on the tab.

The allowed values for "home_tab" are:

"auto" to let the browser determine what to do.
"absent" to tell the browser to not show a home tab.
An object with two sub-properties:
- "url" with the allowed values of "auto" or a URL to lock the home tab to.
- "icons" with the allowed values of "auto" or an array of icons as in the main "icons" property.

New tab button #

The new tab button, if present, should open a new tab at a URL that is within the scope of the app. The app may choose to set a custom URL and icon for this button. Browsers can decide how to handle dragging these tabs around to create new windows or combine with browser tabs.

The allowed values for "new_tab_button" are:

"auto" to let the browser determine what to do.
"absent" to tell the browser to not show a new tab button.
An object with two sub-properties:
- "url" with the allowed values of "auto" or an in-scope URL to open new tabs on.
- "icons" with the allowed values of "auto" or an array of icons as in the main "icons" property.

Complete example #

A complete example to configure the behavior of an app with a tabbed interface may look as follows:

{
  "display_override": ["tabbed"],
  "tab_strip": {
    "home_tab": {
      "url": "./home/",
      "icons": [
        {
          "src": "./home.svg",
          "sizes": "any",
          "type": "image/svg+xml"
        }
      ]
    },
    "new_tab_button": {
      "url": "./new-tab/",
      "icons": [
        {
          "src": "./new-tab.svg",
          "sizes": "any",
          "type": "image/svg+xml"
        }
      ]
    }
  }
}

Demo #

You can try tabbed application mode by setting a browser flag:

Set the #enable-desktop-pwas-tab-strip flag.
Install the app tabbed-application-mode.glitch.me (source code).
Click the different links on the different tabs.

Feedback #

The Chrome team wants to hear about your experiences with tabbed application mode.

Tell us about the API design #

Is there something about tabbed application mode that does not work like you expected? Comment on the web app manifest Issue that we have created.

Report a problem with the implementation #

Did you find a bug with Chrome's implementation? File a bug at new.crbug.com. Be sure to include as much detail as you can, simple instructions for reproducing, and enter UI>Browser>WebAppInstalls in the Components box. Glitch works great for sharing quick and easy reproduction cases.

Show support for the API #

Are you planning to use tabbed application mode? Your public support helps the Chrome team prioritize features and shows other browser vendors how critical it is to support them.

Send a tweet to @ChromiumDev using the hashtag #TabbedApplicationMode and let us know where and how you are using it.

Useful links #

Explainer
Web app manifest spec issue
Chromium bug
Blink Component: UI>Browser>WebAppInstalls

Acknowledgements #

Tabbed application mode was explored by Matt Giuca. The experimental implementation in Chrome was the work of Alan Cutter. This article was reviewed by Joe Medley. Hero image by Till Niermann on Wikimedia Commons.

Streams—The definitive guide

2021-02-19T00:00:00Z

The Streams API allows you to programmatically access streams of data received over the network or created by whatever means locally and process them with JavaScript. Streaming involves breaking down a resource that you want to receive, send, or transform into small chunks, and then processing these chunks bit by bit. While streaming is something browsers do anyway when receiving assets like HTML or videos to be shown on webpages, this capability has never been available to JavaScript before fetch with streams was introduced in 2015.

Previously, if you wanted to process a resource of some kind (be it a video, or a text file, etc.), you would have to download the entire file, wait for it to be deserialized into a suitable format, and then process it. With streams being available to JavaScript, this all changes. You can now process raw data with JavaScript progressively as soon as it is available on the client, without needing to generate a buffer, string, or blob. This unlocks a number of use cases, some of which I list below:

Video effects: piping a readable video stream through a transform stream that applies effects in real time.
Data (de)compression: piping a file stream through a transform stream that selectively (de)compresses it.
Image decoding: piping an HTTP response stream through a transform stream that decodes bytes into bitmap data, and then through another transform stream that translates bitmaps into PNGs. If installed inside the fetch handler of a service worker, this allows you to transparently polyfill new image formats like AVIF.

Browser support #

ReadableStream and WritableStream #

Browser support

Chrome 43, Supported 43
Firefox 65, Supported 65
Edge 14, Supported 14
Safari 10.1, Supported 10.1

Source

TransformStream #

Browser support

Chrome 67, Supported 67
Firefox 102, Supported 102
Edge 79, Supported 79
Safari 14.1, Supported 14.1

Source

Core concepts #

Before I go into details on the various types of streams, let me introduce some core concepts.

Chunks #

A chunk is a single piece of data that is written to or read from a stream. It can be of any type; streams can even contain chunks of different types. Most of the time, a chunk will not be the most atomic unit of data for a given stream. For example, a byte stream might contain chunks consisting of 16 KiB Uint8Array units, instead of single bytes.

Readable streams #

A readable stream represents a source of data from which you can read. In other words, data comes out of a readable stream. Concretely, a readable stream is an instance of the ReadableStream class.

Writable streams #

A writable stream represents a destination for data into which you can write. In other words, data goes in to a writable stream. Concretely, a writable stream is an instance of the WritableStream class.

Transform streams #

A transform stream consists of a pair of streams: a writable stream, known as its writable side, and a readable stream, known as its readable side. A real-world metaphor for this would be a simultaneous interpreter who translates from one language to another on-the-fly. In a manner specific to the transform stream, writing to the writable side results in new data being made available for reading from the readable side. Concretely, any object with a writable property and a readable property can serve as a transform stream. However, the standard TransformStream class makes it easier to create such a pair that is properly entangled.

Pipe chains #

Streams are primarily used by piping them to each other. A readable stream can be piped directly to a writable stream, using the readable stream's pipeTo() method, or it can be piped through one or more transform streams first, using the readable stream's pipeThrough() method. A set of streams piped together in this way is referred to as a pipe chain.

Backpressure #

Once a pipe chain is constructed, it will propagate signals regarding how fast chunks should flow through it. If any step in the chain cannot yet accept chunks, it propagates a signal backwards through the pipe chain, until eventually the original source is told to stop producing chunks so fast. This process of normalizing flow is called backpressure.

Teeing #

A readable stream can be teed (named after the shape of an uppercase 'T') using its tee() method. This will lock the stream, that is, make it no longer directly usable; however, it will create two new streams, called branches, which can be consumed independently. Teeing also is important because streams cannot be rewound or restarted, more about this later.

A pipe chain.

The mechanics of a readable stream #

A readable stream is a data source represented in JavaScript by a ReadableStream object that flows from an underlying source. The ReadableStream() constructor creates and returns a readable stream object from the given handlers. There are two types of underlying source:

Push sources constantly push data at you when you have accessed them, and it is up to you to start, pause, or cancel access to the stream. Examples include live video streams, server-sent events, or WebSockets.
Pull sources require you to explicitly request data from them once connected to. Examples include HTTP operations via fetch() or XMLHttpRequest calls.

Stream data is read sequentially in small pieces called chunks. The chunks placed in a stream are said to be enqueued. This means they are waiting in a queue ready to be read. An internal queue keeps track of the chunks that have not yet been read.

A queuing strategy is an object that determines how a stream should signal backpressure based on the state of its internal queue. The queuing strategy assigns a size to each chunk, and compares the total size of all chunks in the queue to a specified number, known as the high water mark.

The chunks inside the stream are read by a reader. This reader retrieves the data one chunk at a time, allowing you to do whatever kind of operation you want to do on it. The reader plus the other processing code that goes along with it is called a consumer.

The next construct in this context is called a controller. Each readable stream has an associated controller that, as the name suggests, allows you to control the stream.

Only one reader can read a stream at a time; when a reader is created and starts reading a stream (that is, becomes an active reader), it is locked to it. If you want another reader to take over reading your stream, you typically need to release the first reader before you do anything else (although you can tee streams).

Creating a readable stream #

You create a readable stream by calling its constructor ReadableStream(). The constructor has an optional argument underlyingSource, which represents an object with methods and properties that define how the constructed stream instance will behave.

The `underlyingSource` #

This can use the following optional, developer-defined methods:

start(controller): Called immediately when the object is constructed. The method can access the stream source, and do anything else required to set up the stream functionality. If this process is to be done asynchronously, the method can return a promise to signal success or failure. The controller parameter passed to this method is a ReadableStreamDefaultController.
pull(controller): Can be used to control the stream as more chunks are fetched. It is called repeatedly as long as the stream's internal queue of chunks is not full, up until the queue reaches its high water mark. If the result of calling pull() is a promise, pull() will not be called again until said promise fulfills. If the promise rejects, the stream will become errored.
cancel(reason): Called when the stream consumer cancels the stream.

const readableStream = new ReadableStream({
  start(controller) {
    /* … */
  },

  pull(controller) {
    /* … */
  },

  cancel(reason) {
    /* … */
  },
});

The ReadableStreamDefaultController supports the following methods:

ReadableStreamDefaultController.close() closes the associated stream.
ReadableStreamDefaultController.enqueue() enqueues a given chunk in the associated stream.
ReadableStreamDefaultController.error() causes any future interactions with the associated stream to error.

/* … */
start(controller) {
  controller.enqueue('The first chunk!');
},
/* … */

The `queuingStrategy` #

The second, likewise optional, argument of the ReadableStream() constructor is queuingStrategy. It is an object that optionally defines a queuing strategy for the stream, which takes two parameters:

highWaterMark: A non-negative number indicating the high water mark of the stream using this queuing strategy.
size(chunk): A function that computes and returns the finite non-negative size of the given chunk value. The result is used to determine backpressure, manifesting via the appropriate ReadableStreamDefaultController.desiredSize property. It also governs when the underlying source's pull() method is called.

const readableStream = new ReadableStream({
    /* … */
  },
  {
    highWaterMark: 10,
    size(chunk) {
      return chunk.length;
    },
  },
);

The `getReader()` and `read()` methods #

To read from a readable stream, you need a reader, which will be a ReadableStreamDefaultReader. The getReader() method of the ReadableStream interface creates a reader and locks the stream to it. While the stream is locked, no other reader can be acquired until this one is released.

The read() method of the ReadableStreamDefaultReader interface returns a promise providing access to the next chunk in the stream's internal queue. It fulfills or rejects with a result depending on the state of the stream. The different possibilities are as follows:

If a chunk is available, the promise will be fulfilled with an object of the form
{ value: chunk, done: false }.
If the stream becomes closed, the promise will be fulfilled with an object of the form
{ value: undefined, done: true }.
If the stream becomes errored, the promise will be rejected with the relevant error.

const reader = readableStream.getReader();
while (true) {
  const { done, value } = await reader.read();
  if (done) {
    console.log('The stream is done.');
    break;
  }
  console.log('Just read a chunk:', value);
}

The `locked` property #

You can check if a readable stream is locked by accessing its ReadableStream.locked property.

const locked = readableStream.locked;
console.log(`The stream is ${locked ? 'indeed' : 'not'} locked.`);

Readable stream code samples #

The code sample below shows all the steps in action. You first create a ReadableStream that in its underlyingSource argument (that is, the TimestampSource class) defines a start() method. This method tells the stream's controller to enqueue() a timestamp every second during ten seconds. Finally, it tells the controller to close() the stream. You consume this stream by creating a reader via the getReader() method and calling read() until the stream is done.

class TimestampSource {
  #interval

  start(controller) {
    this.#interval = setInterval(() => {
      const string = new Date().toLocaleTimeString();
      // Add the string to the stream.
      controller.enqueue(string);
      console.log(`Enqueued ${string}`);
    }, 1_000);

    setTimeout(() => {
      clearInterval(this.#interval);
      // Close the stream after 10s.
      controller.close();
    }, 10_000);
  }

  cancel() {
    // This is called if the reader cancels.
    clearInterval(this.#interval);
  }
}

const stream = new ReadableStream(new TimestampSource());

async function concatStringStream(stream) {
  let result = '';
  const reader = stream.getReader();
  while (true) {
    // The `read()` method returns a promise that
    // resolves when a value has been received.
    const { done, value } = await reader.read();
    // Result objects contain two properties:
    // `done`  - `true` if the stream has already given you all its data.
    // `value` - Some data. Always `undefined` when `done` is `true`.
    if (done) return result;
    result += value;
    console.log(`Read ${result.length} characters so far`);
    console.log(`Most recently read chunk: ${value}`);
  }
}
concatStringStream(stream).then((result) => console.log('Stream complete', result));

Asynchronous iteration #

Checking upon each read() loop iteration if the stream is done may not be the most convenient API. Luckily there will soon be a better way to do this: asynchronous iteration.

for await (const chunk of stream) {
  console.log(chunk);
}

A workaround to use asynchronous iteration today is to implement the behavior with a polyfill.

if (!ReadableStream.prototype[Symbol.asyncIterator]) {
  ReadableStream.prototype[Symbol.asyncIterator] = async function* () {
    const reader = this.getReader();
    try {
      while (true) {
        const {done, value} = await reader.read();
        if (done) {
          return;
          }
        yield value;
      }
    }
    finally {
      reader.releaseLock();
    }
  }
}

Teeing a readable stream #

The tee() method of the ReadableStream interface tees the current readable stream, returning a two-element array containing the two resulting branches as new ReadableStream instances. This allows two readers to read a stream simultaneously. You might do this, for example, in a service worker if you want to fetch a response from the server and stream it to the browser, but also stream it to the service worker cache. Since a response body cannot be consumed more than once, you need two copies to do this. To cancel the stream, you then need to cancel both resulting branches. Teeing a stream will generally lock it for the duration, preventing other readers from locking it.

const readableStream = new ReadableStream({
  start(controller) {
    // Called by constructor.
    console.log('[start]');
    controller.enqueue('a');
    controller.enqueue('b');
    controller.enqueue('c');
  },
  pull(controller) {
    // Called `read()` when the controller's queue is empty.
    console.log('[pull]');
    controller.enqueue('d');
    controller.close();
  },
  cancel(reason) {
    // Called when the stream is canceled.
    console.log('[cancel]', reason);
  },
});

// Create two `ReadableStream`s.
const [streamA, streamB] = readableStream.tee();

// Read streamA iteratively one by one. Typically, you
// would not do it this way, but you certainly can.
const readerA = streamA.getReader();
console.log('[A]', await readerA.read()); //=> {value: "a", done: false}
console.log('[A]', await readerA.read()); //=> {value: "b", done: false}
console.log('[A]', await readerA.read()); //=> {value: "c", done: false}
console.log('[A]', await readerA.read()); //=> {value: "d", done: false}
console.log('[A]', await readerA.read()); //=> {value: undefined, done: true}

// Read streamB in a loop. This is the more common way
// to read data from the stream.
const readerB = streamB.getReader();
while (true) {
  const result = await readerB.read();
  if (result.done) break;
  console.log('[B]', result);
}

Readable byte streams #

For streams representing bytes, an extended version of the readable stream is provided to handle bytes efficiently, in particular by minimizing copies. Byte streams allow for bring-your-own-buffer (BYOB) readers to be acquired. The default implementation can give a range of different outputs such as strings or array buffers in the case of WebSockets, whereas byte streams guarantee byte output. In addition, BYOB readers have stability benefits. This is because if a buffer detaches, it can guarantee that one does not write into the same buffer twice, hence avoiding race conditions. BYOB readers can reduce the number of times the browser needs to run garbage collection, because it can reuse buffers.

Creating a readable byte stream #

You can create a readable byte stream by passing an additional type parameter to the ReadableStream() constructor.

new ReadableStream({ type: 'bytes' });

The `underlyingSource` #

The underlying source of a readable byte stream is given a ReadableByteStreamController to manipulate. Its ReadableByteStreamController.enqueue() method takes a chunk argument whose value is an ArrayBufferView. The property ReadableByteStreamController.byobRequest returns the current BYOB pull request, or null if there is none. Finally, the ReadableByteStreamController.desiredSize property returns the desired size to fill the controlled stream's internal queue.

The `queuingStrategy` #

highWaterMark: A non-negative number of bytes indicating the high water mark of the stream using this queuing strategy. This is used to determine backpressure, manifesting via the appropriate ReadableByteStreamController.desiredSize property. It also governs when the underlying source's pull() method is called.

The `getReader()` and `read()` methods #

You can then get access to a ReadableStreamBYOBReader by setting the mode parameter accordingly: ReadableStream.getReader({ mode: "byob" }). This allows for more precise control over buffer allocation in order to avoid copies. To read from the byte stream, you need to call ReadableStreamBYOBReader.read(view), where view is an ArrayBufferView.

Readable byte stream code sample #

const reader = readableStream.getReader({ mode: "byob" });

let startingAB = new ArrayBuffer(1_024);
const buffer = await readInto(startingAB);
console.log("The first 1024 bytes, or less:", buffer);

async function readInto(buffer) {
  let offset = 0;

  while (offset < buffer.byteLength) {
    const { value: view, done } =
        await reader.read(new Uint8Array(buffer, offset, buffer.byteLength - offset));
    buffer = view.buffer;
    if (done) {
      break;
    }
    offset += view.byteLength;
  }

  return buffer;
}

The following function returns readable byte streams that allow for efficient zero-copy reading of a randomly generated array. Instead of using a predetermined chunk size of 1,024, it attempts to fill the developer-supplied buffer, allowing for full control.

const DEFAULT_CHUNK_SIZE = 1_024;

function makeReadableByteStream() {
  return new ReadableStream({
    type: 'bytes',

    pull(controller) {
      // Even when the consumer is using the default reader,
      // the auto-allocation feature allocates a buffer and
      // passes it to us via `byobRequest`.
      const view = controller.byobRequest.view;
      view = crypto.getRandomValues(view);
      controller.byobRequest.respond(view.byteLength);
    },

    autoAllocateChunkSize: DEFAULT_CHUNK_SIZE,
  });
}

The mechanics of a writable stream #

A writable stream is a destination into which you can write data, represented in JavaScript by a WritableStream object. This serves as an abstraction over the top of an underlying sink—a lower-level I/O sink into which raw data is written.

The data is written to the stream via a writer, one chunk at a time. A chunk can take a multitude of forms, just like the chunks in a reader. You can use whatever code you like to produce the chunks ready for writing; the writer plus the associated code is called a producer.

When a writer is created and starts writing to a stream (an active writer), it is said to be locked to it. Only one writer can write to a writable stream at one time. If you want another writer to start writing to your stream, you typically need to release it, before you then attach another writer to it.

An internal queue keeps track of the chunks that have been written to the stream but not yet been processed by the underlying sink.

The final construct is called a controller. Each writable stream has an associated controller that allows you to control the stream (for example, to abort it).

Creating a writable stream #

The WritableStream interface of the Streams API provides a standard abstraction for writing streaming data to a destination, known as a sink. This object comes with built-in backpressure and queuing. You create a writable stream by calling its constructor WritableStream(). It has an optional underlyingSink parameter, which represents an object with methods and properties that define how the constructed stream instance will behave.

The `underlyingSink` #

The underlyingSink can include the following optional, developer-defined methods. The controller parameter passed to some of the methods is a WritableStreamDefaultController.

start(controller): This method is called immediately when the object is constructed. The contents of this method should aim to get access to the underlying sink. If this process is to be done asynchronously, it can return a promise to signal success or failure.
write(chunk, controller): This method will be called when a new chunk of data (specified in the chunk parameter) is ready to be written to the underlying sink. It can return a promise to signal success or failure of the write operation. This method will be called only after previous writes have succeeded, and never after the stream is closed or aborted.
close(controller): This method will be called if the app signals that it has finished writing chunks to the stream. The contents should do whatever is necessary to finalize writes to the underlying sink, and release access to it. If this process is asynchronous, it can return a promise to signal success or failure. This method will be called only after all queued-up writes have succeeded.
abort(reason): This method will be called if the app signals that it wishes to abruptly close the stream and put it in an errored state. It can clean up any held resources, much like close(), but abort() will be called even if writes are queued up. Those chunks will be thrown away. If this process is asynchronous, it can return a promise to signal success or failure. The reason parameter contains a DOMString describing why the stream was aborted.

const writableStream = new WritableStream({
  start(controller) {
    /* … */
  },

  write(chunk, controller) {
    /* … */
  },

  close(controller) {
    /* … */
  },

  abort(reason) {
    /* … */
  },
});

The WritableStreamDefaultController interface of the Streams API represents a controller allowing control of a WritableStream's state during set up, as more chunks are submitted for writing, or at the end of writing. When constructing a WritableStream, the underlying sink is given a corresponding WritableStreamDefaultController instance to manipulate. The WritableStreamDefaultController has only one method: WritableStreamDefaultController.error(), which causes any future interactions with the associated stream to error. WritableStreamDefaultController also supports a signal property which returns an instance of AbortSignal, allowing a WritableStream operation to be stopped if needed.

/* … */
write(chunk, controller) {
  try {
    // Try to do something dangerous with `chunk`.
  } catch (error) {
    controller.error(error.message);
  }
},
/* … */

The `queuingStrategy` #

The second, likewise optional, argument of the WritableStream() constructor is queuingStrategy. It is an object that optionally defines a queuing strategy for the stream, which takes two parameters:

highWaterMark: A non-negative number indicating the high water mark of the stream using this queuing strategy.
size(chunk): A function that computes and returns the finite non-negative size of the given chunk value. The result is used to determine backpressure, manifesting via the appropriate WritableStreamDefaultWriter.desiredSize property.

The `getWriter()` and `write()` methods #

To write to a writable stream, you need a writer, which will be a WritableStreamDefaultWriter. The getWriter() method of the WritableStream interface returns a new instance of WritableStreamDefaultWriter and locks the stream to that instance. While the stream is locked, no other writer can be acquired until the current one is released.

The write() method of the WritableStreamDefaultWriter interface writes a passed chunk of data to a WritableStream and its underlying sink, then returns a promise that resolves to indicate the success or failure of the write operation. Note that what "success" means is up to the underlying sink; it might indicate that the chunk has been accepted, and not necessarily that it is safely saved to its ultimate destination.

const writer = writableStream.getWriter();
const resultPromise = writer.write('The first chunk!');

The `locked` property #

You can check if a writable stream is locked by accessing its WritableStream.locked property.

const locked = writableStream.locked;
console.log(`The stream is ${locked ? 'indeed' : 'not'} locked.`);

Writable stream code sample #

The code sample below shows all steps in action.

const writableStream = new WritableStream({
  start(controller) {
    console.log('[start]');
  },
  async write(chunk, controller) {
    console.log('[write]', chunk);
    // Wait for next write.
    await new Promise((resolve) => setTimeout(() => {
      document.body.textContent += chunk;
      resolve();
    }, 1_000));
  },
  close(controller) {
    console.log('[close]');
  },
  abort(reason) {
    console.log('[abort]', reason);
  },
});

const writer = writableStream.getWriter();
const start = Date.now();
for (const char of 'abcdefghijklmnopqrstuvwxyz') {
  // Wait to add to the write queue.
  await writer.ready;
  console.log('[ready]', Date.now() - start, 'ms');
  // The Promise is resolved after the write finishes.
  writer.write(char);
}
await writer.close();

Piping a readable stream to a writable stream #

A readable stream can be piped to a writable stream through the readable stream's pipeTo() method. ReadableStream.pipeTo() pipes the current ReadableStreamto a given WritableStream and returns a promise that fulfills when the piping process completes successfully, or rejects if any errors were encountered.

const readableStream = new ReadableStream({
  start(controller) {
    // Called by constructor.
    console.log('[start readable]');
    controller.enqueue('a');
    controller.enqueue('b');
    controller.enqueue('c');
  },
  pull(controller) {
    // Called when controller's queue is empty.
    console.log('[pull]');
    controller.enqueue('d');
    controller.close();
  },
  cancel(reason) {
    // Called when the stream is canceled.
    console.log('[cancel]', reason);
  },
});

const writableStream = new WritableStream({
  start(controller) {
    // Called by constructor
    console.log('[start writable]');
  },
  async write(chunk, controller) {
    // Called upon writer.write()
    console.log('[write]', chunk);
    // Wait for next write.
    await new Promise((resolve) => setTimeout(() => {
      document.body.textContent += chunk;
      resolve();
    }, 1_000));
  },
  close(controller) {
    console.log('[close]');
  },
  abort(reason) {
    console.log('[abort]', reason);
  },
});

await readableStream.pipeTo(writableStream);
console.log('[finished]');

Creating a transform stream #

The TransformStream interface of the Streams API represents a set of transformable data. You create a transform stream by calling its constructor TransformStream(), which creates and returns a transform stream object from the given handlers. The TransformStream() constructor accepts as its first argument an optional JavaScript object representing the transformer. Such objects can contain any of the following methods:

The `transformer` #

start(controller): This method is called immediately when the object is constructed. Typically this is used to enqueue prefix chunks, using controller.enqueue(). Those chunks will be read from the readable side but do not depend on any writes to the writable side. If this initial process is asynchronous, for example because it takes some effort to acquire the prefix chunks, the function can return a promise to signal success or failure; a rejected promise will error the stream. Any thrown exceptions will be re-thrown by the TransformStream() constructor.
transform(chunk, controller): This method is called when a new chunk originally written to the writable side is ready to be transformed. The stream implementation guarantees that this function will be called only after previous transforms have succeeded, and never before start() has completed or after flush() has been called. This function performs the actual transformation work of the transform stream. It can enqueue the results using controller.enqueue(). This permits a single chunk written to the writable side to result in zero or multiple chunks on the readable side, depending on how many times controller.enqueue() is called. If the process of transforming is asynchronous, this function can return a promise to signal success or failure of the transformation. A rejected promise will error both the readable and writable sides of the transform stream. If no transform() method is supplied, the identity transform is used, which enqueues chunks unchanged from the writable side to the readable side.
flush(controller): This method is called after all chunks written to the writable side have been transformed by successfully passing through transform(), and the writable side is about to be closed. Typically this is used to enqueue suffix chunks to the readable side, before that too becomes closed. If the flushing process is asynchronous, the function can return a promise to signal success or failure; the result will be communicated to the caller of stream.writable.write(). Additionally, a rejected promise will error both the readable and writable sides of the stream. Throwing an exception is treated the same as returning a rejected promise.

const transformStream = new TransformStream({
  start(controller) {
    /* … */
  },

  transform(chunk, controller) {
    /* … */
  },

  flush(controller) {
    /* … */
  },
});

The `writableStrategy` and `readableStrategy` queueing strategies #

The second and third optional parameters of the TransformStream() constructor are optional writableStrategy and readableStrategy queueing strategies. They are defined as outlined in the readable and the writable stream sections respectively.

Transform stream code sample #

The following code sample shows a simple transform stream in action.

// Note that `TextEncoderStream` and `TextDecoderStream` exist now.
// This example shows how you would have done it before.
const textEncoderStream = new TransformStream({
  transform(chunk, controller) {
    console.log('[transform]', chunk);
    controller.enqueue(new TextEncoder().encode(chunk));
  },
  flush(controller) {
    console.log('[flush]');
    controller.terminate();
  },
});

(async () => {
  const readStream = textEncoderStream.readable;
  const writeStream = textEncoderStream.writable;

  const writer = writeStream.getWriter();
  for (const char of 'abc') {
    writer.write(char);
  }
  writer.close();

  const reader = readStream.getReader();
  for (let result = await reader.read(); !result.done; result = await reader.read()) {
    console.log('[value]', result.value);
  }
})();

Piping a readable stream through a transform stream #

The pipeThrough() method of the ReadableStream interface provides a chainable way of piping the current stream through a transform stream or any other writable/readable pair. Piping a stream will generally lock it for the duration of the pipe, preventing other readers from locking it.

const transformStream = new TransformStream({
  transform(chunk, controller) {
    console.log('[transform]', chunk);
    controller.enqueue(new TextEncoder().encode(chunk));
  },
  flush(controller) {
    console.log('[flush]');
    controller.terminate();
  },
});

const readableStream = new ReadableStream({
  start(controller) {
    // called by constructor
    console.log('[start]');
    controller.enqueue('a');
    controller.enqueue('b');
    controller.enqueue('c');
  },
  pull(controller) {
    // called read when controller's queue is empty
    console.log('[pull]');
    controller.enqueue('d');
    controller.close(); // or controller.error();
  },
  cancel(reason) {
    // called when rs.cancel(reason)
    console.log('[cancel]', reason);
  },
});

(async () => {
  const reader = readableStream.pipeThrough(transformStream).getReader();
  for (let result = await reader.read(); !result.done; result = await reader.read()) {
    console.log('[value]', result.value);
  }
})();

The next code sample (a bit contrived) shows how you could implement a "shouting" version of fetch() that uppercases all text by consuming the returned response promise as a stream and uppercasing chunk by chunk. The advantage of this approach is that you do not need to wait for the whole document to be downloaded, which can make a huge difference when dealing with large files.

function upperCaseStream() {
  return new TransformStream({
    transform(chunk, controller) {
      controller.enqueue(chunk.toUpperCase());
    },
  });
}

function appendToDOMStream(el) {
  return new WritableStream({
    write(chunk) {
      el.append(chunk);
    }
  });
}

fetch('./lorem-ipsum.txt').then((response) =>
  response.body
    .pipeThrough(new TextDecoderStream())
    .pipeThrough(upperCaseStream())
    .pipeTo(appendToDOMStream(document.body))
);

Demo #

The demo below shows readable, writable, and transform streams in action. It also includes examples of pipeThrough() and pipeTo() pipe chains, and also demonstrates tee(). You can optionally run the demo in its own window or view the source code.

Useful streams available in the browser #

There are a number of useful streams built right into the browser. You can easily create a ReadableStream from a blob. The Blob interface's stream() method returns a ReadableStream which upon reading returns the data contained within the blob. Also recall that a File object is a specific kind of a Blob, and can be used in any context that a blob can.

const readableStream = new Blob(['hello world'], { type: 'text/plain' }).stream();

The streaming variants of TextDecoder.decode() and TextEncoder.encode() are called TextDecoderStream and TextEncoderStream respectively.

const response = await fetch('https://streams.spec.whatwg.org/');
const decodedStream = response.body.pipeThrough(new TextDecoderStream());

Compressing or decompressing a file is easy with the CompressionStream and DecompressionStream transform streams respectively. The code sample below shows how you can download the Streams spec, compress (gzip) it right in the browser, and write the compressed file directly to disk.

const response = await fetch('https://streams.spec.whatwg.org/');
const readableStream = response.body;
const compressedStream = readableStream.pipeThrough(new CompressionStream('gzip'));

const fileHandle = await showSaveFilePicker();
const writableStream = await fileHandle.createWritable();
compressedStream.pipeTo(writableStream);

The File System Access API's FileSystemWritableFileStream and the experimental fetch() request streams are examples of writable streams in the wild.

The Serial API makes heavy use of both readable and writable streams.

// Prompt user to select any serial port.
const port = await navigator.serial.requestPort();
// Wait for the serial port to open.
await port.open({ baudRate: 9_600 });
const reader = port.readable.getReader();

// Listen to data coming from the serial device.
while (true) {
  const { value, done } = await reader.read();
  if (done) {
    // Allow the serial port to be closed later.
    reader.releaseLock();
    break;
  }
  // value is a Uint8Array.
  console.log(value);
}

// Write to the serial port.
const writer = port.writable.getWriter();
const data = new Uint8Array([104, 101, 108, 108, 111]); // hello
await writer.write(data);
// Allow the serial port to be closed later.
writer.releaseLock();

Finally, the WebSocketStream API integrates streams with the WebSocket API.

const wss = new WebSocketStream(WSS_URL);
const { readable, writable } = await wss.connection;
const reader = readable.getReader();
const writer = writable.getWriter();

while (true) {
  const { value, done } = await reader.read();
  if (done) {
    break;
  }
  const result = await process(value);
  await writer.write(result);
}

Useful resources #

Acknowledgements #

This article was reviewed by Jake Archibald, François Beaufort, Sam Dutton, Mattias Buelens, Surma, Joe Medley, and Adam Rice. Jake Archibald's blog posts have helped me a lot in understanding streams. Some of the code samples are inspired by GitHub user @bellbind's explorations and parts of the prose build heavily on the MDN Web Docs on Streams. The Streams Standard's authors have done a tremendous job on writing this spec. Hero image by Ryan Lara on Unsplash.

Deprecating Excalidraw Electron in favor of the web version

2021-01-07T00:00:00Z

On the Excalidraw project, we have decided to deprecate Excalidraw Desktop, an Electron wrapper for Excalidraw, in favor of the web version that you can—and always could—find at excalidraw.com. After a careful analysis, we have decided that Progressive Web App (PWA) is the future we want to build upon. Read on to learn why.

How Excalidraw Desktop came into being #

Soon after @vjeux created the initial version of Excalidraw in January 2020 and blogged about it, he proposed the following in Issue #561:

Would be great to wrap Excalidraw within Electron (or equivalent) and publish it as a [platform-specific] application to the various app stores.

The immediate reaction by @voluntadpear was to suggest:

What about making it a PWA instead? Android currently supports adding them to the Play Store as Trusted Web Activities and hopefully iOS will do the same soon. On Desktop, Chrome lets you download a desktop shortcut to a PWA.

The decision that @vjeux took in the end was simple:

We should do both :)

While work on converting the version of Excalidraw into a PWA was started by @voluntadpear and later others, @lipis independently went ahead and created a separate repo for Excalidraw Desktop.

To this day, the initial goal set by @vjeux, that is, to submit Excalidraw to the various app stores, has not been reached yet. Honestly, no one has even started the submission process to any of the stores. But why is that? Before I answer, let's look at Electron, the platform.

What is Electron? #

The unique selling point of Electron is that it allows you to "build cross-platform desktop apps with JavaScript, HTML, and CSS". Apps built with Electron are "compatible with Mac, Windows, and Linux", that is, "Electron apps build and run on three platforms". According to the homepage, the hard parts that Electron makes easy are automatic updates, system-level menus and notifications, crash reporting, debugging and profiling, and Windows installers. Turns out, some of the promised features need a detailed look at the small print.

For example, automatic updates "are [currently] only [supported] on macOS and Windows. There is no built-in support for auto-updater on Linux, so it is recommended to use the distribution's package manager to update your app".
Developers can create system-level menus by calling Menu.setApplicationMenu(menu). On Windows and Linux, the menu will be set as each window's top menu, while on macOS there are many system-defined standard menus, like the Services menu. To make one's menus a standard menu, developers should set their menu's role accordingly, and Electron will recognize them and make them become standard menus. This means that a lot of menu-related code will use the following platform check: const isMac = process.platform === 'darwin'.
Windows installers can be made with windows-installer. The README of the project highlights that "for a production app you need to sign your application. Internet Explorer's SmartScreen filter will block your app from being downloaded, and many anti-virus vendors will consider your app as malware unless you obtain a valid cert" [sic].

Looking at just these three examples, it is clear that Electron is far from "write once, run everywhere". Distributing an app on app stores requires code signing, a security technology for certifying app ownership. Packaging an app requires using tools like electron-forge and thinking about where to host packages for app updates. It gets complex relatively quickly, especially when the objective truly is cross platform support. I want to note that it is absolutely possible to create stunning Electron apps with enough effort and dedication. For Excalidraw Desktop, we were not there.

Where Excalidraw Desktop left off #

Excalidraw Desktop so far is basically the Excalidraw web app bundled as an .asar file with an added About Excalidraw window. The look and feel of the application is almost identical to the web version.

Excalidraw Desktop is almost indistinguishable from the web version

The About Excalibur menu providing insights into the versions

On macOS, there is now a system-level menu at the top of the application, but since none of the menu actions—apart from Close Window and About Excalidraw—are hooked up to anything, the menu is, in its current state, pretty useless. Meanwhile, all actions can of course be performed via the regular Excalidraw toolbars and the context menu.

The menu bar of Excalidraw Desktop on macOS

We use electron-builder, which supports file type associations. By double-clicking an .excalidraw file, ideally the Excalidraw Desktop app should open. The relevant excerpt of our electron-builder.json file looks like this:

{
  "fileAssociations": [
    {
      "ext": "excalidraw",
      "name": "Excalidraw",
      "description": "Excalidraw file",
      "role": "Editor",
      "mimeType": "application/json"
    }
  ]
}

Unfortunately, in practice, this does not always work as intended, since, depending on the installation type (for the current user, for all users), apps on Windows 10 do not have the rights to associate a file type to themselves.

These shortcomings and the pending work to make the experience truly app-like on all platforms (which, again, with enough effort is possible) were a strong argument for us to reconsider our investment in Excalidraw Desktop. The way bigger argument for us, though, was that we foresee that for our use case, we do not need all the features Electron offers. The grown and still growing set of capabilities of the web serves us equally well, if not better.

How the web serves us today and in the future #

Even in 2020, jQuery is still incredibly popular. For many developers it has become a habit to use it, despite the fact that today they might not need jQuery. There is a similar resource for Electron, aptly called You Might Not Need Electron. Let me outline why we think we do not need Electron.

Installable Progressive Web App #

Excalidraw today is an installable Progressive Web App with a service worker and a web app manifest. It caches all its resources in two caches, one for fonts and font-related CSS, and one for everything else.

Excalidraw's cache contents

This means the application is fully offline-capable and can run without a network connection. Chromium-based browsers on both desktop and mobile prompt the user to install the app. You can see the installation prompt in the screenshot below.

The Excalidraw install dialog in Chrome

Excalidraw is configured to run as a standalone application, so when you install it, you get an app that runs in its own window. It is fully integrated in the operating system's multitasking UI and gets its own app icon on the home screen, Dock, or task bar; depending on the platform where you install it.

The Excalidraw PWA in a standalone window

The Excalidraw icon on the macOS Dock

File system access #

Excalidraw uses browser-fs-access for accessing the file system of the operating system. On supporting browsers, this allows for a true open→edit→save workflow and actual over-saving and "save as", with a transparent fallback for other browsers. You can learn more about this feature in my blog post Reading and writing files and directories with the browser-fs-access library.

Drag and drop support #

Files can be dragged and dropped onto the Excalidraw window just as in platform-specific applications. On a browser that supports the File System Access API, a dropped file can be immediately edited and the modifications be saved to the original file. This is so intuitive that you sometimes forget that you are dealing with a web app.

Clipboard access #

Excalidraw works well with the operating system's clipboard. Entire Excalidraw drawings or also just individual objects can be copied and pasted in image/png and image/svg+xml formats, allowing for an easy integration with other platform-specific tools like Inkscape or web-based tools like SVGOMG.

The Excalidraw context menu offering clipboard actions

File handling #

Excalidraw already supports the experimental File Handling API, which means .excalidraw files can be double-clicked in the operating system's file manager and open directly in the Excalidraw app, since Excalidraw registers as a file handler for .excalidraw files in the operating system.

Declarative link capturing #

Excalidraw drawings can be shared by link. Here is an example. In the future, if people have Excalidraw installed as a PWA, such links will not open in a browser tab, but launch a new standalone window. Pending implementation, this will work thanks to declarative link capturing, an, at the time of writing, bleeding-edge proposal for a new web platform feature.

Conclusion #

The web has come a long way, with more and more features landing in browsers that only a couple of years or even months ago were unthinkable on the web and exclusive to platform-specific applications. Excalidraw is at the forefront of what is possible in the browser, all while acknowledging that not all browsers on all platforms support each feature we use. By betting on a progressive enhancement strategy, we enjoy the latest and greatest wherever possible, but without leaving anyone behind. Best viewed in any browser.

Electron has served us well, but in 2020 and beyond, we can live without it. Oh, and for that objective of @vjeux: since the Android Play Store now accepts PWAs in a container format called Trusted Web Activity and since the Microsoft Store supports PWAs, too, you can expect Excalidraw in these stores in the not too distant future. Meanwhile, you can always use and install Excalidraw in and from the browser.

Acknowledgements #

This article was reviewed by @lipis, @dwelle, and Joe Medley.

Play the Chrome dino game with your gamepad

2020-11-03T00:00:00Z

Chrome's offline page easter egg is one of the worst-kept secrets in history ([citation needed], but claim made for the dramatic effect). If you press the space key or, on mobile devices, tap the dinosaur, the offline page becomes a playable arcade game. You might be aware that you do not actually have to go offline when you feel like playing: in Chrome, you can just navigate to about://dino, or, for the geek in you, browse to about://network-error/-106. But did you know that there are currently 270 million Chrome dino games played every month?

Press the space bar to play!

Another fact that arguably is more useful to know and that you might not be aware of is that in arcade mode you can play the game with a gamepad. Gamepad support was added roughly one year ago as of the time of this writing in a commit by Reilly Grant. As you can see, the game, just like the rest of the Chromium project, is fully open source. In this post, I want to show you how to use the Gamepad API.

Using the Gamepad API #

Feature detection and browser support #

The Gamepad API has universally great browser support across both desktop and mobile. You can detect if the Gamepad API is supported using the snippet below:

if ('getGamepads' in navigator) {
  // The API is supported!
}

How the browser represents a gamepad #

The browser represents gamepads as Gamepad objects. A Gamepad has the following properties:

id: An identification string for the gamepad. This string identifies the brand or style of connected gamepad device.
displayId: The VRDisplay.displayId of an associated VRDisplay (if relevant).
index: The index of the gamepad in the navigator.
connected: Indicates whether the gamepad is still connected to the system.
hand: An enum defining what hand the controller is being held in, or is most likely to be held in.
timestamp: The last time the data for this gamepad was updated.
mapping: The button and axes mapping in use for this device, either "standard" or "xr-standard".
pose: A GamepadPose object representing the pose information associated with a WebVR controller.
axes: An array of values for all axes of the gamepad, linearly normalized to the range of -1.0–1.0.
buttons: An array of button states for all buttons of the gamepad.

Note that buttons can be digital (pressed or not pressed) or analog (for example, 78% pressed). This is why buttons are reported as GamepadButton objects, with the following attributes:

pressed: The pressed state of the button (true if the button is currently pressed, and false if it is not pressed.
touched: The touched state of the button. If the button is capable of detecting touch, this property is true if the button is currently being touched, and false otherwise.
value: For buttons that have an analog sensor, this property represents the amount by which the button has been pressed, linearly normalized to the range of 0.0–1.0.
hapticActuators: An array containing GamepadHapticActuator objects, each of which represents haptic feedback hardware available on the controller.

One additional thing that you might encounter, depending on your browser and the gamepad you have, is a vibrationActuator property. This field is currently implemented in Chrome and earmarked for merging into the Gamepad Extensions spec.

The schematic overview below, taken straight from the spec, shows the mapping and the arrangement of the buttons and axes on a generic gamepad.

Visual representation of a standard gamepad layout (Source).

Being notified when a gamepad gets connected #

To learn when a gamepad is connected, listen for the gamepadconnected event that triggers on the window object. When the user connects a gamepad, which can either happen via USB or via Bluetooth, a GamepadEvent is fired that has the gamepad's details in an aptly named gamepad property. Below, you can see an example from an Xbox 360 controller that I had lying around (yes, I am into retro gaming).

window.addEventListener('gamepadconnected', (event) => {
  console.log('✅ 🎮 A gamepad was connected:', event.gamepad);
  /*
    gamepad: Gamepad
    axes: (4) [0, 0, 0, 0]
    buttons: (17) [GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton]
    connected: true
    id: "Xbox 360 Controller (STANDARD GAMEPAD Vendor: 045e Product: 028e)"
    index: 0
    mapping: "standard"
    timestamp: 6563054.284999998
    vibrationActuator: GamepadHapticActuator {type: "dual-rumble"}
  */
});

Being notified when a gamepad gets disconnected #

Being notified of gamepad disconnections happens analogously to the way connections are detected. This time the app listens for the gamepaddisconnected event. Note how in the example below connected is now false when I unplug the Xbox 360 controller.

window.addEventListener('gamepaddisconnected', (event) => {
  console.log('❌ 🎮 A gamepad was disconnected:', event.gamepad);
  /*
    gamepad: Gamepad
    axes: (4) [0, 0, 0, 0]
    buttons: (17) [GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton, GamepadButton]
    connected: false
    id: "Xbox 360 Controller (STANDARD GAMEPAD Vendor: 045e Product: 028e)"
    index: 0
    mapping: "standard"
    timestamp: 6563054.284999998
    vibrationActuator: null
  */
});

The gamepad in your game loop #

Getting ahold of a gamepad starts with a call to navigator.getGamepads(), which returns an array with Gamepad items. The array in Chrome always has a fixed length of four items. If zero or less than four gamepads are connected, an item may just be null. Always be sure to check all items of the array and be aware that gamepads "remember" their slot and may not always be present at the first available slot.

// When no gamepads are connected:
navigator.getGamepads();
// (4) [null, null, null, null]

If one or several gamepads are connected, but navigator.getGamepads() still reports null items, you may need to "wake" each gamepad by pressing any of its buttons. You can then poll the gamepad states in your game loop as shown below.

const pollGamepads = () => {
  // Always call `navigator.getGamepads()` inside of
  // the game loop, not outside.
  const gamepads = navigator.getGamepads();
  for (const gamepad of gamepads) {
    // Disregard empty slots.
    if (!gamepad) {
      continue;
    }
    // Process the gamepad state.
    console.log(gamepad);
  }
  // Call yourself upon the next animation frame.
  // (Typically this happens every 60 times per second.)
  window.requestAnimationFrame(pollGamepads);
};
// Kick off the initial game loop iteration.
pollGamepads();

Using the vibration actuator #

The vibrationActuator property returns a GamepadHapticActuator object, which corresponds to a configuration of motors or other actuators that can apply a force for the purposes of haptic feedback. Haptic effects can be played by calling Gamepad.vibrationActuator.playEffect(). The only currently valid effect type is 'dual-rumble'. Dual-rumble describes a haptic configuration with an eccentric rotating mass vibration motor in each handle of a standard gamepad. In this configuration, either motor is capable of vibrating the whole gamepad. The two masses are unequal so that the effects of each can be combined to create more complex haptic effects. Dual-rumble effects are defined by four parameters:

duration: Sets the duration of the vibration effect in milliseconds.
startDelay: Sets the duration of the delay until the vibration is started.
strongMagnitude and weakMagnitude: Set the vibration intensity levels for the heavier and lighter eccentric rotating mass motors, normalized to the range 0.0–1.0.

// This assumes a `Gamepad` as the value of the `gamepad` variable.
const vibrate = (gamepad, delay = 0, duration = 100, weak = 1.0, strong = 1.0) => {
  if (!('vibrationActuator' in gamepad)) {
    return;
  }
  gamepad.vibrationActuator.playEffect('dual-rumble', {
    // Start delay in ms.
    startDelay: delay,
    // Duration is ms.
    duration: duration,
    // The magnitude of the weak actuator (between 0 and 1).
    weakMagnitude: weak,
    // The magnitude of the strong actuator (between 0 and 1).
    strongMagnitude: strong,
  });
};

Integration with Permissions Policy #

The Gamepad API spec defines a policy-controlled feature identified by the string "gamepad". Its default allowlist is "self". A document's permissions policy determines whether any content in that document is allowed to access navigator.getGamepads(). If disabled in any document, no content in the document will be allowed to use navigator.getGamepads(), nor will the gamepadconnected and gamepaddisconnected events fire.

<iframe src="index.html" allow="gamepad"></iframe>

Demo #

A simple gamepad tester demo is embedded below. The source code is available on Glitch. Try the demo by connecting a gamepad via USB or Bluetooth and pressing any of its buttons or moving any of its axis.

Bonus: play Chrome dino on web.dev #

You can play Chrome dino with your gamepad on this very site. The source code is available on GitHub. Check out the gamepad polling implementation in trex-runner.js and note how it is emulating key presses.

For the Chrome dino gamepad demo to work, I have ripped out the Chrome dino game from the core Chromium project (updating an earlier effort by Arnelle Ballane), placed it on a standalone site, extended the existing gamepad API implementation by adding ducking and vibration effects, created a full screen mode, and Mehul Satardekar contributed a dark mode implementation. Happy gaming!

Useful links #

Acknowledgements #

This article was reviewed by François Beaufort and Joe Medley. The Gamepad API spec is currently edited by Steve Agoston, James Hollyer, and Matt Reynolds. The former spec editors are Brandon Jones, Scott Graham, and Ted Mielczarek. The Gamepad Extensions spec is edited by Brandon Jones. Hero image by Laura Torrent Puig.

Create an offline fallback page

2020-09-24T00:00:00Z

What do the Google Assistant app, the Slack app, the Zoom app, and almost any other platform-specific app on your phone or computer have in common? Right, they always at least give you something. Even when you do not have a network connection, you can still open the Assistant app, or enter Slack, or launch Zoom. You might not get anything particularly meaningful or even be unable to achieve what you wanted to achieve, but at least you get something and the app is in control.

Google Assistant.

Slack.

Zoom.

With platform-specific apps, even when you do not have a network connection, you never get nothing.

In contrast, on the Web, traditionally you get nothing when you are offline. Chrome gives you the offline dino game, but that is it.

Google Chrome for iOS.

Google Chrome for macOS.

On the Web, when you do not have a network connection, by default you get nothing.

An offline fallback page with a custom service worker #

It does not have to be this way, though. Thanks to service workers and the Cache Storage API, you can provide a customized offline experience for your users. This can be a simple branded page with the information that the user is currently offline, but it can just as well be a more creative solution, like, for example, the famous trivago offline maze game with a manual Reconnect button and an automatic reconnection attempt countdown.

The trivago offline maze.

Registering the service worker #

The way to make this happen is through a service worker. You can register a service worker from your main page as in the code sample below. Usually you do this once your app has loaded.

window.addEventListener("load", () => {
  if ("serviceWorker" in navigator) {
    navigator.serviceWorker.register("service-worker.js");
  }
});

The service worker code #

The contents of the actual service worker file may seem a little involved at first sight, but the comments in the sample below should clear things up. The core idea is to pre-cache a file named offline.html that only gets served on failing navigation requests, and to let the browser handle all other cases:

/*
Copyright 2015, 2019, 2020, 2021 Google LLC. All Rights Reserved.
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 http://www.apache.org/licenses/LICENSE-2.0
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
*/

// Incrementing OFFLINE_VERSION will kick off the install event and force
// previously cached resources to be updated from the network.
// This variable is intentionally declared and unused.
// Add a comment for your linter if you want:
// eslint-disable-next-line no-unused-vars
const OFFLINE_VERSION = 1;
const CACHE_NAME = "offline";
// Customize this with a different URL if needed.
const OFFLINE_URL = "offline.html";

self.addEventListener("install", (event) => {
  event.waitUntil(
    (async () => {
      const cache = await caches.open(CACHE_NAME);
      // Setting {cache: 'reload'} in the new request ensures that the
      // response isn't fulfilled from the HTTP cache; i.e., it will be
      // from the network.
      await cache.add(new Request(OFFLINE_URL, { cache: "reload" }));
    })()
  );
  // Force the waiting service worker to become the active service worker.
  self.skipWaiting();
});

self.addEventListener("activate", (event) => {
  event.waitUntil(
    (async () => {
      // Enable navigation preload if it's supported.
      // See https://developers.google.com/web/updates/2017/02/navigation-preload
      if ("navigationPreload" in self.registration) {
        await self.registration.navigationPreload.enable();
      }
    })()
  );

  // Tell the active service worker to take control of the page immediately.
  self.clients.claim();
});

self.addEventListener("fetch", (event) => {
  // Only call event.respondWith() if this is a navigation request
  // for an HTML page.
  if (event.request.mode === "navigate") {
    event.respondWith(
      (async () => {
        try {
          // First, try to use the navigation preload response if it's
          // supported.
          const preloadResponse = await event.preloadResponse;
          if (preloadResponse) {
            return preloadResponse;
          }

          // Always try the network first.
          const networkResponse = await fetch(event.request);
          return networkResponse;
        } catch (error) {
          // catch is only triggered if an exception is thrown, which is
          // likely due to a network error.
          // If fetch() returns a valid HTTP response with a response code in
          // the 4xx or 5xx range, the catch() will NOT be called.
          console.log("Fetch failed; returning offline page instead.", error);

          const cache = await caches.open(CACHE_NAME);
          const cachedResponse = await cache.match(OFFLINE_URL);
          return cachedResponse;
        }
      })()
    );
  }

  // If our if() condition is false, then this fetch handler won't
  // intercept the request. If there are any other fetch handlers
  // registered, they will get a chance to call event.respondWith().
  // If no fetch handlers call event.respondWith(), the request
  // will be handled by the browser as if there were no service
  // worker involvement.
});

The offline fallback page #

The offline.html file is where you can get creative and adapt it to your needs and add your branding. The example below shows the bare minimum of what is possible. It demonstrates both manual reload based on a button press as well as automatic reload based on the online event and regular server polling.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />

    <title>You are offline</title>

    <!-- Inline the page's stylesheet. -->
    <style>
      body {
        font-family: helvetica, arial, sans-serif;
        margin: 2em;
      }

      h1 {
        font-style: italic;
        color: #373fff;
      }

      p {
        margin-block: 1rem;
      }

      button {
        display: block;
      }
    </style>
  </head>
  <body>
    <h1>You are offline</h1>

    <p>Click the button below to try reloading.</p>
    <button type="button">⤾ Reload</button>

    <!-- Inline the page's JavaScript file. -->
    <script>
      // Manual reload feature.
      document.querySelector("button").addEventListener("click", () => {
        window.location.reload();
      });

      // Listen to changes in the network state, reload when online.
      // This handles the case when the device is completely offline.
      window.addEventListener('online', () => {
        window.location.reload();
      });

      // Check if the server is responding and reload the page if it is.
      // This handles the case when the device is online, but the server
      // is offline or misbehaving.
      async function checkNetworkAndReload() {
        try {
          const response = await fetch('.');
          // Verify we get a valid response from the server
          if (response.status >= 200 && response.status < 500) {
            window.location.reload();
            return;
          }
        } catch {
          // Unable to connect to the server, ignore.
        }
        window.setTimeout(checkNetworkAndReload, 2500);
      }

      checkNetworkAndReload();
    </script>
  </body>
</html>

Demo #

You can see the offline fallback page in action in the demo embedded below. If you are interested, you can explore the source code on Glitch.

Side note on making your app installable #

Now that your site has an offline fallback page, you might wonder about next steps. To make your app installable, you need to add a web app manifest and optionally come up with an install strategy.

Side note on serving an offline fallback page with Workbox.js #

You may have heard of Workbox. Workbox is a set of JavaScript libraries for adding offline support to web apps. If you prefer to write less service worker code yourself, you can use the Workbox recipe for an offline page only.

Up next, learn how to define an install strategy for your app.

Unblocking clipboard access

2020-07-31T00:00:00Z

The traditional way of getting access to the system clipboard was via document.execCommand() for clipboard interactions. Though widely supported, this method of cutting and pasting came at a cost: clipboard access was synchronous, and could only read and write to the DOM.

That's fine for small bits of text, but there are many cases where blocking the page for clipboard transfer is a poor experience. Time consuming sanitization or image decoding might be needed before content can be safely pasted. The browser may need to load or inline linked resources from a pasted document. That would block the page while waiting on the disk or network. Imagine adding permissions into the mix, requiring that the browser block the page while requesting clipboard access. At the same time, the permissions put in place around document.execCommand() for clipboard interaction are loosely defined and vary between browsers.

The Async Clipboard API addresses these issues, providing a well-defined permissions model that doesn't block the page. The Async Clipboard API is limited to handling text and images on most browsers, but support varies. Be sure to carefully study the browser compatibility overview for each of the following sections.

Copy: writing data to the clipboard #

writeText() #

To copy text to the clipboard call writeText(). Since this API is asynchronous, the writeText() function returns a Promise that resolves or rejects depending on whether the passed text is copied successfully:

async function copyPageUrl() {
  try {
    await navigator.clipboard.writeText(location.href);
    console.log('Page URL copied to clipboard');
  } catch (err) {
    console.error('Failed to copy: ', err);
  }
}

Browser support

Chrome 66, Supported 66
Firefox 63, Supported 63
Edge 79, Supported 79
Safari 13.1, Supported 13.1

Source

write() #

Actually, writeText() is just a convenience method for the generic write() method, which also lets you copy images to the clipboard. Like writeText(), it is asynchronous and returns a Promise.

To write an image to the clipboard, you need the image as a blob. One way to do this is by requesting the image from a server using fetch(), then calling blob() on the response.

Requesting an image from the server may not be desirable or possible for a variety of reasons. Fortunately, you can also draw the image to a canvas and call the canvas' toBlob() method.

Next, pass an array of ClipboardItem objects as a parameter to the write() method. Currently you can only pass one image at a time, but we hope to add support for multiple images in the future. ClipboardItem takes an object with the MIME type of the image as the key and the blob as the value. For blob objects obtained from fetch() or canvas.toBlob(), the blob.type property automatically contains the correct MIME type for an image.

try {
  const imgURL = '/images/generic/file.png';
  const data = await fetch(imgURL);
  const blob = await data.blob();
  await navigator.clipboard.write([
    new ClipboardItem({
      // The key is determined dynamically based on the blob's type.
      [blob.type]: blob
    })
  ]);
  console.log('Image copied.');
} catch (err) {
  console.error(err.name, err.message);
}

Alternatively, you can write a promise to the ClipboardItem object. For this pattern, you need to know the MIME type of the data beforehand.

try {
  const imgURL = '/images/generic/file.png';
  await navigator.clipboard.write([
    new ClipboardItem({
      // Set the key beforehand and write a promise as the value.
      'image/png': fetch(imgURL).then(response => response.blob()),
    })
  ]);
  console.log('Image copied.');
} catch (err) {
  console.error(err.name, err.message);
}

Browser support

Chrome 66, Supported 66
Firefox 87, Behind a flag
Edge 79, Supported 79
Safari 13.1, Supported 13.1

Source

The copy event #

In the case where a user initiates a clipboard copy and does not call preventDefault(), the copy event includes a clipboardData property with the items already in the right format. If you want to implement your own logic, you need to call preventDefault() to prevent the default behavior in favor of your own implementation. In this case, clipboardData will be empty. Consider a page with text and an image, and when the user selects all and initiates a clipboard copy, your custom solution should discard text and only copy the image. You can achieve this as shown in the code sample below. What's not covered in this example is how to fall back to earlier APIs when the Clipboard API isn't supported.

<!-- The image we want on the clipboard. -->
<img src="kitten.webp" alt="Cute kitten.">
<!-- Some text we're not interested in. -->
<p>Lorem ipsum</p>

document.addEventListener("copy", async (e) => {
  // Prevent the default behavior.
  e.preventDefault();
  try {
    // Prepare an array for the clipboard items.
    let clipboardItems = [];
    // Assume `blob` is the blob representation of `kitten.webp`.
    clipboardItems.push(
      new ClipboardItem({
        [blob.type]: blob,
      })
    );
    await navigator.clipboard.write(clipboardItems);
    console.log("Image copied, text ignored.");
  } catch (err) {
    console.error(err.name, err.message);
  }
});

For the copy event:

Browser support

Chrome 1, Supported 1
Firefox 22, Supported 22
Edge 12, Supported 12
Safari 3, Supported 3

Source

For ClipboardItem:

Browser support

Chrome 76, Supported 76
Firefox 87, Behind a flag
Edge 79, Supported 79
Safari 13.1, Supported 13.1

Source

Paste: reading data from clipboard #

readText() #

To read text from the clipboard, call navigator.clipboard.readText() and wait for the returned promise to resolve:

async function getClipboardContents() {
  try {
    const text = await navigator.clipboard.readText();
    console.log('Pasted content: ', text);
  } catch (err) {
    console.error('Failed to read clipboard contents: ', err);
  }
}

Browser support

Chrome 66, Supported 66
Firefox, Not supported
Edge 79, Supported 79
Safari 13.1, Supported 13.1

Source

read() #

The navigator.clipboard.read() method is also asynchronous and returns a promise. To read an image from the clipboard, obtain a list of ClipboardItem objects, then iterate over them.

Each ClipboardItem can hold its contents in different types, so you'll need to iterate over the list of types, again using a for...of loop. For each type, call the getType() method with the current type as an argument to obtain the corresponding blob. As before, this code is not tied to images, and will work with other future file types.

async function getClipboardContents() {
  try {
    const clipboardItems = await navigator.clipboard.read();
    for (const clipboardItem of clipboardItems) {
      for (const type of clipboardItem.types) {
        const blob = await clipboardItem.getType(type);
        console.log(URL.createObjectURL(blob));
      }
    }
  } catch (err) {
    console.error(err.name, err.message);
  }
}

Browser support

Chrome 86, Supported 86
Firefox 90, Behind a flag
Edge 79, Supported 79
Safari 13.1, Supported 13.1

Source

Working with pasted files #

It is useful for users to be able to use clipboard keyboard shortcuts such as ctrl+c and ctrl+v. Chromium exposes read-only files on the clipboard as outlined below. This triggers when the user hits the operating system's default paste shortcut or when the user clicks Edit then Paste in the browser's menu bar. No further plumbing code is needed.

document.addEventListener("paste", async e => {
  e.preventDefault();
  if (!e.clipboardData.files.length) {
    return;
  }
  const file = e.clipboardData.files[0];
  // Read the file's contents, assuming it's a text file.
  // There is no way to write back to it.
  console.log(await file.text());
});

Browser support

Chrome 3, Supported 3
Firefox 3.6, Supported 3.6
Edge 12, Supported 12
Safari 4, Supported 4

Source

The paste event #

As noted before, there are plans to introduce events to work with the Clipboard API, but for now you can use the existing paste event. It works nicely with the new asynchronous methods for reading clipboard text. As with the copy event, don't forget to call preventDefault().

document.addEventListener('paste', async (e) => {
  e.preventDefault();
  const text = await navigator.clipboard.readText();
  console.log('Pasted text: ', text);
});

Browser support

Chrome 1, Supported 1
Firefox 22, Supported 22
Edge 12, Supported 12
Safari 3, Supported 3

Source

Handling multiple MIME types #

Most implementations put multiple data formats on the clipboard for a single cut or copy operation. There are two reasons for this: as an app developer, you have no way of knowing the capabilities of the app that a user wants to copy text or images to, and many applications support pasting structured data as plain text. This is typically presented to users with an Edit menu item with a name such as Paste and match style or Paste without formatting.

The following example shows how to do this. This example uses fetch() to obtain image data, but it could also come from a <canvas> or the File System Access API.

async function copy() {
  const image = await fetch('kitten.png').then(response => response.blob());
  const text = new Blob(['Cute sleeping kitten'], {type: 'text/plain'});
  const item = new ClipboardItem({
    'text/plain': text,
    'image/png': image
  });
  await navigator.clipboard.write([item]);
}

Security and permissions #

Clipboard access has always presented a security concern for browsers. Without proper permissions, a page could silently copy all manner of malicious content to a user's clipboard that would produce catastrophic results when pasted. Imagine a web page that silently copies rm -rf / or a decompression bomb image to your clipboard.

The permission prompt for the Clipboard API.

Giving web pages unfettered read access to the clipboard is even more troublesome. Users routinely copy sensitive information like passwords and personal details to the clipboard, which could then be read by any page without the user's knowledge.

As with many new APIs, the Clipboard API is only supported for pages served over HTTPS. To help prevent abuse, clipboard access is only allowed when a page is the active tab. Pages in active tabs can write to the clipboard without requesting permission, but reading from the clipboard always requires permission.

Permissions for copy and paste have been added to the Permissions API. The clipboard-write permission is granted automatically to pages when they are the active tab. The clipboard-read permission must be requested, which you can do by trying to read data from the clipboard. The code below shows the latter:

const queryOpts = { name: 'clipboard-read', allowWithoutGesture: false };
const permissionStatus = await navigator.permissions.query(queryOpts);
// Will be 'granted', 'denied' or 'prompt':
console.log(permissionStatus.state);

// Listen for changes to the permission state
permissionStatus.onchange = () => {
  console.log(permissionStatus.state);
};

You can also control whether a user gesture is required to invoke cutting or pasting using the allowWithoutGesture option. The default for this value varies by browser, so you should always include it.

Here's where the asynchronous nature of the Clipboard API really comes in handy: attempting to read or write clipboard data automatically prompts the user for permission if it hasn't already been granted. Since the API is promise-based, this is completely transparent, and a user denying clipboard permission causes the promise to reject so the page can respond appropriately.

Because browsers only allow clipboard access when a page is the active tab, you'll find that some of the examples here don't run if pasted directly into the browser's console, since the developer tools themselves are the active tab. There's a trick: defer clipboard access using setTimeout(), then quickly click inside the page to focus it before the functions are called:

setTimeout(async () => {
  const text = await navigator.clipboard.readText();
  console.log(text);
}, 2000);

Permissions policy integration #

To use the API in iframes, you need to enable it with Permissions Policy, which defines a mechanism that allows for selectively enabling and disabling various browser features and APIs. Concretely, you need to pass either or both of clipboard-read or clipboard-write, depending on the needs of your app.

<iframe
    src="index.html"
    allow="clipboard-read; clipboard-write"
>
</iframe>

Feature detection #

To use the Async Clipboard API while supporting all browsers, test for navigator.clipboard and fall back to earlier methods. For example, here's how you might implement pasting to include other browsers.

document.addEventListener('paste', async (e) => {
  e.preventDefault();
  let text;
  if (navigator.clipboard) {
    text = await navigator.clipboard.readText();
  }
  else {
    text = e.clipboardData.getData('text/plain');
  }
  console.log('Got pasted text: ', text);
});

That's not the whole story. Before the Async Clipboard API, there were a mix of different copy and paste implementations across web browsers. In most browsers, the browser's own copy and paste can be triggered using document.execCommand('copy') and document.execCommand('paste'). If the text to be copied is a string not present in the DOM, it must be injected into the DOM and selected:

button.addEventListener('click', (e) => {
  const input = document.createElement('input');
  input.style.display = 'none';
  document.body.appendChild(input);
  input.value = text;
  input.focus();
  input.select();
  const result = document.execCommand('copy');
  if (result === 'unsuccessful') {
    console.error('Failed to copy text.');
  }
  input.remove();
});

Demos #

You can play with the Async Clipboard API in the demos below. On Glitch you can remix the text demo or the image demo to experiment with them.

The first example demonstrates moving text on and off the clipboard.

To try the API with images, use this demo. Recall that only PNGs are supported and only in a few browsers.

Acknowledgements #

The Asynchronous Clipboard API was implemented by Darwin Huang and Gary Kačmarčík. Darwin also provided the demo. Thanks to Kyarik and again Gary Kačmarčík for reviewing parts of this article.

Hero image by Markus Winkler on Unsplash.

Perform efficient per-video-frame operations on video with `requestVideoFrameCallback()`

2020-06-29T00:00:00Z

There's a new Web API on the block, defined in the HTMLVideoElement.requestVideoFrameCallback() specification. The requestVideoFrameCallback() method allows web authors to register a callback that runs in the rendering steps when a new video frame is sent to the compositor. This is intended to allow developers to perform efficient per-video-frame operations on video, such as video processing and painting to a canvas, video analysis, or synchronization with external audio sources.

Difference with requestAnimationFrame() #

Operations like drawing a video frame to a canvas via drawImage() made through this API will be synchronized as a best effort with the frame rate of the video playing on screen. Different from window.requestAnimationFrame(), which usually fires about 60 times per second, requestVideoFrameCallback() is bound to the actual video frame rate—with an important exception:

The effective rate at which callbacks are run is the lesser rate between the video's rate and the browser's rate. This means a 25fps video playing in a browser that paints at 60Hz would fire callbacks at 25Hz. A 120fps video in that same 60Hz browser would fire callbacks at 60Hz.

What's in a name? #

Due to its similarity with window.requestAnimationFrame(), the method initially was proposed as video.requestAnimationFrame(), but I'm happy with the new name, requestVideoFrameCallback(), which was agreed on after a lengthy discussion. Yay, bikeshedding for the win!

Feature detection #

if ('requestVideoFrameCallback' in HTMLVideoElement.prototype) {
  // The API is supported! 
}

Browser support #

Browser support

Chrome 83, Supported 83
Firefox, Not supported
Edge 83, Supported 83
Safari 15.4, Supported 15.4

Polyfill #

A polyfill for the requestVideoFrameCallback() method based on Window.requestAnimationFrame() and HTMLVideoElement.getVideoPlaybackQuality() is available. Before using this, be aware of the limitations mentioned in the README.

Using the requestVideoFrameCallback() method #

If you have ever used the requestAnimationFrame() method, you will immediately feel at home with the requestVideoFrameCallback() method. You register an initial callback once, and then re-register whenever the callback fires.

const doSomethingWithTheFrame = (now, metadata) => {
  // Do something with the frame.
  console.log(now, metadata);
  // Re-register the callback to be notified about the next frame.
  video.requestVideoFrameCallback(doSomethingWithTheFrame);
};
// Initially register the callback to be notified about the first frame.
video.requestVideoFrameCallback(doSomethingWithTheFrame);

In the callback, now is a DOMHighResTimeStamp and metadata is a VideoFrameMetadata dictionary with the following properties:

presentationTime, of type DOMHighResTimeStamp: The time at which the user agent submitted the frame for composition.
expectedDisplayTime, of type DOMHighResTimeStamp: The time at which the user agent expects the frame to be visible.
width, of type unsigned long: The width of the video frame, in media pixels.
height, of type unsigned long: The height of the video frame, in media pixels.
mediaTime, of type double: The media presentation timestamp (PTS) in seconds of the frame presented (e.g., its timestamp on the video.currentTime timeline).
presentedFrames, of type unsigned long: A count of the number of frames submitted for composition. Allows clients to determine if frames were missed between instances of VideoFrameRequestCallback.
processingDuration, of type double: The elapsed duration in seconds from submission of the encoded packet with the same presentation timestamp (PTS) as this frame (e.g., same as the mediaTime) to the decoder until the decoded frame was ready for presentation.

For WebRTC applications, additional properties may appear:

captureTime, of type DOMHighResTimeStamp: For video frames coming from either a local or remote source, this is the time at which the frame was captured by the camera. For a remote source, the capture time is estimated using clock synchronization and RTCP sender reports to convert RTP timestamps to capture time.
receiveTime, of type DOMHighResTimeStamp: For video frames coming from a remote source, this is the time the encoded frame was received by the platform, i.e., the time at which the last packet belonging to this frame was received over the network.
rtpTimestamp, of type unsigned long: The RTP timestamp associated with this video frame.

Of special interest in this list is mediaTime. In Chromium's implementation, we use the audio clock as the time source that backs video.currentTime, whereas the mediaTime is directly populated by the presentationTimestamp of the frame. The mediaTime is what you should use if you want to exactly identify frames in a reproducible way, including to identify exactly which frames you missed.

If things seem one frame off… #

Vertical synchronization (or just vsync), is a graphics technology that synchronizes the frame rate of a video and the refresh rate of a monitor. Since requestVideoFrameCallback() runs on the main thread, but, under the hood, video compositing happens on the compositor thread, everything from this API is a best effort, and we do not offer any strict guarantees. What may be happening is that the API can be one vsync late relative to when a video frame is rendered. It takes one vsync for changes made to the web page through the API to appear on screen (same as window.requestAnimationFrame()). So if you keep updating the mediaTime or frame number on your web page and compare that against the numbered video frames, eventually the video will look like it is one frame ahead.

What is really happening is that the frame is ready at vsync x, the callback is fired and the frame is rendered at vsync x+1, and changes made in the callback are rendered at vsync x+2. You can check whether the callback is a vsync late (and the frame is already rendered on screen) by checking whether the metadata.expectedDisplayTime is roughly now or one vsync in the future. If it is within about five to ten microseconds of now, the frame is already rendered; if the expectedDisplayTime is approximately sixteen milliseconds in the future (assuming your browser/screen is refreshing at 60Hz), then you are in sync with the frame.

Demo #

I have created a small demo on Glitch that shows how frames are drawn on a canvas at exactly the frame rate of the video and where the frame metadata is logged for debugging purposes. The core logic is just a couple of lines of JavaScript.

let paintCount = 0;
let startTime = 0.0;

const updateCanvas = (now, metadata) => {
  if (startTime === 0.0) {
    startTime = now;
  }

  ctx.drawImage(video, 0, 0, canvas.width, canvas.height);

  const elapsed = (now - startTime) / 1000.0;
  const fps = (++paintCount / elapsed).toFixed(3);
  fpsInfo.innerText = `video fps: ${fps}`;
  metadataInfo.innerText = JSON.stringify(metadata, null, 2);

  video.requestVideoFrameCallback(updateCanvas);
};

video.requestVideoFrameCallback(updateCanvas);

Conclusions #

I have done frame-level processing for a long time—without having access to the actual frames, only based on video.currentTime. I implemented video shot segmentation in JavaScript in a rough-and-ready manner; you can still read the accompanying research paper. Had the requestVideoFrameCallback() existed back then, my life would have been much simpler…

Acknowledgements #

The requestVideoFrameCallback API was specified and implemented by Thomas Guilbert. This article was reviewed by Joe Medley and Kayce Basques. Hero image by Denise Jans on Unsplash.

Progressively enhance your Progressive Web App

2020-06-29T00:00:00Z

Back in March 2003, Nick Finck and Steve Champeon stunned the web design world with the concept of progressive enhancement, a strategy for web design that emphasizes loading core web page content first, and that then progressively adds more nuanced and technically rigorous layers of presentation and features on top of the content. While in 2003, progressive enhancement was about using—at the time—modern CSS features, unobtrusive JavaScript, and even just Scalable Vector Graphics. Progressive enhancement in 2020 and beyond is about using modern browser capabilities.

Slide: Inclusive Web Design for the Future With Progressive Enhancement. (Source)

Modern JavaScript #

Speaking of JavaScript, the browser support situation for the latest core ES 2015 JavaScript features is great. The new standard includes promises, modules, classes, template literals, arrow functions, let and const, default parameters, generators, the destructuring assignment, rest and spread, Map/Set, WeakMap/WeakSet, and many more. All are supported.

The ECMAScript 2015 (ES6) browser support table. (Source)

Async functions, an ES 2017 feature and one of my personal favorites, can be used in all major browsers. The async and await keywords enable asynchronous, promise-based behavior to be written in a cleaner style, avoiding the need to explicitly configure promise chains.

The Async functions browser support table. (Source)

And even super recent ES 2020 language additions like optional chaining and nullish coalescing have reached support really quickly. You can see a code sample below. When it comes to core JavaScript features, the grass couldn't be much greener than it is today.

const adventurer = {
  name: 'Alice',
  cat: {
    name: 'Dinah',
  },
};
console.log(adventurer.dog?.name);
// Expected output: undefined
console.log(0 ?? 42);
// Expected output: 0

The grass is green when it comes to core JavaScript features. (Microsoft product screenshot, used with permission.)

The sample app: Fugu Greetings #

For this article, I work with a simple PWA, called Fugu Greetings (GitHub). The name of this app is a tip of the hat to Project Fugu 🐡, an effort to give the web all the powers of Android/iOS/desktop applications. You can read more about the project on its landing page.

Fugu Greetings is a drawing app that lets you create virtual greeting cards, and send them to your loved ones. It exemplifies PWA's core concepts. It's reliable and fully offline enabled, so even if you don't have a network, you can still use it. It's also Installable to a device's home screen and integrates seamlessly with the operating system as a stand-alone application.

The Fugu Greetings sample app.

Progressive enhancement #

With this out of the way, it's time to talk about progressive enhancement. The MDN Web Docs Glossary defines the concept as follows:

Progressive enhancement is a design philosophy that provides a baseline of essential content and functionality to as many users as possible, while delivering the best possible experience only to users of the most modern browsers that can run all the required code.

Feature detection is generally used to determine whether browsers can handle more modern functionality, while polyfills are often used to add missing features with JavaScript.

[…]

Progressive enhancement is a useful technique that allows web developers to focus on developing the best possible websites while making those websites work on multiple unknown user agents. Graceful degradation is related, but is not the same thing and is often seen as going in the opposite direction to progressive enhancement. In reality, both approaches are valid and can often complement one another.

MDN contributors

Starting each greeting card from scratch can be really cumbersome. So why not have a feature that allows users to import an image, and start from there? With a traditional approach, you'd have used an <input type=file> element to make this happen. First, you'd create the element, set its type to 'file' and add MIME types to the accept property, and then programmatically "click" it and listen for changes. When you select an image, it is imported straight onto the canvas.

const importImage = async () => {
  return new Promise((resolve) => {
    const input = document.createElement('input');
    input.type = 'file';
    input.accept = 'image/*';
    input.addEventListener('change', () => {
      resolve(input.files[0]);
    });
    input.click();
  });
};

When there's an import feature, there probably should be an export feature so users can save their greeting cards locally. The traditional way of saving files is to create an anchor link with a download attribute and with a blob URL as its href. You'd also programmatically "click" it to trigger the download, and, to prevent memory leaks, hopefully not forget to revoke the blob object URL.

const exportImage = async (blob) => {
  const a = document.createElement('a');
  a.download = 'fugu-greeting.png';
  a.href = URL.createObjectURL(blob);
  a.addEventListener('click', (e) => {
    setTimeout(() => URL.revokeObjectURL(a.href), 30 * 1000);
  });
  a.click();
};

But wait a minute. Mentally, you haven't "downloaded" a greeting card, you have "saved" it. Rather than showing you a "save" dialog that lets you choose where to put the file, the browser has directly downloaded the greeting card without user interaction and has put it straight into your Downloads folder. This isn't great.

What if there were a better way? What if you could just open a local file, edit it, and then save the modifications, either to a new file, or back to the original file that you had initially opened? Turns out there is. The File System Access API allows you to open and create files and directories, as well as modify and save them .

So how do I feature-detect an API? The File System Access API exposes a new method window.chooseFileSystemEntries(). Consequently, I need to conditionally load different import and export modules depending on whether this method is available. I've shown how to do this below.

const loadImportAndExport = () => {
  if ('chooseFileSystemEntries' in window) {
    Promise.all([
      import('./import_image.mjs'),
      import('./export_image.mjs'),
    ]);
  } else {
    Promise.all([
      import('./import_image_legacy.mjs'),
      import('./export_image_legacy.mjs'),
    ]);
  }
};

But before I dive into the File System Access API details, let me just quickly highlight the progressive enhancement pattern here. On browsers that currently don't support the File System Access API, I load the legacy scripts. You can see the network tabs of Firefox and Safari below.

Safari Web Inspector network tab.

Firefox Developer Tools network tab.

However, on Chrome, a browser that supports the API, only the new scripts are loaded. This is made elegantly possible thanks to dynamic import(), which all modern browsers support. As I said earlier, the grass is pretty green these days.

Chrome DevTools network tab.

The File System Access API #

So now that I have addressed this, it's time to look at the actual implementation based on the File System Access API. For importing an image, I call window.chooseFileSystemEntries() and pass it an accepts property where I say I want image files. Both file extensions as well as MIME types are supported. This results in a file handle, from which I can get the actual file by calling getFile().

const importImage = async () => {
  try {
    const handle = await window.chooseFileSystemEntries({
      accepts: [
        {
          description: 'Image files',
          mimeTypes: ['image/*'],
          extensions: ['jpg', 'jpeg', 'png', 'webp', 'svg'],
        },
      ],
    });
    return handle.getFile();
  } catch (err) {
    console.error(err.name, err.message);
  }
};

Exporting an image is almost the same, but this time I need to pass a type parameter of 'save-file' to the chooseFileSystemEntries() method. From this I get a file save dialog. With file open, this wasn't necessary since 'open-file' is the default. I set the accepts parameter similarly to before, but this time limited to just PNG images. Again I get back a file handle, but rather than getting the file, this time I create a writable stream by calling createWritable(). Next, I write the blob, which is my greeting card image, to the file. Finally, I close the writable stream.

Everything can always fail: The disk could be out of space, there could be a write or read error, or maybe simply the user cancels the file dialog. This is why I always wrap the calls in a try...catch statement.

const exportImage = async (blob) => {
  try {
    const handle = await window.chooseFileSystemEntries({
      type: 'save-file',
      accepts: [
        {
          description: 'Image file',
          extensions: ['png'],
          mimeTypes: ['image/png'],
        },
      ],
    });
    const writable = await handle.createWritable();
    await writable.write(blob);
    await writable.close();
  } catch (err) {
    console.error(err.name, err.message);
  }
};

Using progressive enhancement with the File System Access API, I can open a file as before. The imported file is drawn right onto the canvas. I can make my edits and finally save them with a real save dialog box where I can choose the name and storage location of the file. Now the file is ready to be preserved for eternity.

The file open dialog.

The imported image.

Saving the modified image to a new file.

Apart from storing for eternity, maybe I actually want to share my greeting card. This is something that the Web Share API and Web Share Target API allow me to do. Mobile, and more recently desktop operating systems have gained built-in sharing mechanisms. For example, below is desktop Safari's share sheet on macOS triggered from an article on my blog. When you click the Share Article button, you can share a link to the article with a friend, for example, via the macOS Messages app.

Web Share API on desktop Safari on macOS.

The code to make this happen is pretty straightforward. I call navigator.share() and pass it an optional title, text, and url in an object. But what if I want to attach an image? Level 1 of the Web Share API doesn't support this yet. The good news is that Web Share Level 2 has added file sharing capabilities.

try {
  await navigator.share({
    title: 'Check out this article:',
    text: `"${document.title}" by @tomayac:`,
    url: document.querySelector('link[rel=canonical]').href,
  });
} catch (err) {
  console.warn(err.name, err.message);
}

Let me show you how to make this work with the Fugu Greeting card application. First, I need to prepare a data object with a files array consisting of one blob, and then a title and a text. Next, as a best practice, I use the new navigator.canShare() method which does what its name suggests: It tells me if the data object I'm trying to share can technically be shared by the browser. If navigator.canShare() tells me the data can be shared, I'm ready to call navigator.share() as before. Because everything can fail, I'm again using a try...catch block.

const share = async (title, text, blob) => {
  const data = {
    files: [
      new File([blob], 'fugu-greeting.png', {
        type: blob.type,
      }),
    ],
    title: title,
    text: text,
  };
  try {
    if (!(navigator.canShare(data))) {
      throw new Error("Can't share data.", data);
    }
    await navigator.share(data);
  } catch (err) {
    console.error(err.name, err.message);
  }
};

As before, I use progressive enhancement. If both 'share' and 'canShare' exist on the navigator object, only then I go forward and load share.mjs via dynamic import(). On browsers like mobile Safari that only fulfill one of the two conditions, I don't load the functionality.

const loadShare = () => {
  if ('share' in navigator && 'canShare' in navigator) {
    import('./share.mjs');
  }
};

In Fugu Greetings, if I tap the Share button on a supporting browser like Chrome on Android, the built-in share sheet opens. I can, for example, choose Gmail, and the email composer widget pops up with the image attached.

Choosing an app to share the file to.

The file gets attached to a new email in Gmail's composer.

The Contact Picker API #

Next, I want to talk about contacts, meaning a device's address book or contacts manager app. When you write a greeting card, it may not always be easy to correctly write someone's name. For example, I have a friend Sergey who prefers his name to be spelled in Cyrillic letters. I'm using a German QWERTZ keyboard and have no idea how to type their name. This is a problem that the Contact Picker API can solve. Since I have my friend stored in my phone's contacts app, via the Contacts Picker API, I can tap into my contacts from the web.

First, I need to specify the list of properties I want to access. In this case, I only want the names, but for other use cases I might be interested in telephone numbers, emails, avatar icons, or physical addresses. Next, I configure an options object and set multiple to true, so that I can select more than one entry. Finally, I can call navigator.contacts.select(), which returns the desired properties for the user-selected contacts.

const getContacts = async () => {
  const properties = ['name'];
  const options = { multiple: true };
  try {
    return await navigator.contacts.select(properties, options);
  } catch (err) {
    console.error(err.name, err.message);
  }
};

And by now you've probably learned the pattern: I only load the file when the API is actually supported.

if ('contacts' in navigator) {
  import('./contacts.mjs');
}

In Fugu Greeting, when I tap the Contacts button and select my two best pals, Сергей Михайлович Брин and 劳伦斯·爱德华·"拉里"·佩奇, you can see how the contacts picker is limited to only show their names, but not their email addresses, or other information like their phone numbers. Their names are then drawn onto my greeting card.

Selecting two names with the contact picker from the address book.

The two names then get drawn onto the greeting card.

The Asynchronous Clipboard API #

Up next is copying and pasting. One of our favorite operations as software developers is copy and paste. As a greeting card author, at times, I may want to do the same. I may want to either paste an image into a greeting card I'm working on, or copy my greeting card so I can continue editing it from somewhere else. The Async Clipboard API, supports both text and images. Let me walk you through how I added copy and paste support to the Fugu Greetings app.

In order to copy something onto the system's clipboard, I need to write to it. The navigator.clipboard.write() method takes an array of clipboard items as a parameter. Each clipboard item is essentially an object with a blob as a value, and the blob's type as the key.

const copy = async (blob) => {
  try {
    await navigator.clipboard.write([
      new ClipboardItem({
        [blob.type]: blob,
      }),
    ]);
  } catch (err) {
    console.error(err.name, err.message);
  }
};

To paste, I need to loop over the clipboard items that I obtain by calling navigator.clipboard.read(). The reason for this is that multiple clipboard items might be on the clipboard in different representations. Each clipboard item has a types field that tells me the MIME types of the available resources. I call the clipboard item's getType() method, passing the MIME type I obtained before.

const paste = async () => {
  try {
    const clipboardItems = await navigator.clipboard.read();
    for (const clipboardItem of clipboardItems) {
      try {
        for (const type of clipboardItem.types) {
          const blob = await clipboardItem.getType(type);
          return blob;
        }
      } catch (err) {
        console.error(err.name, err.message);
      }
    }
  } catch (err) {
    console.error(err.name, err.message);
  }
};

And it's almost needless to say by now. I only do this on supporting browsers.

if ('clipboard' in navigator && 'write' in navigator.clipboard) {
  import('./clipboard.mjs');
}

So how does this work in practice? I have an image open in the macOS Preview app and copy it to the clipboard. When I click Paste, the Fugu Greetings app then asks me whether I want to allow the app to see text and images on the clipboard.

The clipboard permission prompt.

Finally, after accepting the permission, the image is then pasted into the application. The other way round works, too. Let me copy a greeting card to the clipboard. When I then open Preview and click File and then New from Clipboard, the greeting card gets pasted into a new untitled image.

An image pasted into the macOS Preview app.

The Badging API #

Another useful API is the Badging API. As an installable PWA, Fugu Greetings of course does have an app icon that users can place on the app dock or the home screen. A fun and easy way to demonstrate the API is to (ab)use it in Fugu Greetings as a pen strokes counter. I have added an event listener that increments the pen strokes counter whenever the pointerdown event occurs and then sets the updated icon badge. Whenever the canvas gets cleared, the counter resets, and the badge is removed.

let strokes = 0;

canvas.addEventListener('pointerdown', () => {
  navigator.setAppBadge(++strokes);
});

clearButton.addEventListener('click', () => {
  strokes = 0;
  navigator.setAppBadge(strokes);
});

This feature is a progressive enhancement, so the loading logic is as usual.

if ('setAppBadge' in navigator) {
  import('./badge.mjs');
}

In this example, I have drawn the numbers from one to seven, using one pen stroke per number. The badge counter on the icon is now at seven.

Drawing the numbers from 1 to 7, using seven pen strokes.

The pen strokes counter in the form of the app icon badge.

The Periodic Background Sync API #

Want to start each day fresh with something new? A neat feature of the Fugu Greetings app is that it can inspire you each morning with a new background image to start your greeting card. The app uses the Periodic Background Sync API to achieve this.

The first step is to register a periodic sync event in the service worker registration. It listens for a sync tag called 'image-of-the-day' and has a minimum interval of one day, so the user can get a new background image every 24 hours.

const registerPeriodicBackgroundSync = async () => {
  const registration = await navigator.serviceWorker.ready;
  try {
    registration.periodicSync.register('image-of-the-day-sync', {
      // An interval of one day.
      minInterval: 24 * 60 * 60 * 1000,
    });
  } catch (err) {
    console.error(err.name, err.message);
  }
};

The second step is to listen for the periodicsync event in the service worker. If the event tag is 'image-of-the-day', that is, the one that was registered before, the image of the day is retrieved via the getImageOfTheDay() function, and the result propagated to all clients, so they can update their canvases and caches.

self.addEventListener('periodicsync', (syncEvent) => {
  if (syncEvent.tag === 'image-of-the-day-sync') {
    syncEvent.waitUntil(
      (async () => {
        const blob = await getImageOfTheDay();
        const clients = await self.clients.matchAll();
        clients.forEach((client) => {
          client.postMessage({
            image: blob,
          });
        });
      })()
    );
  }
});

Again this is truly a progressive enhancement, so the code is only loaded when the API is supported by the browser. This applies to both the client code and the service worker code. On non-supporting browsers, neither of them is loaded. Note how in the service worker, instead of a dynamic import() (that isn't supported in a service worker context yet), I use the classic importScripts().

// In the client:
const registration = await navigator.serviceWorker.ready;
if (registration && 'periodicSync' in registration) {
  import('./periodic_background_sync.mjs');
}

// In the service worker:
if ('periodicSync' in self.registration) {
  importScripts('./image_of_the_day.mjs');
}

In Fugu Greetings, pressing the Wallpaper button reveals the greeting card image of the day that is updated every day via the Periodic Background Sync API.

Pressing the Wallpaper button displays the image of the day.

Notification Triggers API #

Sometimes even with a lot of inspiration, you need a nudge to finish a started greeting card. This is a feature that is enabled by the Notification Triggers API. As a user, I can enter a time when I want to be nudged to finish my greeting card. When that time comes, I will get a notification that my greeting card is waiting.

After prompting for the target time, the application schedules the notification with a showTrigger. This can be a TimestampTrigger with the previously selected target date. The reminder notification will be triggered locally, no network or server side is needed.

const targetDate = promptTargetDate();
if (targetDate) {
  const registration = await navigator.serviceWorker.ready;
  registration.showNotification('Reminder', {
    tag: 'reminder',
    body: "It's time to finish your greeting card!",
    showTrigger: new TimestampTrigger(targetDate),
  });
}

As with everything else I have shown so far, this is a progressive enhancement, so the code is only conditionally loaded.

if ('Notification' in window && 'showTrigger' in Notification.prototype) {
  import('./notification_triggers.mjs');
}

When I check the Reminder checkbox in Fugu Greetings, a prompt asks me when I want to be reminded to finish my greeting card.

Scheduling a local notification to be reminded to finish a greeting card.

When a scheduled notification triggers in Fugu Greetings, it is shown just like any other notification, but as I wrote before, it didn't require a network connection.

The triggered notification appears in the macOS Notification Center.

The Wake Lock API #

I also want to include the Wake Lock API. Sometimes you just need to stare long enough at the screen until inspiration kisses you. The worst that can happen then is for the screen to turn off. The Wake Lock API can prevent this from happening.

The first step is to obtain a wake lock with the navigator.wakelock.request method(). I pass it the string 'screen' to obtain a screen wake lock. I then add an event listener to be informed when the wake lock is released. This can happen, for example, when the tab visibility changes. If this happens, I can, when the tab becomes visible again, re-obtain the wake lock.

let wakeLock = null;
const requestWakeLock = async () => {
  wakeLock = await navigator.wakeLock.request('screen');
  wakeLock.addEventListener('release', () => {
    console.log('Wake Lock was released');
  });
  console.log('Wake Lock is active');
};

const handleVisibilityChange = () => {
  if (wakeLock !== null && document.visibilityState === 'visible') {
    requestWakeLock();
  }
};

document.addEventListener('visibilitychange', handleVisibilityChange);
document.addEventListener('fullscreenchange', handleVisibilityChange);

Yes, this is a progressive enhancement, so I only need to load it when the browser supports the API.

if ('wakeLock' in navigator && 'request' in navigator.wakeLock) {
  import('./wake_lock.mjs');
}

In Fugu Greetings, there's an Insomnia checkbox that, when checked, keeps the screen awake.

The Insomnia checkbox keeps app awake.

The Idle Detection API #

At times, even if you stare at the screen for hours, it's just useless and you can't come up with the slightest idea what to do with your greeting card. The Idle Detection API allows the app to detect user idle time. If the user is idle for too long, the app resets to the initial state and clears the canvas. This API is currently gated behind the notifications permission, since a lot of production use cases of idle detection are notifications-related, for example, to only send a notification to a device the user is currently actively using.

After making sure that the notifications permission is granted, I then instantiate the idle detector. I register an event listener that listens for idle changes, which includes the user and the screen state. The user can be active or idle, and the screen can be unlocked or locked. If the user is idle, the canvas clears. I give the idle detector a threshold of 60 seconds.

const idleDetector = new IdleDetector();
idleDetector.addEventListener('change', () => {
  const userState = idleDetector.userState;
  const screenState = idleDetector.screenState;
  console.log(`Idle change: ${userState}, ${screenState}.`);
  if (userState === 'idle') {
    clearCanvas();
  }
});

await idleDetector.start({
  threshold: 60000,
  signal,
});

And as always, I only load this code when the browser supports it.

if ('IdleDetector' in window) {
  import('./idle_detection.mjs');
}

In the Fugu Greetings app, the canvas clears when the Ephemeral checkbox is checked and the user is idle for for too long.

When the Ephemeral checkbox is checked and the user has been idle for too long, the canvas is cleared.

Closing #

Phew, what a ride. So many APIs in just one sample app. And, remember, I never make the user pay the download cost for a feature that their browser doesn't support. By using progressive enhancement, I make sure only the relevant code gets loaded. And since with HTTP/2, requests are cheap, this pattern should work well for a lot of applications, although you might want to consider a bundler for really large apps.

Chrome DevTools Network tab showing only requests for files with code that the current browser supports.

The app may look a little different on each browser since not all platforms support all features, but the core functionality is always there—progressively enhanced according to the particular browser's capabilities. Note that these capabilities may change even in one and the same browser, depending on whether the app is running as an installed app or in a browser tab.

Fugu Greetings running on Android Chrome.

Fugu Greetings running on desktop Safari.

Fugu Greetings running on desktop Chrome.

If you're interested in the Fugu Greetings app, go find and fork it on GitHub.

Fugu Greetings app on GitHub.

The Chromium team is working hard on making the grass greener when it comes to advanced Fugu APIs. By applying progressive enhancement in the development of my app, I make sure that everybody gets a good, solid baseline experience, but that people using browsers that support more Web platform APIs get an even better experience. I'm looking forward to seeing what you do with progressive enhancement in your apps.

Acknowledgements #

I'm grateful to Christian Liebel and Hemanth HM who both have contributed to Fugu Greetings. This article was reviewed by Joe Medley and Kayce Basques. Jake Archibald helped me find out the situation with dynamic import() in a service worker context.

Boldly link where no one has linked before: Text Fragments

2020-06-17T00:00:00Z

Fragment Identifiers #

Chrome 80 was a big release. It contained a number of highly anticipated features like ECMAScript Modules in Web Workers, nullish coalescing, optional chaining, and more. The release was, as usual, announced through a blog post on the Chromium blog. You can see an excerpt of the blog post in the screenshot below.

Chromium blog post with red boxes around elements with an id attribute.

You are probably asking yourself what all the red boxes mean. They are the result of running the following snippet in DevTools. It highlights all elements that have an id attribute.

document.querySelectorAll('[id]').forEach((el) => {
  el.style.border = 'solid 2px red';
});

I can place a deep link to any element highlighted with a red box thanks to the fragment identifier which I then use in the hash of the page's URL. Assuming I wanted to deep link to the Give us feedback in our Product Forums box in the aside, I could do so by handcrafting the URL https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#HTML1. As you can see in the Elements panel of the Developer Tools, the element in question has an id attribute with the value HTML1.

Dev Tools showing the id of an element.

If I parse this URL with JavaScript's URL() constructor, the different components are revealed. Notice the hash property with the value #HTML1.

new URL('https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#HTML1');
/* Creates a new `URL` object
URL {
  hash: "#HTML1"
  host: "blog.chromium.org"
  hostname: "blog.chromium.org"
  href: "https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#HTML1"
  origin: "https://blog.chromium.org"
  password: ""
  pathname: "/2019/12/chrome-80-content-indexing-es-modules.html"
  port: ""
  protocol: "https:"
  search: ""
  searchParams: URLSearchParams {}
  username: ""
}
*/

The fact though that I had to open the Developer Tools to find the id of an element speaks volumes about the probability this particular section of the page was meant to be linked to by the author of the blog post.

What if I want to link to something without an id? Say I want to link to the ECMAScript Modules in Web Workers heading. As you can see in the screenshot below, the <h1> in question does not have an id attribute, meaning there is no way I can link to this heading. This is the problem that Text Fragments solve.

Dev Tools showing a heading without an id.

Text Fragments #

The Text Fragments proposal adds support for specifying a text snippet in the URL hash. When navigating to a URL with such a text fragment, the user agent can emphasize and/or bring it to the user's attention.

Browser compatibility #

Browser support

Chrome 89, Supported 89
Firefox, Not supported
Edge 89, Supported 89
Safari, Not supported

Source

For security reasons, the feature requires links to be opened in a noopener context. Therefore, make sure to include rel="noopener" in your <a> anchor markup or add noopener to your Window.open() list of window functionality features.

`start` #

In its simplest form, the syntax of Text Fragments is as follows: The hash symbol # followed by :~:text= and finally start, which represents the percent-encoded text I want to link to.

#:~:text=start

For example, say that I want to link to the ECMAScript Modules in Web Workers heading in the blog post announcing features in Chrome 80, the URL in this case would be:

https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#:~:text=ECMAScript%20Modules%20in%20Web%20Workers

The text fragment is emphasized like this. If you click the link in a supporting browser like Chrome, the text fragment is highlighted and scrolls into view:

Text fragment scrolled into view and highlighted.

`start` and `end` #

Now what if I want to link to the entire section titled ECMAScript Modules in Web Workers, not just its heading? Percent-encoding the entire text of the section would make the resulting URL impracticably long.

Luckily there is a better way. Rather than the entire text, I can frame the desired text using the start,end syntax. Therefore, I specify a couple of percent-encoded words at the beginning of the desired text, and a couple of percent-encoded words at the end of the desired text, separated by a comma ,.

That looks like this:

https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#:~:text=ECMAScript%20Modules%20in%20Web%20Workers,ES%20Modules%20in%20Web%20Workers..

For start, I have ECMAScript%20Modules%20in%20Web%20Workers, then a comma , followed by ES%20Modules%20in%20Web%20Workers. as end. When you click through on a supporting browser like Chrome, the whole section is highlighted and scrolled into view:

Text fragment scrolled into view and highlighted.

Now you may wonder about my choice of start and end. Actually, the slightly shorter URL https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#:~:text=ECMAScript%20Modules,Web%20Workers. with only two words on each side would have worked, too. Compare start and end with the previous values.

If I take it one step further and now use only one word for both start and end, you can see that I am in trouble. The URL https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#:~:text=ECMAScript,Workers. is even shorter now, but the highlighted text fragment is no longer the originally desired one. The highlighting stops at the first occurrence of the word Workers., which is correct, but not what I intended to highlight. The problem is that the desired section is not uniquely identified by the current one-word start and end values:

Non-intended text fragment scrolled into view and highlighted.

`prefix-` and `-suffix` #

Using long enough values for start and end is one solution for obtaining a unique link. In some situations, however, this is not possible. On a side note, why did I choose the Chrome 80 release blog post as my example? The answer is that in this release Text Fragments were introduced:

Text Fragments announcement blog post excerpt.

Notice how in the screenshot above the word "text" appears four times. The forth occurrence is written in a green code font. If I wanted to link to this particular word, I would set start to text. Since the word "text" is, well, only one word, there cannot be a end. What now? The URL https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#:~:text=text matches at the first occurrence of the word "Text" already in the heading:

Text Fragment matching at the first occurrence of "Text".

Luckily there is a solution. In cases like this, I can specify a prefix- and a -suffix. The word before the green code font "text" is "the", and the word after is "parameter". None of the other three occurrences of the word "text" has the same surrounding words. Armed with this knowledge, I can tweak the previous URL and add the prefix- and the -suffix. Like the other parameters, they, too, need to be percent-encoded and can contain more than one word. https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#:~:text=the-,text,-parameter. To allow the parser to clearly identify the prefix- and the -suffix, they need to be separated from the start and the optional end with a dash -.

Text Fragment matching at the desired occurrence of "text".

The full syntax #

The full syntax of Text Fragments is shown below. (Square brackets indicate an optional parameter.) The values for all parameters need to be percent-encoded. This is especially important for the dash -, ampersand &, and comma , characters, so they are not being interpreted as part of the text directive syntax.

#:~:text=[prefix-,]start[,end][,-suffix]

Each of prefix-, start, end, and -suffix will only match text within a single block-level element, but full start,end ranges can span multiple blocks. For example, :~:text=The quick,lazy dog will fail to match in the following example, because the starting string "The quick" does not appear within a single, uninterrupted block-level element:

<div>
  The
  <div></div>
  quick brown fox
</div>
<div>jumped over the lazy dog</div>

It does, however, match in this example:

<div>The quick brown fox</div>
<div>jumped over the lazy dog</div>

Creating Text Fragment URLs with a browser extension #

Creating Text Fragments URLs by hand is tedious, especially when it comes to making sure they are unique. If you really want to, the specification has some tips and lists the exact steps for generating Text Fragment URLs. We provide an open-source browser extension called Link to Text Fragment that lets you link to any text by selecting it, and then clicking "Copy Link to Selected Text" in the context menu. This extension is available for the following browsers:

Link to Text Fragment browser extension.

Multiple text fragments in one URL #

Note that multiple text fragments can appear in one URL. The particular text fragments need to be separated by an ampersand character &. Here is an example link with three text fragments: https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#:~:text=Text%20URL%20Fragments&text=text,-parameter&text=:~:text=On%20islands,%20birds%20can%20contribute%20as%20much%20as%2060%25%20of%20a%20cat's%20diet.

Three text fragments in one URL.

Mixing element and text fragments #

Traditional element fragments can be combined with text fragments. It is perfectly fine to have both in the same URL, for example, to provide a meaningful fallback in case the original text on the page changes, so that the text fragment does not match anymore. The URL https://blog.chromium.org/2019/12/chrome-80-content-indexing-es-modules.html#HTML1:~:text=Give%20us%20feedback%20in%20our%20Product%20Forums. linking to the Give us feedback in our Product Forums section contains both an element fragment (HTML1), as well as a text fragment (text=Give%20us%20feedback%20in%20our%20Product%20Forums.):

Linking with both element fragment and text fragment.

The fragment directive #

There is one element of the syntax I have not explained yet: the fragment directive :~:. To avoid compatibility issues with existing URL element fragments as shown above, the Text Fragments specification introduces the fragment directive. The fragment directive is a portion of the URL fragment delimited by the code sequence :~:. It is reserved for user agent instructions, such as text=, and is stripped from the URL during loading so that author scripts cannot directly interact with it. User agent instructions are also called directives. In the concrete case, text= is therefore called a text directive.

Feature detection #

To detect support, test for the read-only fragmentDirective property on document. The fragment directive is a mechanism for URLs to specify instructions directed to the browser rather than the document. It is meant to avoid direct interaction with author script, so that future user agent instructions can be added without fear of introducing breaking changes to existing content. One potential example of such future additions could be translation hints.

if ('fragmentDirective' in document) {
  // Text Fragments is supported.
}

Feature detection is mainly intended for cases where links are dynamically generated (for example by search engines) to avoid serving text fragments links to browsers that do not support them.

Styling text fragments #

By default, browsers style text fragments the same way they style mark (typically black on yellow, the CSS system colors for mark). The user-agent stylesheet contains CSS that looks like this:

:root::target-text {
  color: MarkText;
  background: Mark;
}

As you can see, the browser exposes a pseudo selector ::target-text that you can use to customize the applied highlighting. For example, you could design your text fragments to be black text on a red background. As always, be sure to check the color contrast so your override styling does not cause accessibility issues and make sure the highlighting actually visually stands out from the rest of the content.

:root::target-text {
  color: black;
  background-color: red;
}

Polyfillability #

The Text Fragments feature can be polyfilled to some extent. We provide a polyfill, which is used internally by the extension, for browsers that do not provide built-in support for Text Fragments where the functionality is implemented in JavaScript.

Programmatic Text Fragment link generation #

The polyfill contains a file fragment-generation-utils.js that you can import and use to generate Text Fragment links. This is outlined in the code sample below:

const { generateFragment } = await import('https://unpkg.com/text-fragments-polyfill/dist/fragment-generation-utils.js');
const result = generateFragment(window.getSelection());
if (result.status === 0) {
  let url = `${location.origin}${location.pathname}${location.search}`;
  const fragment = result.fragment;
  const prefix = fragment.prefix ?
    `${encodeURIComponent(fragment.prefix)}-,` :
    '';
  const suffix = fragment.suffix ?
    `,-${encodeURIComponent(fragment.suffix)}` :
    '';
  const start = encodeURIComponent(fragment.textStart);
  const end = fragment.textEnd ?
    `,${encodeURIComponent(fragment.textEnd)}` :
    '';
  url += `#:~:text=${prefix}${start}${end}${suffix}`;
  console.log(url);
}

Obtaining Text Fragments for analytics purposes #

Plenty of sites use the fragment for routing, which is why browsers strip out Text Fragments so as to not break those pages. There is an acknowledged need to expose Text Fragments links to pages, for example, for analytics purposes, but the proposed solution is not implemented yet. As a workaround for now, you can use the code below to extract the desired information.

new URL(performance.getEntries().find(({ type }) => type === 'navigate').name).hash;

Security #

Text fragment directives are invoked only on full (non-same-page) navigations that are the result of a user activation. Additionally, navigations originating from a different origin than the destination will require the navigation to take place in a noopener context, such that the destination page is known to be sufficiently isolated. Text fragment directives are only applied to the main frame. This means that text will not be searched inside iframes, and iframe navigation will not invoke a text fragment.

Privacy #

It is important that implementations of the Text Fragments specification do not leak whether a text fragment was found on a page or not. While element fragments are fully under the control of the original page author, text fragments can be created by anyone. Remember how in my example above there was no way to link to the ECMAScript Modules in Web Workers heading, since the <h1> did not have an id, but how anyone, including me, could just link to anywhere by carefully crafting the text fragment?

Imagine I ran an evil ad network evil-ads.example.com. Further imagine that in one of my ad iframes I dynamically created a hidden cross-origin iframe to dating.example.com with a Text Fragment URL dating.example.com#:~:text=Log%20Out once the user interacts with the ad. If the text "Log Out" is found, I know the victim is currently logged in to dating.example.com, which I could use for user profiling. Since a naive Text Fragments implementation might decide that a successful match should cause a focus switch, on evil-ads.example.com I could listen for the blur event and thus know when a match occurred. In Chrome, we have implemented Text Fragments in such a way that the above scenario cannot happen.

Another attack might be to exploit network traffic based on scroll position. Assume I had access to network traffic logs of my victim, like as the admin of a company intranet. Now imagine there existed a long human resources document What to Do If You Suffer From… and then a list of conditions like burn out, anxiety, etc. I could place a tracking pixel next to each item on the list. If I then determine that loading the document temporally co-occurs with the loading of the tracking pixel next to, say, the burn out item, I can then, as the intranet admin, determine that an employee has clicked through on a text fragment link with :~:text=burn%20out that the employee may have assumed was confidential and not visible to anyone. Since this example is somewhat contrived to begin with and since its exploitation requires very specific preconditions to be met, the Chrome security team evaluated the risk of implementing scroll on navigation to be manageable. Other user agents may decide to show a manual scroll UI element instead.

For sites that wish to opt-out, Chromium supports a Document Policy header value that they can send so user agents will not process Text Fragment URLs.

Document-Policy: force-load-at-top

Disabling text fragments #

The easiest way for disabling the feature is by using an extension that can inject HTTP response headers, for example, ModHeader (not a Google product), to insert a response (not request) header as follows:

Document-Policy: force-load-at-top

Another, more involved, way to opt out is by using the enterprise setting ScrollToTextFragmentEnabled. To do this on macOS, paste the command below in the terminal.

defaults write com.google.Chrome ScrollToTextFragmentEnabled -bool false

On Windows, follow the documentation on the Google Chrome Enterprise Help support site.

Text fragments in web search #

For some searches, the search engine Google provides a quick answer or summary with a content snippet from a relevant website. These featured snippets are most likely to show up when a search is in the form of a question. Clicking a featured snippet takes the user directly to the featured snippet text on the source web page. This works thanks to automatically created Text Fragments URLs.

Google search engine results page showing a featured snippet. The status bar shows the Text Fragments URL.

After clicking through, the relevant section of the page is scrolled into view.

Conclusion #

Text Fragments URL is a powerful feature to link to arbitrary text on webpages. The scholarly community can use it to provide highly accurate citation or reference links. Search engines can use it to deeplink to text results on pages. Social networking sites can use it to let users share specific passages of a webpage rather than inaccessible screenshots. I hope you start using Text Fragment URLs and find them as useful as I do. Be sure to install the Link to Text Fragment browser extension.

Acknowledgements #

Text Fragments was implemented and specified by Nick Burris and David Bokan, with contributions from Grant Wang. Thanks to Joe Medley for the thorough review of this article. Hero image by Greg Rakozy on Unsplash.

Make your PWA feel more like an app

2020-06-15T00:00:00Z

When you play Progressive Web App buzzword bingo, it is a safe bet to set on "PWAs are just websites". Microsoft's PWA documentation agrees, we say it on this very site, and even PWA nominators Frances Berriman and Alex Russell write so, too. Yes, PWAs are just websites, but they are also way more than that. If done right, a PWA will not feel like a website, but like a "real" app. Now what does it mean to feel like a real app?

To answer this question, let me use the Apple Podcasts app as an example. It is available on macOS on desktop and on iOS (and iPadOS respectively) on mobile. While Podcasts is a media application, the core ideas I illustrate with its help apply to other categories of apps, too.

Apple Podcasts on iPhone and on macOS (Source).

Capable of running offline #

If you take a step back and think of the platform-specific applications you have on your mobile phone or desktop computer, one thing clearly stands out: you never get nothing. In the Podcasts app, even if I am offline, there is always something. When there is no network connection, the app naturally still opens. The Top Charts section does not show any content, but instead falls back to a Can't connect right now message paired with a Retry button. It may not be the most welcoming experience, but I get something.

Podcasts app without network connection.

How to do this on the web

The Podcasts app follows the so-called app shell model. All the static content that is needed to show the core app is cached locally, including decorative images like the left-hand menu icons and the core player UI icons. Dynamic content like the Top Charts data is only loaded on demand, with locally cached fallback content available should the loading fail. Read the article The App Shell Model to learn how to apply this architectural model to your web app.

Offline content available and media playable #

While offline, via the left-hand drawer, I can still navigate to the Downloaded section and enjoy downloaded podcast episodes that are ready to be played and are displayed with all metadata like artwork and descriptions.

Downloaded podcast episodes can be played even without network.

How to do this on the web

Previously downloaded media content can be served from the cache, for example using the Serve cached audio and video recipe from the Workbox library. Other content can always be stored in the cache, or in IndexedDB. Read the article Storage for the web for all details and to know when to use what storage technology. If you have data that should be persistently stored without the risk of being purged when the available amount of memory gets low, you can use the Persistent Storage API.

Proactive background downloading #

When I am back online, I can of course search for content with a query like http 203, and when I decide to subscribe to the search result, the HTTP 203 podcast, the latest episode of the series is immediately downloaded, no questions asked.

After subscribing to a podcast, the latest episode is immediately downloaded.

How to do this on the web

Downloading a podcast episode is an operation that potentially can take longer. The Background Fetch API lets you delegate downloads to the browser, which takes care of them in the background. On Android, the browser in turn can even delegate these downloads further on to the operating system, so the browser does not need to be continuously running. Once the download has completed, your app's service worker gets woken up and you can decide what to do with the response.

The Podcasts app integrates naturally with other applications. For example, when I right-click an episode that I like, I can share it to other apps on my device, like the Messages app. It also naturally integrates with the system clipboard. I can right-click any episode and copy a link to it.

Sharing a podcast episode to the Messages app.

How to do this on the web

The Web Share API and the Web Share Target API allow your app to share and receive texts, files, and links to and from other applications on the device. Although it is not yet possible for a web app to add menu items to the operating system's built-in right-click menu, there are lots of other ways to link to and from other apps on the device. With the Async Clipboard API, you can programmatically read and write text and image data (PNG images) to the system clipboard. On Android, you can use the Contact Picker API to select entries from the device's contacts manager. If you offer both a platform-specific app and a PWA, you can use the Get Installed Related Apps API to check if the platform-specific app is installed, in which case you do not need to encourage the user to install the PWA or accept web push notifications.

Background app refreshing #

In the Podcasts app's settings, I can configure the app to download new episodes automatically. Like that, I do not even have to think about it, updated content will always just be there. Magic.

Podcasts configured to check for new podcast episode every hour.

How to do this on the web

The Periodic Background Sync API allows your app to refresh its content regularly in the background without the need for it to be running. This means new content is proactively available, so your users can start delving into it right away whenever they decide.

State synchronized over the cloud #

At the same time, my subscriptions are synchronized across all devices I own. In a seamless world, I do not have to worry about manually keeping my podcast subscriptions in sync. Likewise, I do not have to be afraid that my mobile device's memory will be consumed by episodes I have already listened to on my desktop. The play state is kept in sync, and listened-to episodes are automatically deleted.

State is synchronized over the cloud.

How to do this on the web

Syncing app state data is a task that you can delegate to the Background Sync API. The sync operation itself does not have to happen immediately, just eventually, and maybe even when the user has closed the app again already.

Hardware media key controls #

When I am busy with another application, say, reading a news page in the Chrome browser, I can still control the Podcasts app with the media keys on my laptop. There is no need to switch to the app just to skip forward or backward.

The media keys allow for controlling the Podcasts app (Source).

How to do this on the web

Media keys are supported by the Media Session API. Like that, users can make use of the hardware media keys on their physical keyboards, headphones, or even control the web app from the software media keys on their smartwatches. An additional idea to smooth seeking operations is to send a vibration pattern when the user seeks over a significant part of the content, for example, passing the opening credits or a chapter boundary.

Multitasking and app shortcut #

Of course I can always multitask back to the Podcasts app from anywhere. The app has a clearly distinguishable icon that I can also put on my desktop or application dock so Podcasts can be launched immediately when I feel like it.

Multitasking back to the Podcasts app.

How to do this on the web

Progressive Web Apps on both desktop and mobile can be installed to the home screen, start menu, or application dock. Installation can happen based on a proactive prompt, or fully controlled by the app developer. The article What does it take to be installable? covers everything you need to know. When multitasking, PWAs appear independent from the browser.

The most common app actions, Search for new content and Check for New Episodes, are available right from the context menu of the app in the Dock. Via the Options menu, I can also decide to open the app at login time.

Quick actions are immediately available right from the app icon.

How to do this on the web

By specifying app icon shortcuts in the PWA's web app manifest, you can register quick routes to common tasks that users can reach directly from the app icon. On operating systems like macOS, users can also right-click the app icon and set the app to launch at login time. There is ongoing work on a proposal for run on login.

Act as default app #

Other iOS applications and even websites or emails can integrate with the Podcasts app by leveraging the podcasts:// URL scheme. If I follow a link like podcasts://podcasts.apple.com/podcast/the-css-podcast/id1042283903 while in the browser, I am brought right into the Podcasts app and can decide to subscribe or listen to the podcast.

The Podcasts app can be opened right from the browser.

How to do this on the web

Handling fully custom URL schemes is not yet possible, but there is ongoing work on a proposal for URL Protocol Handling for PWAs. Currently, registerProtocolHandler() with a web+ scheme prefix is the best alternative.

Local file system integration #

You may not immediately think of it, but the Podcasts app naturally integrates with the local file system. When I download a podcast episode, on my laptop it is stored in ~/Library/Group Containers/243LU875E5.groups.com.apple.podcasts/Library/Cache. Unlike, say ~/Documents, this directory is of course not meant to be accessed directly by regular users, but it is there. Other storage mechanisms than files are referenced in the offline content section.

Podcast episodes are stored in a special system app folder.

How to do this on the web

The File System Access API enables developers to get access to the local file system of the device. You can use it directly or via the browser-fs-access support library that transparently provides a fallback for browsers that do not support the API. For security reasons, system directories are not web-accessible.

Platform look and feel #

There is a more subtle thing that is self-evident for an iOS application like Podcasts: none of the text labels are selectable and all text blends in with the system font of the machine. Also my choice of system color theme (dark mode) is respected.

The Podcasts app supports light and dark mode.

The app uses the default system font.

How to do this on the web

By leveraging the user-select CSS property with the value of none, you can protect UI elements from being accidentally selected. Be sure, though, to not abuse this property for making app contents unselectable. It should only be used for UI elements like button texts, etc. The system-ui value for the font-family CSS property allows you to specify the default UI font of the system to be used for your app. Finally, your app can obey to the user's color scheme preference by respecting their prefers-color-scheme choice, with an optional dark mode toggle to override it. Another thing to decide on might be what the browser should do when reaching the boundary of a scrolling area, for example, to implement custom pull to refresh. This is possible with the overscroll-behavior CSS property.

Customized title bar #

When you look at the Podcasts app window, you notice that it does not have a classic integrated title bar and toolbar, like, for example, the Safari browser window, but a customized experience that looks like a sidebar docked to the main player window.

Customized title bars of Safari and Podcasts.

How to do this on the web

While not currently possible, title bar customization is being worked on at the moment. You can (and should), however, specify the display and the theme-color properties of the web app manifest to determine the look and feel of your application window and to decide which default browser controls—potentially none of them—should be shown.

Snappy animations #

In-app animations are snappy and smooth in Podcasts. For example, when I open the Episode Notes drawer on the right, it elegantly slides in. When I remove one episode from my downloads, the remaining episodes float up and consume the screen real estate that was freed by the deleted episode.

In-app animations like when opening a drawer are snappy.

How to do this on the web

Performant animations on the web are certainly possible if you take into account a number of best practices outlined in the article Animations and Performance. Scroll animations as commonly seen in paginated content or media carousels can be massively improved by using the CSS Scroll Snap feature. For full control, you can use the Web Animations API.

Content surfaced outside of app #

The Podcasts app on iOS can surface content in other locations than the actual application, for example, in the system's Widgets view, or in the form of a Siri Suggestion. Having proactive, usage-based calls-to-action that just require a tap to interact with can greatly increase the re-engagement rate of an app like Podcasts.

App content is surfaced outside of the main Podcasts app.

How to do this on the web

The Content Index API allows your application to tell the browser which content of the PWA is available offline. This allows the browser to surface this content outside of the main app. By marking up interesting content in your app as suitable for speakable audio playback and by using structured markup in general, you can help search engines and virtual assistants like the Google Assistant present your offerings in an ideal light.

When a podcast episode is playing, the Podcasts app shows a beautiful control widget on the lock screen that features metadata like the episode artwork, the episode title, and the podcast name.

Media playing in the app can be controlled from the lock screen.

How to do this on the web

The Media Session API lets you specify metadata like artwork, track titles, etc. that then gets displayed on the lock screen, smartwatches, or other media widgets in the browser.

Push notifications #

Push notifications have become a bit of an annoyance on the web (albeit notification prompts are a lot quieter now). But if used properly, they can add a lot of value. For example, the iOS Podcasts app can optionally notify me of new episodes of podcasts I am subscribed to or recommend new ones, as well as alert me of new app features.

Apps can send push notifications to inform the user about new content.

How to do this on the web

The Push API allows your app to receive push notifications so you can notify your users about noteworthy events around your PWA. For notifications that should fire at a known time in the future and that do not require a network connection, you can use the Notification Triggers API.

App icon badging #

Whenever there are new episodes available for one of the podcasts I am subscribed to, an app icon badge on the Podcasts home screen icon appears, again encouraging me to re-engage with the app in a way that is not intrusive.

Badges are a subtle way for applications to inform users about new content.

How to do this on the web

You can set app icon badges with the Badging API. This is especially useful when your PWA has some notion of "unread" items or when you need a means to unobtrusively draw the user's attention back to the app.

Media playback takes precedence over energy saver settings #

When podcast media is playing, the screen may turn off, but the system will not enter standby mode. Apps can optionally keep the screen awake, too, for example to display lyrics or captions.

Apps can keep the screen awake.

How to do this on the web

The Screen Wake Lock API allows you to prevent the screen from turning off. Media playback on the web automatically prevents the system from entering standby mode.

App discovery through an app store #

While the Podcasts app is part of the macOS desktop experience, on iOS it needs to be installed from the App Store. A quick search for podcast, podcasts, or apple podcasts immediately turns the app up in the App Store.

Users have learned to discover apps in app stores.

How to do this on the web

While Apple does not allow PWAs on the App Store, on Android, you can submit your PWA wrapped in a Trusted Web Activity. The bubblewrap script makes this a painless operation. This script is also what internally powers PWABuilder's Android app export feature, which you can use without touching the command line.

Feature summary #

The table below shows a compact overview of all features and provides a list of useful resources for realizing them on the web.

Feature	Useful resources for doing this on the web
Capable of running offline	App shell model
Offline content available and media playable	Serve cached audio and video Workbox library Storage API Persistent Storage API
Proactive background downloading	Background Fetch API
Sharing to and interacting with other applications	Web Share API Web Share Target API Async Clipboard API Contact Picker API Get Installed Related Apps API
Background app refreshing	Periodic Background Sync API
State synchronized over the cloud	Background Sync API
Hardware media key controls	Media Session API
Multitasking and app shortcut	Installability criteria
Quick actions in context menu	App icon shortcuts Run on login (early stage)
Act as default app	URL protocol handling (early stage) `registerProtocolHandler()`
Local file system integration	File System Access API browser-fs-access library
Platform look and feel	`user-select: none` `font-family: system-ui` `prefers-color-scheme` Dark mode toggle
Customized title bar	Title bar customization (early stage) Display mode Theme color
Snappy animations	Animations and performance tips CSS Scroll Snap Web Animations API
Content surfaced outside of app	Content Index API Speakable content Structured markup
Lock screen media control widget	Media Session API
Push notifications	Push API Notification Triggers API
App icon badging	Badging API
Media playback trumps energy saver settings	Screen Wake Lock API
App discovery through an app store	Trusted Web Activity `bubblewrap` library PWABuilder tool

Conclusion #

PWAs have come a long way since their introduction in 2015. In the context of Project Fugu 🐡, the cross-company Chromium team is working on closing the last remaining gaps. By following even only some of the pieces of advice in this article, you can piece by piece get closer to that app-like feeling and make your users forget that they are dealing with "just a website", because, honestly, most of them do not care how your app is built (and why should they?), as long as it feels like a real app.

Acknowledgements #

This article was reviewed by Kayce Basques, Joe Medley, Joshua Bell, Dion Almaer, Ade Oshineye, Pete LePage, Sam Thorogood, Reilly Grant, and Jeffrey Yasskin.

Wake Lock API case study: 300% increase in purchase intent indicators on BettyCrocker.com

2020-05-19T00:00:00Z

For nearly a century, Betty Crocker has been America's source for modern cooking instruction and trusted recipe development. Launched in 1997, their site BettyCrocker.com today receives more than 12 million visitors per month. After they implemented the Wake Lock API, their indicators of purchase intent were about 300% higher for wake lock users compared to all users.

The retired iOS and Android apps #

Released to much fanfare in 2014, Betty Crocker recently took their apps out of the Apple App Store and the Google Play Store after they had been deprioritized. For a long time, the Betty Crocker team has preferred adding new features to the mobile site instead of the iOS/Android apps. The technical platform the iOS/Android apps were created on was outdated, and the business did not have the resources to support updating and maintaining the apps moving forward. The web app also was objectively way bigger traffic-wise, more modern, and easier to enhance.

The iOS/Android apps did have one killer feature, though, that their users loved:

Millennial cooking pro tip: the @BettyCrocker mobile app doesn't dim or lock when you're following a recipe. —@AvaBeilke

80% of people cook with a device in the kitchen, but screen dimming and locking is a problem. What did @BettyCrocker do? Updated their app to NOT dim when users are in a recipe. —@Katie_Tweedy_

Bringing the killer feature to the web with the Wake Lock API #

When cooking with a device, there is nothing more frustrating than having to touch the screen with messy hands or even your nose when the screen turns off. Betty Crocker asked themselves how they could port the killer feature of their iOS/Android apps over to the web app. This is when they learned about Project Fugu and the Wake Lock API.

The Wake Lock API provides a way to prevent the device from dimming or locking the screen. This capability enables new experiences that, until now, required an iOS/Android app. The Wake Lock API reduces the need for hacky and potentially power-hungry workarounds.

Requesting a wake lock #

To request a wake lock, you need to call the navigator.wakeLock.request() method that returns a WakeLockSentinel object. You will use this object as a sentinel value. The browser can refuse the request for various reasons (for example, because the battery is too low), so it is a good practice to wrap the call in a try…catch statement.

Releasing a wake lock #

You also need a way to release a wake lock, which is achieved by calling the release() method of the WakeLockSentinel object. If you want to automatically release the wake lock after a certain period of time has passed, you can use window.setTimeout() to call release(), as shown in the example below.

// The wake lock sentinel.
let wakeLock = null;

// Function that attempts to request a wake lock.
const requestWakeLock = async () => {
  try {
    wakeLock = await navigator.wakeLock.request('screen');
    wakeLock.addEventListener('release', () => {
      console.log('Wake Lock was released');
    });
    console.log('Wake Lock is active');
  } catch (err) {
    console.error(`${err.name}, ${err.message}`);
  }
};

// Request a wake lock…
await requestWakeLock();
// …and release it again after 5s.
window.setTimeout(() => {
  wakeLock.release();
  wakeLock = null;
}, 5000);

The implementation #

With the new web app, users should be enabled to easily navigate through a recipe, complete steps, and even walk away without the screen locking. To achieve this goal, the team first built a quick front-end prototype as a proof of concept and to gather UX input.

After the prototype proved useful, they designed a Vue.js component that could be shared across all their brands (BettyCrocker, Pillsbury, and Tablespoon). Even though only Betty Crocker had iOS and Android apps, the three sites do have a shared code base, so they were able to implement the component once, and deploy it everywhere, as shown in the screenshots below.

BettyCrocker.com wake lock toggle.

Pillsbury.com wake lock toggle.

Tablespoon.com wake lock toggle.

When developing the component based on the new site's modernized framework, there was a strong focus on the ViewModel layer of the MVVM pattern. The team also programmed with interoperability in mind to enable functionality on legacy and new frameworks of the site.

To keep track of viewability and usability, Betty Crocker integrated analytics tracking for core events in the wake lock lifecycle. The team utilized feature management to deploy the wake lock component to a single site for initial production rollout, and then deployed the feature to the rest of the sites after monitoring usage and page health. They continue to monitor analytics data based on the usage of this component.

As a failsafe for users, the team created a forced timeout to disable the wake lock after one hour of inactivity. The final implementation they settled on was in the short-term a toggle switch on all recipe pages across their sites. In the long-term, they envision a revamped recipe page view.

The wake lock container #

var wakeLockControl = () => {
  return import(/* webpackChunkName: 'wakeLock' */ './wakeLock');
};

export default {
  components: {
    wakeLockControl: wakeLockControl,
  },
  data() {
    return {
      config: {},
      wakeLockComponent: '',
    };
  },
  methods: {
    init: function(config) {
      this.config = config || {};
      if ('wakeLock' in navigator && 'request' in navigator.wakeLock) {
        this.wakeLockComponent = 'wakeLockControl';
      } else {
        console.log('Browser not supported');
      }
    },
  },
};

The wake lock component #

<template>
  <div class="wakeLock">
    <div class="textAbove"></div>
    <label class="switch" :aria-label="settingsInternal.textAbove">
      <input type="checkbox" @change="onChange()" v-model="isChecked">
      <span class="slider round"></span>
    </label>
  </div>
</template>

<script type="text/javascript">
  import debounce from 'lodash.debounce';

  const scrollDebounceMs = 1000;

  export default {
    props: {
      settings: { type: Object },
    },
    data() {
      return {
        settingsInternal: this.settings || {},
        isChecked: false,
        wakeLock: null,
        timerId: 0,
      };
    },
    created() {
      this.$_raiseAnalyticsEvent('Wake Lock Toggle Available');
    },
    methods: {
      onChange: function() {
        if (this.isChecked) {
          this.$_requestWakeLock();
        } else {
          this.$_releaseWakeLock();
        }
      },
      $_requestWakeLock: async function() {
        try {
          this.wakeLock = await navigator.wakeLock.request('screen');
          //Start new timer
          this.$_handleAbortTimer();
          //Only add event listeners after wake lock is successfully enabled
          document.addEventListener(
            'visibilitychange',
            this.$_handleVisibilityChange,
          );
          window.addEventListener(
            'scroll',
            debounce(this.$_handleAbortTimer, scrollDebounceMs),
          );
          this.$_raiseAnalyticsEvent('Wake Lock Toggle Enabled');
        } catch (e) {
          this.isChecked = false;
        }
      },
      $_releaseWakeLock: function() {
        try {
          this.wakeLock.release();
          this.wakeLock = null;
          //Clear timer
          this.$_handleAbortTimer();
          //Clean up event listeners
          document.removeEventListener(
            'visibilitychange',
            this.$_handleVisibilityChange,
          );
          window.removeEventListener(
            'scroll',
            debounce(this.$_handleAbortTimer, scrollDebounceMs),
          );
        } catch (e) {
          console.log(`Wake Lock Release Error: ${e.name}, ${e.message}`);
        }
      },
      $_handleAbortTimer: function() {
        //If there is an existing timer then clear it and set to zero
        if (this.timerId !== 0) {
          clearTimeout(this.timerId);
          this.timerId = 0;
        }
        //Start new timer; Will be triggered from toggle enabled or scroll event
        if (this.isChecked) {
          this.timerId = setTimeout(
            this.$_releaseWakeLock,
            this.settingsInternal.timeoutDurationMs,
          );
        }
      },
      $_handleVisibilityChange: function() {
        //Handle navigating away from page/tab
        if (this.isChecked) {
          this.$_releaseWakeLock();
          this.isChecked = false;
        }
      },
      $_raiseAnalyticsEvent: function(eventType) {
        let eventParams = {
          EventType: eventType,
          Position: window.location.pathname || '',
        };
        Analytics.raiseEvent(eventParams);
      },
    },
  };
</script>

Results #

The Vue.js component has been deployed on all three sites and delivered great results. During the period from December 10th, 2019 to January 10th, 2020, BettyCrocker.com reported the following metrics:

Of all Betty Crocker users with a browser compatible with the Wake Lock API, 3.5% of them enabled the feature immediately, making it a top-5 action.
The session duration for users who enabled the wake lock was 3.1× longer than for users who did not.
The bounce rate for users who enabled the wake lock was 50% lower than for those not using the wake lock feature.
Indicators of purchase intent were about 300% higher for wake lock users compared to all users.

3.1_×

Longer session duration

50_%

Lower bounce rate

300_%

Higher purchase intent indicators

Conclusions #

Betty Crocker has seen fantastic results using the Wake Lock API. You can test the feature yourself by searching for your favorite recipe on any of their sites (BettyCrocker, Pillsbury, or Tablespoon) and enabling the Prevent your screen from going dark while you cook toggle.

Use cases for wake locks do not stop at recipe sites. Other examples are boarding pass or ticket apps that need to keep the screen on until the barcode has been scanned, kiosk-style apps that keep the screen on continuously, or web-based presentation apps that prevent the screen from sleeping during a presentation.

We have compiled everything you need to know about the Wake Lock API in a comprehensive article on this very site. Happy reading, and happy cooking!

Acknowledgements #

The person kneading dough photo courtesy of Julian Hochgesang on Unsplash.

Improved dark mode default styling with the `color-scheme` CSS property and the corresponding meta tag

2020-04-08T00:00:00Z

Background #

The `prefers-color-scheme` user preference media feature #

The prefers-color-scheme user preference media feature gives developers full control over their pages' appearances. If you are unfamiliar with it, please read my article prefers-color-scheme: Hello darkness, my old friend, where I documented everything I know about creating amazing dark mode experiences.

One puzzle piece that was only mentioned briefly in the article is the color-scheme CSS property and the corresponding meta tag of the same name. They both make your life as a developer easier by allowing you to opt your page in to theme-specific defaults of the user agent stylesheet, such as, for example, form controls, scroll bars, as well as CSS system colors. At the same time, this feature prevents browsers from applying any transformations on their own.

Browser support #

`prefers-color-scheme` #

Browser support

Chrome 76, Supported 76
Firefox 67, Supported 67
Edge 79, Supported 79
Safari 12.1, Supported 12.1

Source

`color-scheme` #

Browser support

Chrome 81, Supported 81
Firefox 96, Supported 96
Edge 81, Supported 81
Safari 13, Supported 13

Source

The user agent stylesheet #

Before I continue, let me briefly describe what a user agent stylesheet is. Most of the time, you can think of the word user agent (UA) as a fancy way to say browser. The UA stylesheet determines the default look and feel of a page. As the name suggests, a UA stylesheet is something that depends on the UA in question. You can have a look at Chrome's (and Chromium's) UA stylesheet and compare it to Firefox's or Safari's (and WebKit's). Typically, UA stylesheets agree on the majority of things. For example, they all make links blue, general text black, and background color white, but there are also important (and sometimes annoying) differences, for instance, how they style form controls.

Have a closer look at WebKit's UA stylesheet and what it does regarding dark mode. (Do a full text search for "dark" in the stylesheet.) The default provided by the stylesheet changes based on whether dark mode is on or off. To illustrate this, here is one such CSS rule using the :matches pseudo class and WebKit-internal variables like -apple-system-control-background, as well as the WebKit-internal preprocessor directive #if defined:

input,
input:matches([type="password"], [type="search"]) {
  -webkit-appearance: textfield;
  #if defined(HAVE_OS_DARK_MODE_SUPPORT) &&
      HAVE_OS_DARK_MODE_SUPPORT
    color: text;
    background-color: -apple-system-control-background;
  #else
    background-color: white;
  #endif
  /* snip */
}

You will notice some non-standard values for the color and background-color properties above. Neither text nor -apple-system-control-background are valid CSS colors. They are WebKit-internal semantic colors.

Turns out, CSS has standardized semantic system colors. They are specified in CSS Color Module Level 4. For example, Canvas (not to be confused with the <canvas> tag) is for the background of application content or documents, whereas CanvasText is for text in application content or documents. The two go together and should not be used in isolation.

UA stylesheets can use either their own proprietary or the standardized semantic system colors, to determine how HTML elements should be rendered by default. If the operating system is set to dark mode or uses a dark theme, CanvasText (or text) would be conditionally set to white, and Canvas (or -apple-system-control-background) would be set to black. The UA stylesheet then assigns the following CSS only once, and covers both light and dark mode.

/**
  Not actual UA stylesheet code.
  For illustrative purposes only.
*/
body {
  color: CanvasText;
  background-color: Canvas
}

The `color-scheme` CSS property #

The CSS Color Adjustment Module Level 1 specification introduces a model and controls over automatic color adjustment by the user agent with the objective of handling user preferences such as dark mode, contrast adjustment, or specific desired color schemes.

The color-scheme property defined therein allows an element to indicate which color schemes it is comfortable being rendered with. These values are negotiated with the user's preferences, resulting in a chosen color scheme that affects user interface (UI) things such as the default colors of form controls and scroll bars, as well as the used values of the CSS system colors. The following values are currently supported:

normal Indicates that the element is not aware of color schemes at all, and so the element should be rendered with the browser's default color scheme.
[ light | dark ]+ Indicates that the element is aware of and can handle the listed color schemes, and expresses an ordered preference between them.

In this list, light represents a light color scheme, with light background colors and dark foreground colors, whereas dark represents the opposite, with dark background colors and light foreground colors.

For all elements, rendering with a color scheme should cause the colors used in all browser-provided UI for the element to match with the intent of the color scheme. Examples are scroll bars, spellcheck underlines, form controls, etc.

On the :root element, rendering with a color scheme additionally must affect the surface color of the canvas (that is, the global background color), the initial value of the color property, and the used values of the system colors, and should also affect the viewport's scroll bars.

/*
  The page supports both dark and light color schemes,
  and the page author prefers dark.
*/
:root {
  color-scheme: dark light;
}

The `color-scheme` meta tag #

Honoring the color-scheme CSS property requires the CSS to be first downloaded (if it is referenced via <link rel="stylesheet">) and to be parsed. To aid user agents in rendering the page background with the desired color scheme immediately, a color-scheme value can also be provided in a <meta name="color-scheme"> element.

<!--
  The page supports both dark and light color schemes,
  and the page author prefers dark.
-->
<meta name="color-scheme" content="dark light">

Combining `color-scheme` and `prefers-color-scheme` #

Since both the meta tag and the CSS property (if applied to the :root element) eventually result in the same behavior, I always recommend specifying the color scheme via the meta tag, so the browser can adopt to the preferred scheme faster.

While for absolute baseline pages no additional CSS rules are necessary, in the general case you should always combine color-scheme with prefers-color-scheme. For example, the proprietary WebKit CSS color -webkit-link, used by WebKit and Chrome for the classic link blue rgb(0,0,238), has an insufficient contrast ratio of 2.23:1 on a black background and fails both the WCAG AA as well as the WCAG AAA requirements.

I have opened bugs for Chrome, WebKit, and Firefox as well as a meta issue in the HTML Standard to get this fixed.

Interplay with `prefers-color-scheme` #

The interplay of the color-scheme CSS property and the corresponding meta tag with the prefers-color-scheme user preference media feature may seem confusing at first. In fact, they play together really well. The most important thing to understand is that color-scheme exclusively determines the default appearance, whereas prefers-color-scheme determines the stylable appearance. To make this clearer, assume the following page:

<head>
  <meta name="color-scheme" content="dark light">
  <style>
    fieldset {
      background-color: gainsboro;
    }
    @media (prefers-color-scheme: dark) {
      fieldset {
        background-color: darkslategray;
      }
    }
  </style>
</head>
<body>
  <p>
    Lorem ipsum dolor sit amet, legere ancillae ne vis.
  </p>
  <form>
    <fieldset>
      <legend>Lorem ipsum</legend>
      <button type="button">Lorem ipsum</button>
    </fieldset>
  </form>
</body>

The inline CSS code on the page sets the <fieldset> element's background-color to gainsboro in the general case, and to darkslategray if the user prefers a dark color scheme according to the prefers-color-scheme user preference media feature.

Via the <meta name="color-scheme" content="dark light"> element, the page tells the browser that it supports a dark and a light theme, with a preference for a dark theme.

Depending on whether the operating system is set to dark or light mode, the whole page appears light on dark, or vice versa, based on the user agent stylesheet. There is no additional developer-provided CSS involved to change the paragraph text or the background color of the page.

Note how the <fieldset> element's background-color changes based on whether dark mode is enabled, following the rules in the developer-provided inline stylesheet on the page. It is either gainsboro or darkslategray.

Light mode: Styles specified by the developer and the user agent. The text is black and the background is white as per the user agent stylesheet. The <fieldset> element's background-color is gainsboro as per the inlined developer stylesheet.

Dark mode: Styles specified by the developer and the user agent. The text is white and the background is black as per the user agent stylesheet. The <fieldset> element's background-color is darkslategray as per the inlined developer stylesheet.

The <button> element's appearance is controlled by the user agent stylesheet. Its color is set to the ButtonText system color, and its background-color and the four border-colors are set to the system color ButtonFace.

Light mode: The background-color and the various border-colors are set to the ButtonFace system color.

Now note how the <button> element's border-color changes. The computed value for the border-top-color and the border-bottom-color switches from rgba(0, 0, 0, 0.847) (blackish) to rgba(255, 255, 255, 0.847) (whitish), since the user agent updates ButtonFace dynamically based on the color scheme. The same applies for the <button> element's color that is set to the corresponding system color ButtonText.

Light mode: The computed values of the border-top-color and the border-bottom-color that are both set to ButtonFace in the user agent stylesheet are now rgba(0, 0, 0, 0.847).

Dark mode: The computed values of the border-top-color and the border-bottom-color that are both set to ButtonFace in the user agent stylesheet are now rgba(255, 255, 255, 0.847).

Demo #

You can see the effects of color-scheme applied to a large number of HTML elements in a demo on Glitch. The demo deliberately shows the WCAG AA and WCAG AAA violation with the link colors mentioned in the warning above.

The demo toggled to color-scheme: light.

The demo toggled to color-scheme: dark. Note the WCAG AA and WCAG AAA violation with the link colors.

Acknowledgements #

The color-scheme CSS property and the corresponding meta tag were implemented by Rune Lillesveen. Rune is also a co-editor of the CSS Color Adjustment Module Level 1 specification. Hero image by Philippe Leone on Unsplash.

Adaptive icon support in PWAs with maskable icons

2019-12-19T00:00:00Z

What are maskable icons? #

If you've installed a Progressive Web App on a recent Android phone, you might notice the icon shows up with a white background. Android Oreo introduced adaptive icons, which display app icons in a variety of shapes across different device models. Icons that don't follow this new format are given white backgrounds.

Transparent PWA icons appear inside white circles on Android.

Maskable icons are a new icon format that give you more control and let your Progressive Web App use adaptive icons. If you supply a maskable icon, your icon can fill up the entire shape and look great on all Android devices. Firefox and Chrome have recently added support for this new format, and you can adopt it in your apps.

Maskable icons cover the entire circle instead.

Are my current icons ready? #

Since maskable icons need to support a variety of shapes, you supply an opaque image with some padding that the browser can crop to the required shape and size. It's best not to rely on any particular shape, since the ultimate shape varies by browser and platform.

Different platform-specific shapes.

Luckily, there's a well-defined and standardized "minimum safe zone" that all platforms respect. The important parts of your icon, such as your logo, should be within a circular area in the center of the icon with a radius equal to 40% of the icon width. The outer 10% edge may be cropped.

You can check which parts of your icons land within the safe zone with Chrome DevTools. With your Progressive Web App open, launch DevTools and navigate to the Application panel. In the Icons section, you can choose to Show only the minimum safe area for maskable icons. Your icons will be trimmed so that only the safe area is visible. If your logo is visible within this safe area, you're good to go.

The Applications panel.

To test your maskable icon with the variety of Android shapes, use the Maskable.app tool Tiger created. Open an icon, then Maskable.app will let you try various shapes and sizes, and you can share the preview with others on your team.

How do I adopt maskable icons? #

If you want to create a maskable icon based on an existing icon, you can use the Maskable.app Editor. Upload your icon, adjust the color and size, then export the image.

Creating icons in the Maskable.app Editor.

Once you've created a maskable icon and tested it in DevTools, you'll need to update your web app manifest to point to the new asset. The web app manifest provides information about your web app in a JSON file, and includes an icons array.

For the inclusion of maskable icons, the purpose field tells the browser how your icon should be used. By default, icons will have a purpose of "any". These icons will be resized on top of a white background on Android.

{
  …
  "icons": [
    …
    {
      "src": "path/to/regular_icon.png",
      "sizes": "196x196",
      "type": "image/png",
      "purpose": "any"
    },
    {
      "src": "path/to/maskable_icon.png",
      "sizes": "196x196",
      "type": "image/png",
      "purpose": "maskable" // <-- New property value `"maskable"`
    },
    …
  ],
  …
}

Maskable icons should use a different value for purpose: "maskable". This indicates that an image is meant to be used with icon masks, giving you more control over the result. This way, your icons will not have a white background. You can also specify multiple space-separated purposes (for example, "any maskable"), if you want your maskable icon to be used without a mask on other devices.

With this, you can go forth and create your own maskable icons, making sure your app looks great edge-to-edge (and for what it's worth, circle-to-circle, oval-to-oval 😄).

Acknowledgements #

This article was reviewed by Joe Medley.

prefers-color-scheme: Hello darkness, my old friend

2019-06-27T00:00:00Z

Introduction #

Dark mode before Dark Mode #

Green screen (Source)

We have gone full circle with dark mode. In the dawn of personal computing, dark mode wasn't a matter of choice, but a matter of fact: Monochrome CRT computer monitors worked by firing electron beams on a phosphorescent screen and the phosphor used in early CRTs was green. Because text was displayed in green and the rest of the screen was black, these models were often referred to as green screens.

Dark-on-white (Source)

The subsequently introduced Color CRTs displayed multiple colors through the use of red, green, and blue phosphors. They created white by activating all three phosphors simultaneously. With the advent of more sophisticated WYSIWYG desktop publishing, the idea of making the virtual document resemble a physical sheet of paper became popular.

The WorldWideWeb browser (Source)

This is where dark-on-white as a design trend started, and this trend was carried over to the early document-based web. The first ever browser, WorldWideWeb (remember, CSS wasn't even invented yet), displayed webpages this way. Fun fact: the second ever browser, Line Mode Browser—a terminal-based browser—was green on dark. These days, web pages and web apps are typically designed with dark text on a light background, a baseline assumption that is also hard-coded in user agent stylesheets, including Chrome's.

Smartphone used in bed (Source: Unsplash)

The days of CRTs are long over. Content consumption and creation has shifted to mobile devices that use backlit LCD or energy-saving AMOLED screens. Smaller and more transportable computers, tablets, and smartphones led to new usage patterns. Leisure tasks like web browsing, coding for fun, and high-end gaming frequently happen after-hours in dim environments. People even enjoy their devices in their beds at night-time. The more people use their devices in the dark, the more the idea of going back to the roots of light-on-dark becomes popular.

Why dark mode #

Dark mode for aesthetic reasons #

When people get asked why they like or want dark mode, the most popular response is that "it's easier on the eyes," followed by "it's elegant and beautiful." Apple in their Dark Mode developer documentation explicitly writes: "The choice of whether to enable a light or dark appearance is an aesthetic one for most users, and might not relate to ambient lighting conditions."

System 7 CloseView (Source)

Dark mode as an accessibility tool #

There are also people who actually need dark mode and use it as another accessibility tool, for example, users with low vision. The earliest occurrence of such an accessibility tool I could find is System 7's CloseView feature, which had a toggle for Black on White and White on Black. While System 7 supported color, the default user interface was still black-and-white.

These inversion-based implementations demonstrated their weaknesses once color was introduced. User research by Szpiro et al. on how people with low vision access computing devices showed that all interviewed users disliked inverted images, but that many preferred light text on a dark background. Apple accommodates for this user preference with a feature called Smart Invert, which reverses the colors on the display, except for images, media, and some apps that use dark color styles.

A special form of low vision is Computer Vision Syndrome, also known as Digital Eye Strain, which is defined as "the combination of eye and vision problems associated with the use of computers (including desktop, laptop, and tablets) and other electronic displays (e.g. smartphones and electronic reading devices)." It has been proposed that the use of electronic devices by adolescents, particularly at night time, leads to an increased risk of shorter sleep duration, longer sleep-onset latency, and increased sleep deficiency. Additionally, exposure to blue light has been widely reported to be involved in the regulation of circadian rhythm and the sleep cycle, and irregular light environments may lead to sleep deprivation, possibly affecting mood and task performance, according to research by Rosenfield. To limit these negative effects, reducing blue light by adjusting the display color temperature through features like iOS' Night Shift or Android's Night Light can help, as well as avoiding bright lights or irregular lights in general through dark themes or dark modes.

Dark mode power savings on AMOLED screens #

Finally, dark mode is known to save a lot of energy on AMOLED screens. Android case studies that focused on popular Google apps like YouTube have shown that the power savings can be up to 60%. The video below has more details on these case studies and the power savings per app.

Activating dark mode in the operating system #

Now that I have covered the background of why dark mode is such a big deal for many users, let's review how you can support it.

Android Q dark theme settings

Operating systems that support a dark mode or dark theme typically have an option to activate it somewhere in the settings. On macOS X, it's in the system preference's General section and called Appearance (screenshot), and on Windows 10, it's in the Colors section and called Choose your color (screenshot). For Android Q, you can find it under Display as a Dark Theme toggle switch (screenshot), and on iOS 13, you can change the Appearance in the Display & Brightness section of the settings (screenshot).

The `prefers-color-scheme` media query #

One last bit of theory before I get going. Media queries allow authors to test and query values or features of the user agent or display device, independent of the document being rendered. They are used in the CSS @media rule to conditionally apply styles to a document, and in various other contexts and languages, such as HTML and JavaScript. Media Queries Level 5 introduces so-called user preference media features, that is, a way for sites to detect the user's preferred way to display content.

The prefers-color-scheme media feature is used to detect if the user has requested the page to use a light or dark color theme. It works with the following values:

light: Indicates that the user has notified the system that they prefer a page that has a light theme (dark text on light background).
dark: Indicates that the user has notified the system that they prefer a page that has a dark theme (light text on dark background).

Supporting dark mode #

Finding out if dark mode is supported by the browser #

As dark mode is reported through a media query, you can easily check if the current browser supports dark mode by checking if the media query prefers-color-scheme matches at all. Note how I don't include any value, but purely check if the media query alone matches.

if (window.matchMedia('(prefers-color-scheme)').media !== 'not all') {
  console.log('🎉 Dark mode is supported');
}

At the time of writing, prefers-color-scheme is supported on both desktop and mobile (where available) by Chrome and Edge as of version 76, Firefox as of version 67, and Safari as of version 12.1 on macOS and as of version 13 on iOS. For all other browsers, you can check the Can I use support tables.

Learning about a user's preferences at request time #

The Sec-CH-Prefers-Color-Scheme client hint header allows sites to obtain the user's color scheme preferences optionally at request time, allowing servers to inline the right CSS and therefore avoid a flash of incorrect color theme.

Dark mode in practice #

Let's finally see how supporting dark mode looks like in practice. Just like with the Highlander, with dark mode there can be only one: dark or light, but never both! Why do I mention this? Because this fact should have an impact on the loading strategy. Please don't force users to download CSS in the critical rendering path that is for a mode they don't currently use. To optimize load speed, I have therefore split my CSS for the example app that shows the following recommendations in practice into three parts in order to defer non-critical CSS:

style.css that contains generic rules that are used universally on the site.
dark.css that contains only the rules needed for dark mode.
light.css that contains only the rules needed for light mode.

Loading strategy #

The two latter ones, light.css and dark.css, are loaded conditionally with a <link media> query. Initially, not all browsers will support prefers-color-scheme (detectable using the pattern above), which I deal with dynamically by loading the default light.css file via a conditionally inserted <link rel="stylesheet"> element in a minuscule inline script (light is an arbitrary choice, I could also have made dark the default fallback experience). To avoid a flash of unstyled content, I hide the content of the page until light.css has loaded.

<script>
  // If `prefers-color-scheme` is not supported, fall back to light mode.
  // In this case, light.css will be downloaded with `highest` priority.
  if (window.matchMedia('(prefers-color-scheme: dark)').media === 'not all') {
    document.documentElement.style.display = 'none';
    document.head.insertAdjacentHTML(
      'beforeend',
      '<link rel="stylesheet" href="/light.css" onload="document.documentElement.style.display = \'\'">',
    );
  }
</script>
<!--
  Conditionally either load the light or the dark stylesheet. The matching file
  will be downloaded with `highest`, the non-matching file with `lowest`
  priority. If the browser doesn't support `prefers-color-scheme`, the media
  query is unknown and the files are downloaded with `lowest` priority (but
  above I already force `highest` priority for my default light experience).
-->
<link rel="stylesheet" href="/dark.css" media="(prefers-color-scheme: dark)" />
<link
  rel="stylesheet"
  href="/light.css"
  media="(prefers-color-scheme: light)"
/>
<!-- The main stylesheet -->
<link rel="stylesheet" href="/style.css" />

Stylesheet architecture #

I make maximum use of CSS variables, this allows my generic style.css to be, well, generic, and all the light or dark mode customization happens in the two other files dark.css and light.css. Below you can see an excerpt of the actual styles, but it should suffice to convey the overall idea. I declare two variables, -⁠-⁠color and -⁠-⁠background-color that essentially create a dark-on-light and a light-on-dark baseline theme.

/* light.css: 👉 dark-on-light */
:root {
  --color: rgb(5, 5, 5);
  --background-color: rgb(250, 250, 250);
}

/* dark.css: 👉 light-on-dark */
:root {
  --color: rgb(250, 250, 250);
  --background-color: rgb(5, 5, 5);
}

In my style.css, I then use these variables in the body { … } rule. As they are defined on the :root CSS pseudo-class—a selector that in HTML represents the <html> element and is identical to the selector html, except that its specificity is higher—they cascade down, which serves me for declaring global CSS variables.

/* style.css */
:root {
  color-scheme: light dark;
}

body {
  color: var(--color);
  background-color: var(--background-color);
}

In the code sample above, you will probably have noticed a property color-scheme with the space-separated value light dark.

This tells the browser which color themes my app supports and allows it to activate special variants of the user agent stylesheet, which is useful to, for example, let the browser render form fields with a dark background and light text, adjust the scroll bars, or to enable a theme-aware highlight color. The exact details of color-scheme are specified in CSS Color Adjustment Module Level 1.

Everything else is then just a matter of defining CSS variables for things that matter on my site. Semantically organizing styles helps a lot when working with dark mode. For example, rather than -⁠-⁠highlight-yellow, consider calling the variable -⁠-⁠accent-color, as "yellow" may actually not be yellow in dark mode or vice versa. Below is an example of some more variables that I use in my example.

/* dark.css */
:root {
  --color: rgb(250, 250, 250);
  --background-color: rgb(5, 5, 5);
  --link-color: rgb(0, 188, 212);
  --main-headline-color: rgb(233, 30, 99);
  --accent-background-color: rgb(0, 188, 212);
  --accent-color: rgb(5, 5, 5);
}

/* light.css */
:root {
  --color: rgb(5, 5, 5);
  --background-color: rgb(250, 250, 250);
  --link-color: rgb(0, 0, 238);
  --main-headline-color: rgb(0, 0, 192);
  --accent-background-color: rgb(0, 0, 238);
  --accent-color: rgb(250, 250, 250);
}

Full example #

In the following Glitch embed, you can see the complete example that puts the concepts from above into practice. Try toggling dark mode in your particular operating system's settings and see how the page reacts.

Loading impact #

When you play with this example, you can see why I load my dark.css and light.css via media queries. Try toggling dark mode and reload the page: the particular currently non-matching stylesheets are still loaded, but with the lowest priority, so that they never compete with resources that are needed by the site right now.

Site in light mode loads the dark mode CSS with lowest priority.

Site in dark mode loads the light mode CSS with lowest priority.

Site in default light mode on a browser that doesn't support prefers-color-scheme loads the dark mode CSS with lowest priority.

Reacting on dark mode changes #

Like any other media query change, dark mode changes can be subscribed to via JavaScript. You can use this to, for example, dynamically change the favicon of a page or change the <meta name="theme-color"> that determines the color of the URL bar in Chrome. The full example above shows this in action, in order to see the theme color and favicon changes, open the demo in a separate tab.

const darkModeMediaQuery = window.matchMedia('(prefers-color-scheme: dark)');
darkModeMediaQuery.addEventListener('change', (e) => {
  const darkModeOn = e.matches;
  console.log(`Dark mode is ${darkModeOn ? '🌒 on' : '☀️ off'}.`);
});

As of Chromium 93 and Safari 15, you can adjust the color based on a media query with the media attribute of the meta theme color element. The first one that matches will be picked. For example, you could have one color for light mode and another one for dark mode. At the time of writing, you can't define those in your manifest. See w3c/manifest#975 GitHub issue.

<meta
  name="theme-color"
  media="(prefers-color-scheme: light)"
  content="white"
/>
<meta name="theme-color" media="(prefers-color-scheme: dark)" content="black" />

Debugging and testing dark mode #

Emulating `prefers-color-scheme` in DevTools #

Switching the entire operating system's color scheme can get annoying real quick, so Chrome DevTools now allows you to emulate the user's preferred color scheme in a way that only affects the currently visible tab. Open the Command Menu, start typing Rendering, run the Show Rendering command, and then change the Emulate CSS media feature prefers-color-scheme option.

Screenshotting `prefers-color-scheme` with Puppeteer #

Puppeteer is a Node.js library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol. With dark-mode-screenshot, we provide a Puppeteer script that lets you create screenshots of your pages in both dark and light mode. You can run this script as a one-off, or alternatively make it part of your Continuous Integration (CI) test suite.

npx dark-mode-screenshot --url https://googlechromelabs.github.io/dark-mode-toggle/demo/ --output screenshot --fullPage --pause 750

Dark mode best practices #

Avoid pure white #

A small detail you may have noticed is that I don't use pure white. Instead, to prevent glowing and bleeding against the surrounding dark content, I choose a slightly darker white. Something like rgb(250, 250, 250) works well.

Re-colorize and darken photographic images #

If you compare the two screenshots below, you will notice that not only the core theme has changed from dark-on-light to light-on-dark, but that also the hero image looks slightly different. My user research has shown that the majority of the surveyed people prefer slightly less vibrant and brilliant images when dark mode is active. I refer to this as re-colorization.

Hero image slightly darkened in dark mode.

Regular hero image in light mode.

Re-colorization can be achieved through a CSS filter on my images. I use a CSS selector that matches all images that don't have .svg in their URL, the idea being that I can give vector graphics (icons) a different re-colorization treatment than my images (photos), more about this in the next paragraph. Note how I again use a CSS variable, so I can later on flexibly change my filter.

As re-colorization is only needed in dark mode, that is, when dark.css is active, there are no corresponding rules in light.css.

/* dark.css */
--image-filter: grayscale(50%);

img:not([src*='.svg']) {
  filter: var(--image-filter);
}

Customizing dark mode re-colorization intensities with JavaScript #

Not everyone is the same and people have different dark mode needs. By sticking to the re-colorization method described above, I can easily make the grayscale intensity a user preference that I can change via JavaScript, and by setting a value of 0%, I can also disable re-colorization completely. Note that document.documentElement provides a reference to the root element of the document, that is, the same element I can reference with the :root CSS pseudo-class.

const filter = 'grayscale(70%)';
document.documentElement.style.setProperty('--image-filter', value);

Invert vector graphics and icons #

For vector graphics—that in my case are used as icons that I reference via <img> elements—I use a different re-colorization method. While research has shown that people don't like inversion for photos, it does work very well for most icons. Again I use CSS variables to determine the inversion amount in the regular and in the :hover state.

Icons are inverted in dark mode.

Regular icons in light mode.

Note how again I only invert icons in dark.css but not in light.css, and how :hover gets a different inversion intensity in the two cases to make the icon appear slightly darker or slightly brighter, dependent on the mode the user has selected.

/* dark.css */
--icon-filter: invert(100%);
--icon-filter_hover: invert(40%);

img[src*='.svg'] {
  filter: var(--icon-filter);
}

/* light.css */
--icon-filter_hover: invert(60%);

/* style.css */
img[src*='.svg']:hover {
  filter: var(--icon-filter_hover);
}

Use `currentColor` for inline SVGs #

For inline SVG images, instead of using inversion filters, you can leverage the currentColor CSS keyword that represents the value of an element's color property. This lets you use the color value on properties that do not receive it by default. Conveniently, if currentColor is used as the value of the SVG fill or stroke attributes, it instead takes its value from the inherited value of the color property. Even better: this also works for <svg><use href="…"></svg>, so you can have separate resources and currentColor will still be applied in context. Please note that this only works for inline or <use href="…"> SVGs, but not SVGs that are referenced as the src of an image or somehow via CSS. You can see this applied in the demo below.

<!-- Some inline SVG -->
<svg xmlns="http://www.w3.org/2000/svg"
    stroke="currentColor"
>
  […]
</svg>

Smooth transitions between modes #

Switching from dark mode to light mode or vice versa can be smoothed thanks to the fact that both color and background-color are animatable CSS properties. Creating the animation is as easy as declaring two transitions for the two properties. The example below illustrates the overall idea, you can experience it live in the demo.

body {
  --duration: 0.5s;
  --timing: ease;

  color: var(--color);
  background-color: var(--background-color);

  transition: color var(--duration) var(--timing), background-color var(
        --duration
      ) var(--timing);
}

Art direction with dark mode #

While for loading performance reasons in general I recommend to exclusively work with prefers-color-scheme in the media attribute of <link> elements (rather than inline in stylesheets), there are situations where you actually may want to work with prefers-color-scheme directly inline in your HTML code. Art direction is such a situation. On the web, art direction deals with the overall visual appearance of a page and how it communicates visually, stimulates moods, contrasts features, and psychologically appeals to a target audience.

With dark mode, it's up to the judgment of the designer to decide what is the best image at a particular mode and whether re-colorization of images is maybe not good enough. If used with the <picture> element, the <source> of the image to be shown can be made dependent on the media attribute. In the example below, I show the Western hemisphere for dark mode, and the Eastern hemisphere for light mode or when no preference is given, defaulting to the Eastern hemisphere in all other cases. This is of course purely for illustrative purposes. Toggle dark mode on your device to see the difference.

<picture>
  <source srcset="western.webp" media="(prefers-color-scheme: dark)" />
  <source srcset="eastern.webp" media="(prefers-color-scheme: light)" />
  <img src="eastern.webp" />
</picture>

Dark mode, but add an opt-out #

As mentioned in the why dark mode section above, dark mode is an aesthetic choice for most users. In consequence, some users may actually like to have their operating system UI in dark, but still prefer to see their webpages the way they are used to seeing them. A great pattern is to initially adhere to the signal the browser sends through prefers-color-scheme, but to then optionally allow users to override their system-level setting.

The `<dark-mode-toggle>` custom element #

You can of course create the code for this yourself, but you can also just use a ready-made custom element (web component) that I have created right for this purpose. It's called <dark-mode-toggle> and it adds a toggle (dark mode: on/off) or a theme switcher (theme: light/dark) to your page that you can fully customize. The demo below shows the element in action (oh, and I have also 🤫 silently snuck it in all of the other examples above).

<dark-mode-toggle
  legend="Theme Switcher"
  appearance="switch"
  dark="Dark"
  light="Light"
  remember="Remember this"
></dark-mode-toggle>

<dark-mode-toggle> in light mode.

<dark-mode-toggle> in dark mode.

Try clicking or tapping the dark mode controls in the upper right corner in the demo below. If you check the checkbox in the third and the fourth control, see how your mode selection is remembered even when you reload the page. This allows your visitors to keep their operating system in dark mode, but enjoy your site in light mode or vice versa.

Conclusions #

Working with and supporting dark mode is fun and opens up new design avenues. For some of your visitors it can be the difference between not being able to handle your site and being a happy user. There are some pitfalls and careful testing is definitely required, but dark mode is definitely a great opportunity for you to show that you care about all of your users. The best practices mentioned in this post and helpers like the <dark-mode-toggle> custom element should make you confident in your ability to create an amazing dark mode experience. Let me know on Twitter what you create and if this post was useful or also suggestions for improving it. Thanks for reading! 🌒

Resources for the prefers-color-scheme media query:

Resources for the color-scheme meta tag and CSS property:

General dark mode links:

Background research articles for this post:

Acknowledgements #

The prefers-color-scheme media feature, the color-scheme CSS property, and the related meta tag are the implementation work of 👏 Rune Lillesveen. Rune is also a co-editor of the CSS Color Adjustment Module Level 1 spec. I would like to 🙏 thank Lukasz Zbylut, Rowan Merewood, Chirag Desai, and Rob Dodson for their thorough reviews of this article. The loading strategy is the brainchild of Jake Archibald. Emilio Cobos Álvarez has pointed me to the correct prefers-color-scheme detection method. The tip with referenced SVGs and currentColor came from Timothy Hatcher. Finally, I am thankful to the many anonymous participants of the various user studies that have helped shape the recommendations in this article. Hero image by Nathan Anderson.

prefers-reduced-motion: Sometimes less movement is more

2019-03-11T00:00:00Z

Not everyone likes decorative animations or transitions, and some users outright experience motion sickness when faced with parallax scrolling, zooming effects, and so on. The user preference media query prefers-reduced-motion lets you design a motion-reduced variant of your site for users who have expressed this preference.

Browser support

Chrome 74, Supported 74
Firefox 63, Supported 63
Edge 79, Supported 79
Safari 10.1, Supported 10.1

Source

Too much motion in real life and on the web #

The other day, I was ice skating with my kids. It was a lovely day, the sun was shining, and the ice rink was crammed with people ⛸. The only issue with that: I don't cope with crowds well. With so many moving targets, I fail to focus on anything, and end up lost and with a feeling of complete visual overload, almost like staring at an anthill 🐜.

Visual overload in real life.

Occasionally, the same can happen on the web: with flashing ads, fancy parallax effects, surprising reveal animations, autoplaying videos, and so on, the web sometimes can be quite overwhelming… Happily, unlike in real life, there is a solution to that. The CSS media query prefers-reduced-motion lets developers create a variant of a page for users who, well, prefer reduced motion. This can comprise anything from refraining from having autoplaying videos to disabling certain purely decorative effects, to completely redesigning a page for certain users.

Before I dive into the feature, let's take one step back and think of what animations are used for on the web. If you want, you can also skip the background information and jump right into the technical details below.

Animation on the web #

Animation is oftentimes used to provide feedback to the user, for example, to let them know that an action was received and is being processed. For example, on a shopping website, a product could be animated to "fly" into a virtual shopping cart, depicted as an icon in the top-right corner of the site.

Another use case involves using motion to hack user perception by using a mixture of skeleton screens, contextual metadata, and low quality image previews to occupy a lot of the user's time and make the whole experience feel faster. The idea is to give context to the user of what's coming and meanwhile load in things as quickly as possible.

Finally, there are decorative effects like animated gradients, parallax scrolling, background videos, and several others. While many users enjoy such animations, some users dislike them because they feel distracted or slowed down by them. In the worst case, users may even suffer from motion sickness as if it were a real life experience, so for these users reducing animations is a medical necessity.

Motion-triggered vestibular spectrum disorder #

Some users experience distraction or nausea from animated content. For example, scrolling animations can cause vestibular disorders when elements other than the main element associated with the scrolling move around a lot. For example, parallax scrolling animations can cause vestibular disorders because background elements move at a different rate than foreground elements. Vestibular (inner ear) disorder reactions include dizziness, nausea, and migraine headaches, and sometimes require bed rest to recover.

Remove motion on operating systems #

Many operating systems have had accessibility settings for specifying a preference for reduced motion for a long time. The screenshots below show macOS Mojave's Reduce motion preference and Android Pie's Remove animations preference. When checked, these preferences cause the operating system to not use decorative effects like app launching animations. Applications themselves can and should honor this setting, too, and remove all unnecessary animations.

Remove motion on the web #

Media Queries Level 5 brings the reduced motion user preference to the web as well. Media queries allow authors to test and query values or features of the user agent or display device independent of the document being rendered. The media query prefers-reduced-motion is used to detect if the user has set an operating system preference to minimize the amount of animation or motion it uses. It can take two possible values:

no-preference: Indicates that the user has made no preference in the underlying operating system. This keyword value evaluates as false in the boolean context.
reduce: Indicates that the user has set an operating system preference indicating that interfaces should minimize movement or animation, preferably to the point where all non-essential movement is removed.

Working with the media query from CSS and JavaScript contexts #

As with all media queries, prefers-reduced-motion can be checked from a CSS context and from a JavaScript context.

To illustrate both, let's say I have an important sign-up button that I want the user to click. I could define an attention-catching "vibrate" animation, but as a good web citizen I will only play it for those users who are explicitly OK with animations, but not everyone else, which can be users who have opted out of animations, or users on browsers that don't understand the media query.

/*
  If the user has expressed their preference for
  reduced motion, then don't use animations on buttons.
*/
@media (prefers-reduced-motion: reduce) {
  button {
    animation: none;
  }
}

/*
  If the browser understands the media query and the user
  explicitly hasn't set a preference, then use animations on buttons.
*/
@media (prefers-reduced-motion: no-preference) {
  button {
    /* `vibrate` keyframes are defined elsewhere */
    animation: vibrate 0.3s linear infinite both;
  }
}

To illustrate how to work with prefers-reduced-motion with JavaScript, let's imagine I have defined a complex animation with the Web Animations API. While CSS rules will be dynamically triggered by the browser when the user preference changes, for JavaScript animations I have to listen for changes myself, and then manually stop my potentially in-flight animations (or restart them if the user lets me):

const mediaQuery = window.matchMedia('(prefers-reduced-motion: reduce)');
mediaQuery.addEventListener('change', () => {
  console.log(mediaQuery.media, mediaQuery.matches);
  // Stop JavaScript-based animations.
});

Note that the parentheses around the actual media query are obligatory:

Don't

window.matchMedia('prefers-reduced-motion: reduce');

window.matchMedia('(prefers-reduced-motion: reduce)');

Working with the media query from `<picture>` contexts #

An interesting use case is to make playing of an animated AVIF, WebP, or GIF dependent on the media attribute. If (prefers-reduced-motion: no-preference) evaluates to true, it's safe to display the animated version, else the static version:

<picture>
  <!-- Animated versions. -->
  <source
    srcset="nyancat.avifs"
    type="image/avif"
    media="(prefers-reduced-motion: no-preference)"
  />
  <source
    srcset="nyancat.gif"
    type="image/gif"
    media="(prefers-reduced-motion: no-preference)"
  />
  <!-- Static versions. -->
  <img src="nyancat.png" alt="Nyan cat" width="250" height="250" />
</picture>

You can see the example below. Try toggling your device's motion preferences to see the difference.

Discover the user's preferences at request time #

The Sec-CH-Prefers-Reduced-Motion client hint header allows sites to obtain the user's motion preferences optionally at request time, allowing servers to inline the right CSS for performance reasons.

Demo #

I have created a little demo based on Rogério Vicente's amazing 🐈 HTTP status cats. First, take a moment to appreciate the joke, it's hilarious and I'll wait. Now that you're back, let me introduce the demo. When you scroll down, each HTTP status cat alternatingly appears from either the right or the left side. It's a buttery smooth 60 FPS animation, but as outlined above, some users may dislike it or even get motion sick by it, so the demo is programmed to respect prefers-reduced-motion. This even works dynamically, so users can change their preference on-the-fly, no reload required. If a user prefers reduced motion, the non-necessary reveal animations are gone, and just the regular scrolling motion is left. The screencast below shows the demo in action:

Video of the prefers-reduced-motion demo app

Conclusions #

Respecting user preferences is key for modern websites, and browsers are exposing more and more features to enable web developers to do so. Another launched example is prefers-color-scheme, which detects if the user prefers a light or dark color scheme. You can read everything about prefers-color-scheme in my article Hello Darkness, My Old Friend 🌒.

The CSS Working Group is currently standardizing more user preference media queries like prefers-reduced-transparency (detects if the user prefers reduced transparency), prefers-contrast (detects if the user has requested the system to increase or decrease the amount of contrast between adjacent colors), and inverted-colors (detects if the user prefers inverted colors).

(Bonus) Forcing reduced motion on all websites #

Not every site will use prefers-reduced-motion, or maybe not significantly enough for your taste. If you, for whatever reason, want to stop motion on all websites, you actually can. One way to make this happen is to inject a stylesheet with the following CSS into every web page you visit. There are several browser extensions out there (use at your own risk!) that allow for this.

@media (prefers-reduced-motion: reduce) {
  *,
  ::before,
  ::after {
    animation-delay: -1ms !important;
    animation-duration: 1ms !important;
    animation-iteration-count: 1 !important;
    background-attachment: initial !important;
    scroll-behavior: auto !important;
    transition-duration: 0s !important;
    transition-delay: 0s !important;
  }
}

The way this works is that the CSS above overrides the durations of all animations and transitions to such a short time that they are not noticeable anymore. As some websites depend on an animation to be run in order to work correctly (maybe because a certain step depends on the firing of the animationend event), the more radical animation: none !important; approach wouldn't work. Even the above hack is not guaranteed to succeed on all websites (for example, it can't stop motion that was initiated via the Web Animations API), so be sure to deactivate it when you notice breakage.

Latest Editor's Draft of the Media Queries Level 5 spec.
prefers-reduced-motion on Chrome Platform Status.
prefers-reduced-motion Chromium bug.
Blink Intent to Implement posting.

Acknowledgements #

Massive shout-out to Stephen McGruer who has implemented prefers-reduced-motion in Chrome and—together with Rob Dodson—has also reviewed this article. Hero image by Hannah Cauhepe on Unsplash.

Trust is good, observation is better: Intersection Observer v2

2019-02-15T00:00:00Z

Intersection Observer v1 is one of those APIs that's probably universally loved, and, now that Safari supports it as well, it's also finally universally usable in all major browsers. For a quick refresher of the API, I recommend watching Surma's Supercharged Microtip on Intersection Observer v1 that is embedded below. You can also read Surma's in-depth article. People have used Intersection Observer v1 for a wide range of use cases like lazy loading of images and videos, being notified when elements reach position: sticky, firing analytics events, and many more.

For the full details, check out the Intersection Observer docs on MDN, but as a short reminder, this is what the Intersection Observer v1 API looks like in the most basic case:

const onIntersection = (entries) => {
  for (const entry of entries) {
    if (entry.isIntersecting) {
      console.log(entry);
    }
  }
};

const observer = new IntersectionObserver(onIntersection);
observer.observe(document.querySelector('#some-target'));

What's challenging with Intersection Observer v1? #

To be clear, Intersection Observer v1 is great, but it's not perfect. There are some corner cases where the API falls short. Let's have a closer look! The Intersection Observer v1 API can tell you when an element is scrolled into the window's viewport, but it doesn't tell you whether the element is covered by any other page content (that is, when the element is occluded) or whether the element's visual display has been modified by visual effects like transform, opacity, filter, etc., which effectively can make it invisible.

For an element in the top-level document, this information can be determined by analyzing the DOM via JavaScript, for example via DocumentOrShadowRoot.elementFromPoint() and then digging deeper. In contrast, the same information cannot be obtained if the element in question is located in a third-party iframe.

Why is actual visibility such a big deal? #

The Internet is, unfortunately, a place that attracts bad actors with worse intentions. For example, a shady publisher that serves pay-per-click ads on a content site might be incentivized to trick people into clicking their ads to increase the publisher's ad payout (at least for a short period, until the ad network catches them). Typically, such ads are served in iframes. Now if the publisher wanted to get users to click such ads, they could make the ad iframes completely transparent by applying a CSS rule iframe { opacity: 0; } and overlaying the iframes on top of something attractive, like a cute cat video that users would actually want to click. This is called clickjacking. You can see such a clickjacking attack in action in the upper section of this demo (try "watching" the cat video and activate "trick mode"). You will notice that the ad in the iframe "thinks" it received legitimate clicks, even if it was completely transparent when you (pretend-involuntarily) clicked it.

How does Intersection Observer v2 fix this? #

Intersection Observer v2 introduces the concept of tracking the actual "visibility" of a target element as a human being would define it. By setting an option in the IntersectionObserver constructor, intersecting IntersectionObserverEntry instances will then contain a new boolean field named isVisible. A true value for isVisible is a strong guarantee from the underlying implementation that the target element is completely unoccluded by other content and has no visual effects applied that would alter or distort its display on screen. In contrast, a false value means that the implementation cannot make that guarantee.

An important detail of the spec is that the implementation is permitted to report false negatives (that is, setting isVisible to false even when the target element is completely visible and unmodified). For performance or other reasons, browsers limit themselves to working with bounding boxes and rectilinear geometry; they don't try to achieve pixel-perfect results for modifications like border-radius.

That said, false positives are not permitted under any circumstances (that is, setting isVisible to true when the target element is not completely visible and unmodified).

What does the new code look like in practice? #

The IntersectionObserver constructor now takes two additional configuration properties: delay and trackVisibility. The delay is a number indicating the minimum delay in milliseconds between notifications from the observer for a given target. The trackVisibility is a boolean indicating whether the observer will track changes in a target's visibility.

It's important to note here that when trackVisibility is true, delay is required to be at least 100 (that is, no more than one notification every 100ms). As noted before, visibility is expensive to calculate, and this requirement is a precaution against performance degradation and battery consumption. The responsible developer will use the largest tolerable value for delay.

According to the current spec, visibility is calculated as follows:

If the observer's trackVisibility attribute is false, then the target is considered visible. This corresponds to the current v1 behavior.
If the target has an effective transformation matrix other than a 2D translation or proportional 2D upscaling, then the target is considered invisible.
If the target, or any element in its containing block chain, has an effective opacity other than 1.0, then the target is considered invisible.
If the target, or any element in its containing block chain, has any filters applied, then the target is considered invisible.
If the implementation cannot guarantee that the target is completely unoccluded by other page content, then the target is considered invisible.

This means current implementations are pretty conservative with guaranteeing visibility. For example, applying an almost unnoticeable grayscale filter like filter: grayscale(0.01%) or setting an almost invisible transparency with opacity: 0.99 would all render the element invisible.

Below is a short code sample that illustrates the new API features. You can see its click tracking logic in action in the second section of the demo (but now, try "watching" the puppy video). Be sure to activate "trick mode" again to immediately convert yourself into a shady publisher and see how Intersection Observer v2 prevents non-legitimate ad clicks from being tracked. This time, Intersection Observer v2 has our back! 🎉

<!DOCTYPE html>
<!-- This is the ad running in the iframe -->
<button id="callToActionButton">Buy now!</button>

// This is code running in the iframe.

// The iframe must be visible for at least 800ms prior to an input event
// for the input event to be considered valid.
const minimumVisibleDuration = 800;

// Keep track of when the button transitioned to a visible state.
let visibleSince = 0;

const button = document.querySelector('#callToActionButton');
button.addEventListener('click', (event) => {
  if ((visibleSince > 0) &&
      (performance.now() - visibleSince >= minimumVisibleDuration)) {
    trackAdClick();
  } else {
    rejectAdClick();
  }
});

const observer = new IntersectionObserver((changes) => {
  for (const change of changes) {
    // ⚠️ Feature detection
    if (typeof change.isVisible === 'undefined') {
      // The browser doesn't support Intersection Observer v2, falling back to v1 behavior.
      change.isVisible = true;
    }
    if (change.isIntersecting && change.isVisible) {
      visibleSince = change.time;
    } else {
      visibleSince = 0;
    }
  }
}, {
  threshold: [1.0],
  // 🆕 Track the actual visibility of the element
  trackVisibility: true,
  // 🆕 Set a minimum delay between notifications
  delay: 100
}));

// Require that the entire iframe be visible.
observer.observe(document.querySelector('#ad'));

Latest Editor's Draft of the Intersection Observer spec.
Intersection Observer v2 on Chrome Platform Status.
Intersection Observer v2 Chromium bug.
Blink Intent to Implement posting.

Acknowledgements #

Thanks to Simeon Vincent, Yoav Weiss, and Mathias Bynens for reviewing this article, as well as Stefan Zager likewise for reviewing and for implementing the feature in Chrome. Hero image by Sergey Semin on Unsplash.

Add a web app manifest

2018-11-05T00:00:00Z

The web app manifest is a JSON file that tells the browser about your Progressive Web App and how it should behave when installed on the user's desktop or mobile device. A typical manifest file includes the app name, the icons the app should use, and the URL that should be opened when the app is launched, among other things.

Browser support

Chrome 39, Supported 39
Firefox, Not supported
Edge 79, Supported 79
Safari, Not supported

Source

Create the manifest file #

The manifest file can have any name, but is commonly named manifest.json and served from the root (your website's top-level directory). The specification suggests the extension should be .webmanifest, but browsers also support .json extensions, which may be easier for developers to understand.

A typical manifest looks something like this:

{
  "short_name": "Weather",
  "name": "Weather: Do I need an umbrella?",
  "icons": [
    {
      "src": "/images/icons-vector.svg",
      "type": "image/svg+xml",
      "sizes": "512x512"
    },
    {
      "src": "/images/icons-192.png",
      "type": "image/png",
      "sizes": "192x192"
    },
    {
      "src": "/images/icons-512.png",
      "type": "image/png",
      "sizes": "512x512"
    }
  ],
  "id": "/?source=pwa",
  "start_url": "/?source=pwa",
  "background_color": "#3367D6",
  "display": "standalone",
  "scope": "/",
  "theme_color": "#3367D6",
  "shortcuts": [
    {
      "name": "How's weather today?",
      "short_name": "Today",
      "description": "View weather information for today",
      "url": "/today?source=pwa",
      "icons": [{ "src": "/images/today.png", "sizes": "192x192" }]
    },
    {
      "name": "How's weather tomorrow?",
      "short_name": "Tomorrow",
      "description": "View weather information for tomorrow",
      "url": "/tomorrow?source=pwa",
      "icons": [{ "src": "/images/tomorrow.png", "sizes": "192x192" }]
    }
  ],
  "description": "Weather forecast information",
  "screenshots": [
    {
      "src": "/images/screenshot1.png",
      "type": "image/png",
      "sizes": "540x720",
      "form_factor": "narrow"
    },
    {
      "src": "/images/screenshot2.jpg",
      "type": "image/jpg",
      "sizes": "720x540",
      "form_factor": "wide"
    }
  ]
}

Key manifest properties #

`short_name` and/or `name` #

You must provide at least the short_name or name property. If both are provided, short_name is used on the user's home screen, launcher, or other places where space may be limited. name is used when the app is installed.

`icons` #

When a user installs your PWA, you can define a set of icons for the browser to use on the home screen, app launcher, task switcher, splash screen, and so on.

The icons property is an array of image objects. Each object must include the src, a sizes property, and the type of image. To use maskable icons, sometimes referred to as adaptive icons on Android, you'll also need to add "purpose": "any maskable" to the icon property.

For Chromium, you must provide at least a 192x192 pixel icon, and a 512x512 pixel icon. If only those two icon sizes are provided, Chrome automatically scales the icons to fit the device. If you'd prefer to scale your own icons, and adjust them for pixel-perfection, provide icons in increments of 48dp.

`id` #

The id property allows you to explicitly define the identifier used for your application. Adding the id property to the manifest removes the dependency on the start_url or the location of the manifest, and makes it possible for them to be updated in the future. For more information, see Uniquely identifying PWAs with the web app manifest id property.

`start_url` #

The start_url is required and tells the browser where your application should start when it is launched, and prevents the app from starting on whatever page the user was on when they added your app to their home screen.

Your start_url should direct the user straight into your app, rather than a product landing page. Think about what the user will want to do once they open your app, and place them there.

`background_color` #

The background_color property is used on the splash screen when the application is first launched on mobile.

`display` #

You can customize what browser UI is shown when your app is launched. For example, you can hide the address bar and browser user interface elements. Games can even be made to launch full screen. The display property takes one of the following values:

Property	Use
`fullscreen`	Opens the web application without any browser UI and takes up the entirety of the available display area.
`standalone`	Opens the web app to look and feel like a standalone app. The app runs in its own window, separate from the browser, and hides standard browser UI elements such as the URL bar.
`minimal-ui`	This mode is similar to `standalone`, but provides the user a minimal set of UI elements for controlling navigation (such as back and reload).
`browser`	A standard browser experience.

`display_override` #

Web apps can choose how they are displayed by setting a display mode in their manifest as explained above. Browsers are not required to support all display modes, but they are required to support the spec-defined fallback chain ("fullscreen" → "standalone" → "minimal-ui" → "browser"). If they don't support a given mode, they fall back to the next display mode in the chain. This inflexible behavior can be problematic in rare cases. For example, a developer cannot request "minimal-ui" without being forced back into the "browser" display mode when "minimal-ui" is not supported. Another problem is that the current behavior makes it impossible to introduce new display modes in a backward compatible way, since explorations like tabbed application mode don't have a natural place in the fallback chain.

These problems are solved by the display_override property, which the browser considers before the display property. Its value is a sequence of strings that are considered in the listed order, and the first supported display mode is applied. If none are supported, the browser falls back to evaluating the display field.

Consider the example below. (The details of "window-control-overlay" are out-of-scope for this article.)

{
  "display_override": ["window-control-overlay", "minimal-ui"],
  "display": "standalone",
}

As stated, the browser will look at display_override first.

"window-control-overlay"
"minimal-ui"

If neither option is available, it falls back to display. If "standalone" is not available, it resumes spec-defined fallabck chain from that point.

"standalone"
"minimal-ui"
"browser"

`scope` #

The scope defines the set of URLs that the browser considers to be within your app, and is used to decide when the user has left the app. The scope controls the URL structure that encompasses all the entry and exit points in your web app. Your start_url must reside within the scope.

A few other notes on scope:

If you don't include a scope in your manifest, then the default implied scope is the directory that your web app manifest is served from.
The scope attribute can be a relative path (../), or any higher level path (/) which would allow for an increase in coverage of navigations in your web app.
The start_url must be in the scope.
The start_url is relative to the path defined in the scope attribute.
A start_url starting with / will always be the root of the origin.

`theme_color` #

The theme_color sets the color of the tool bar, and may be reflected in the app's preview in task switchers. The theme_color should match the meta theme color specified in your document head.

An example of a PWA window with custom theme_color.

As of Chromium 93 and Safari 15, you can adjust this color in a media query with the media attribute of the meta theme color element. The first one that matches will be picked. For example, you could have one color for light mode and another one for dark mode. At the time of writing, you can't define those in your manifest. See w3c/manifest#975 GitHub issue.

<meta name="theme-color" media="(prefers-color-scheme: light)" content="white">
<meta name="theme-color" media="(prefers-color-scheme: dark)"  content="black">

`shortcuts` #

The shortcuts property is an array of app shortcut objects whose goal is to provide quick access to key tasks within your app. Each member is a dictionary that contains at least a name and a url.

`description` #

The description property describes the purpose of your app.

`screenshots` #

The screenshots property is an array of image objects representing your app in common usage scenarios. Each object must include the src, a sizes property, and the type of image. The form_factor property is optional. You can set it either to "wide" for screenshots applicable to wide screens only or "narrow" for narrow screenshots. You should only use it when the layout varies by screen size.

In Chrome, the image must respond to certain criteria:

Width and height must be at least 320px and at most 3840px.
The maximum dimension can't be more than 2.3 times as long as the minimum dimension.
All screenshots matching the appropriate form factor must have the same aspect ratio.

Richer installation UI on desktop and mobile.

Add the web app manifest to your pages #

After creating the manifest, add a <link> tag to all the pages of your Progressive Web App. For example:

<link rel="manifest" href="/manifest.json">

Test your manifest #

To verify your manifest is setup correctly, use the Manifest pane in the Application panel of Chrome DevTools.

Test your manifest in DevTools.

This pane provides a human-readable version of many of your manifest's properties, and makes it easy to verify that all of the images are loading properly.

Splash screens on mobile #

When your app first launches on mobile, it can take a moment for the browser to spin up, and the initial content to begin rendering. Instead of showing a white screen that may look to the user like the app is stalled, the browser will show a splash screen until the first paint.

Chrome automatically creates the splash screen from the manifest properties, specifically:

name
background_color
icons

The background_color should be the same color as the load page, to provide a smooth transition from the splash screen to your app.

Chrome will choose the icon that closely matches the device resolution for the device. Providing 192px and 512px icons is sufficient for most cases, but you can provide additional icons for pixel perfection.

Introduction to variable fonts on the web

2018-02-19T00:00:00Z

In this article, we will look at what variable fonts are, the benefits they offer, and how we can use them in our work. First, let's review how typography works on the web, and what innovations variable fonts bring.

Browser compatibility #

As of May 2020 variable fonts are supported in most browsers. See Can I use variable fonts? and Fallbacks.

Introduction #

The terms font and typeface are often used interchangeably by developers. However there is a difference: A typeface is the underlying visual design that can exist in many different typesetting technologies, and a font is one of these implementations, in a digital file format. In other words, a typeface is what you see, and the font is what you use.

Another concept that is often overlooked is the distinction between a style, and a family. A style is a single and specific typeface, such as Bold Italic, and a family is the complete set of styles.

Before variable fonts, each style was implemented as a separate font file. With variable fonts, all styles can be contained in a single file.

Left: a specimen of the Roboto typeface family. Right: named styles within the family.

Challenges for the designer and developer #

When a designer creates a print project they face some constraints, such as the physical size of the page layout, the number of colors they can use (which is determined by the kind of printing press that will be used), and so on. But they can use as many typeface styles as they like. This means that the typography of print media is often rich and sophisticated, so that the reading experience is truly delightful. Think of the last time you enjoyed browsing an excellent magazine.

Web designers and developers have different constraints than print designers, and an important one is the associated bandwidth costs of our designs. This has been a sticking point for richer typographic experiences, as they come at a cost. With traditional web fonts, each style used in our designs requires users to download a separate font file, which increases latency and page rendering time. Only including the Regular and Bold styles, plus their italic counterparts, can amount to 500 KB or more of font data. This is even before we have dealt with how the fonts are rendered, the fallback patterns we need to use, or undesirable side-effects such as FOIT and FOUT.

Many font families offer a much wider range of styles, from Thin to Black weights, narrow and wide widths, a variety of stylistic details, and even size-specific designs (optimized for large or small text sizes.) Since you'd have to load a new font file for every style (or combinations of styles), many web developers choose not to use these capabilities, reducing the reading experience of their users.

Anatomy of a variable font #

Variable fonts address these challenges, by packing styles into a single file.

This works by starting with a central or 'default' style, usually the 'Regular'–an upright roman design with the most typical weight and width that is most suitable for plain text. This is then connected to other styles in a continuous range, called an 'axis.' The most common axis is Weight, which can connect the default style through to a Bold style. Any individual style can be located along an axis, and is called an 'instance' of the variable font. Some instances are named by the font developer, for example Weight axis location 600 is called SemiBold.

The variable font Roboto Flex has three styles for its Weight axis. The Regular style is at the center, and there are two styles at the opposite ends of the axis, one lighter and the other heavier. Between these, you can choose from 900 instances:

Above: Illustrated anatomy of the Weight axis for the typeface Roboto.

The font developer can offer a set of different axes. You can combine them because they all share the same default styles. Roboto has three styles in a Width axis: the Regular is at the center of the axis, and two styles, narrower and wider, are at each end. These provide all the widths of the Regular style, and combine with the Weight axis to provide all the widths for every weight.

Roboto Flex in random combinations of Width and Weight

This means there are thousands of styles! This may seem like massive overkill, but the quality of the reading experience can be remarkably enhanced by this diversity of type styles. And, if it is without performance penalty, web developers can use a few or as many styles as they wish–it's up to their design.

Italics #

The way that Italics are handled in variable fonts is interesting, as there are two difference approaches. Typefaces like Helvetica or Roboto have interpolation compatible contours, so their Roman and Italic styles can be interpolated between and the Slant axis can be used to get from Roman to Italic.

Other typefaces (such as Garamond, Baskerville, or Bodoni) have Roman and Italic glyph contours that are not interpolation compatible. For example, the contours that typically define a Roman lowercase "n" do not match the contours used to define an Italic lowercase "n". Instead of interpolating one contour to the other, the Italic axis toggles from Roman to Italic contours.

Amstelvar's "n" contours in Italic (12 point, regular weight, normal width), and in Roman. Image supplied by David Berlow, type designer and typographer at Font Bureau.

After the switch to Italic, the axes available to the user should be the same as those for the Roman, just as the character set should be the same.

A glyph substitution capability can also be seen for individual glyphs, and used anywhere in the design space of a variable font. For example, a dollar sign design with two vertical bars works best at larger point sizes, but at smaller point sizes a design with only one bar is better. When we have fewer pixels for rendering the glyph, a two bar design can become illegible. To combat this, much like the Italic axis, a glyph substitution of one glyph for another can occur along the Optical Size axis at a point decided by the type designer.

In summary, where the contours allow for it, type designers can create fonts that interpolate between various styles in a multi-dimensional design space. This gives you granular control over your typography, and a great deal of power.

Axes definitions #

There are five registered axes, which control known, predictable features of the font: weight, width, optical size, slant and italics. Besides those, a font can contain custom axes. These can control any design aspect of the font the type designer wishes: the size of serifs, the length of swashes, the height of ascenders or the size of the dot on the i.

Even though axes can control the same feature, they might use different values. For example, in the Oswald and Hepta Slab variable fonts there is only one axis available, Weight, but the ranges are different–Oswald has the same range as before it was upgraded to be variable, 200 to 700, but Hepta Slab has an extreme hairline weight at 1 that goes all the up to 900.

The five registered axes have 4-character lowercase tags that are used to set their values in CSS:

Axis names and CSS values
Weight	`wght`
Width	`wdth`
Slant	`slnt`
Optical Size	`opsz`
Italics	`ital`

Since the font developer defines which axes are available in a variable font, and which values they can have, it is essential to find out what each font offers. The font's documentation should provide this, or you can inspect the font using a tool like Wakamai Fondue.

Use cases and benefits #

Setting the axes values comes down to personal taste and applying typographic best practices. The danger with any new technology is possible misuse, and settings that are overly artistic or exploratory could also decrease legibility of the actual text. For titles, exploring different axes to create great artistic designs are exciting, but for body copy this risks making the text illegible.

Exciting expression #

One great example of artistic expression is shown above, an exploration of the typeface Decovar by Mandy Michael.

You can view the working example and source code for the above sample here.

Animation #

Typeface Zycon, designed for animation by David Berlow, type designer and typographer at Font Bureau.

There is also a possibility to explore animating characters with variable fonts. Above is an example of different axes being used with the typeface Zycon. See the live animation example on Axis Praxis.

Anicons is the world's first animated color icon font, based on Material Design Icons. Anicons is an experiment that combines two cutting edge font technologies: variable fonts and color fonts.

A few examples of hover animations from Anicon's color icon font

Finesse #

Amstelvar using little bits of XTRA in opposite directions so the words' widths are evened out

Roboto Flex and Amstelvar offer a set of "Parametric Axes." In these axes, the letters are deconstructed into 4 fundamental aspects of form: black or positive shapes, white or negative shapes, and the x and y dimensions. In the same way that primary colors can be blended with any other color to adjust it, these 4 aspects can be used to fine tune any other axis.

The XTRA axis in Amstelvar allows you to adjust the "white" per mille value, as shown above. Using little bits of XTRA in opposite directions, the words' widths are evened out.

Variable fonts in CSS #

Loading variable font files #

Variable fonts are loaded though the same @font-face mechanism as traditional static web fonts, but with two new enhancements:

@font-face {
	font-family: 'Roboto Flex';
	src: url('RobotoFlex-VF.woff2') format('woff2') tech('variations'),
	     url('RobotoFlex-VF.woff2') format('woff2-variations');
	font-weight: 100 1000;
	font-stretch: 25% 151%;
}

1. Source Formats: We don't want the browser to download the font if it doesn't support variable fonts, so we add format and tech descriptions: once in the future syntax (format('woff2') tech('variations')), once in the deprecated but supported among browsers syntax (format('woff2-variations')). If the browser supports variable fonts and supports the upcoming syntax, it will use the first declaration. If it supports variable fonts and the current syntax, it will use the second declaration. They both point to the same font file.

2. Style Ranges: You'll notice we're supplying two values for font-weight and font-stretch. Instead of telling the browser which specific weight this font provides (for example font-weight: 500;), we now give it the range of weights supported by the font. For Roboto Flex, the Weight axis ranges from 100 to 1000, and CSS directly maps the axis range to the font-weight style property. By specifying the range in @font-face, any value outside this range will be "capped" to the nearest valid value. The Width axis range is mapped in the same way to the font-stretch property.

If you're using the Google Fonts API, this will all be taken care of. Not only will the CSS contain the proper source formats and ranges, Google Fonts will also send static fallback fonts in case variable fonts aren't supported.

Using weights and widths #

Currently, the axes you can reliably set from CSS are the wght axis through font-weight, and the wdth axis through font-stretch.

Traditionally, you would set font-weight as a keyword (light, bold) or as a numerical value between 100 and 900, in steps of 100. With variable fonts, you can set any value within the font's Width range:

.kinda-light {
  font-weight: 125;
}

.super-heavy {
  font-weight: 1000;
}

Roboto Flex' Weight axis being changed from its minimum to its maximum.

Likewise, we can set font-stretch with keywords (condensed, ultra-expanded) or with percentage values:

.kinda-narrow {
  font-stretch: 33.3%;
}

.super-wide {
  font-stretch: 151%;
}

Roboto Flex' Width axis being changed from its minimum to its maximum.

Using italics and obliques #

The ital axis is intended for fonts that contain both a regular style, and an italic style. The axis is meant to be an on/off switch: value 0 is off and will show the regular style, value 1 will show the italics. Unlike other axes, there's no transition. A value of 0.5 won't give you "half italics".

The slnt axis is different from italics in that it's not a new style, but just slants the regular style. By default its value is 0, which means the default upright letter shapes. Roboto Flex has a maximum slant of -10 degrees, meaning the letters will lean to the right when going from 0 to -10.

It would be intuitive to set these axis through the font-style property, but as of April 2020, how to exactly do this is still being worked out. So for now, you should treat these as custom axes, and set them through font-variation-settings:

i, em, .italic {
	/* Should be font-style: italic; */
	font-variation-settings: 'ital' 1;
}

.slanted {
	/* Should be font-style: oblique 10deg; */
	font-variation-settings: 'slnt' 10;
}

Roboto Flex' Slant axis being changed from its minimum to its maximum.

Using optical sizes #

A typeface can be rendered very small (a 12px footnote) or very large (a 80px headline). Fonts can respond to these size changes by changing its letter shapes to better suit its size. A small size might be better off without fine details, while a large size might benefit from more details and thinner strokes.

The letter 'a' in Roboto Flex at different pixel sizes, then scaled to be the same size, shows the differences in design. Try it yourself on Codepen

A new CSS property has been introduced for this axis: font-optical-sizing. By default it's set to auto, which makes the browser set the axis value based on the font-size. This means the browser will pick the best optical size automatically, but if you wish to turn this off you can set font-optical-sizing to none.

You can also set a custom value for the opsz axis, if you deliberately want an optical size that doesn't match the font size. The following CSS would cause text to be displayed at a large size, but at an optical size as if it were printed in 8pt:

.small-yet-large {
  font-size: 100px;
  font-variation-settings: 'opsz' 8;
}

Using custom axes #

Unlike registered axes, custom axes will not be mapped to an existing CSS property, so you will always have to set them through font-variation-settings. Tags for custom axes are always in uppercase, to distinguish them from registered axes.

Roboto Flex offers a few custom axes, and the most important is Grade (GRAD). A Grade axis is interesting as it changes the weight of the font without changing the widths, so line breaks do not change. By playing with a Grade axis, you can avoid being forced to fiddle with changes to Weight axis that affects the overall width, and then changes to the Width axis that affect the overall weight.

Roboto Flex' Grade axis being changed from its minimum to its maximum.

As GRAD is a custom axis, with a range of -200 to 150 in Roboto Flex. We need to address it with font-variation-settings:

.grade-light {
	font-variation-settings: `GRAD` -200;
}

.grade-normal {
	font-variation-settings: `GRAD` 0;
}

.grade-heavy {
	font-variation-settings: `GRAD` 150;
}

Variable fonts on Google Fonts #

Google Fonts has expanded their catalog with variable fonts, and adding new ones regularly. The interface is currently aimed at picking single instances from the font: you select the variation you want, click "Select this style", and it will be added to the <link> element that fetches the CSS and fonts from Google Fonts.

To use all the available axes, or ranges of values, you will have to manually compose the URL to the Google Fonts API. The variable fonts overview lists all axes and values.

The Google Variable Fonts Links tool can also give you the latest URLs for the full variable fonts.

Font-variation-settings inheritance #

While all registered axes will soon be supported through existing CSS properties, for the time being you might need to rely on font-variation-settings as a fallback. And if your font has custom axes, you'll need font-variation-settings too.

However, here's a little gotcha with font-variation-settings. Every property you don't explicitly set will automatically be reset to its default. Previously set values aren't inherited! This means the following will not work as expected:

<span class="slanted grade-light">
	I should be slanted and have a light grade
</span>

First the browser will apply font-variation-settings: 'slnt' 10 from the .slanted class. Then it will apply font-variation-settings: 'GRAD' -200 from the .grade-light class. But this will reset the slnt back to its default of 0! The result will be text in a light grade, but not slanted.

Luckily, we can work around this by using CSS variables:

/* Set the default values */
:root {
	--slnt: 0;
	--GRAD: 0;
}

/* Change value for these elements and their children */
.slanted {
	--slnt: 10;
}

.grade-light {
	--GRAD: -200;
}

.grade-normal {
	--GRAD: 0;
}

.grade-heavy {
	--GRAD: 150;
}

/* Apply whatever value is kept in the CSS variables */
.slanted,
.grade-light,
.grade-normal,
.grade-heavy {
	font-variation-settings: 'slnt' var(--slnt), 'GRAD' var(--GRAD);
}

CSS variables will cascade, so if an element (or one of its parents) will have set the slnt to 10, it will keep that value, even if you set GRAD to something else. See Fixing variable font inheritance for an in-depth explanation of this technique.

Note that animating CSS variables doesn't work (by design), so something like this doesn't work:

@keyframes width-animation {
   from { --wdth: 25; }
   to   { --wdth: 151; }
}

These animations will have to happen directly on font-variation-settings.

Performance gains #

OpenType variable fonts allow us to store multiple variations of a type family into a single font file. Monotype ran an experiment by combining 12 input fonts to generate eight weights, across three widths, across both the Italic and Roman styles. Storing 48 individual fonts in a single variable font file meant a 88% reduction in file size.

However, if you are using a single font such as Roboto Regular and nothing else, you might not see a net gain in font size if you were to switch to a variable font with many axes. As always, it depends on your use-case.

On the flip side, animating the font between settings may cause performance issues. Although this will improve once variable font support in browsers gets more mature, the problem can be reduced somewhat by only animating fonts that are currently on screen. This handy snippet by Dinamo pauses animations in elements with the class vf-animation, when they're not on screen:

var observer = new IntersectionObserver(function(entries, observer) {
  entries.forEach(function(entry) {
    // Pause/Play the animation
    if (entry.isIntersecting) entry.target.style.animationPlayState = "running"
    else entry.target.style.animationPlayState = "paused"
  });
});

var variableTexts = document.querySelectorAll(".vf-animation");
variableTexts.forEach(function(el) { observer.observe(el); });

If your font responds to user interaction, it's a good idea to throttle or debounce input events. This will prevent the browser from rendering instances of the variable font that changed so little from the previous instance the human eye wouldn't see the difference.

If you're using Google Fonts, it's a good idea to preconnect to https://fonts.gstatic.com, the domain where Google's fonts are hosted. This will make sure the browser knows early on where to get the fonts when it comes across them in the CSS:

<link rel="preconnect" href="https://fonts.gstatic.com" />

This tip works for other CDNs as well: the sooner you let the browser set up a network connection, the sooner it can download your fonts.

Find more performance tips for loading Google Fonts in The Fastest Google Fonts.

Fallbacks and browser support #

All modern browsers support variable fonts. In case you need to support older browsers, you can choose to build your site with static fonts, and use variable fonts as progressive enhancement:

/* Set up Roboto for old browsers, only regular + bold */
@supports not (font-variation-settings: normal) {
  @font-face {
    font-family: Roboto;
    src: url('Roboto-Regular.woff2');
    font-weight: normal;
  }

  @font-face {
    font-family: Roboto;
    src: url('Roboto-Bold.woff2');
    font-weight: bold;
  }

  body {
    font-family: Roboto;
  }

  .super-bold {
    font-weight: bold;
  }
}

/* Set up Roboto for modern browsers, all weights */
@supports (font-variation-settings: normal) {
  @font-face {
    font-family: 'Roboto';
    src: url('RobotoFlex-VF.woff2') format('woff2') tech('variations'),
         url('RobotoFlex-VF.woff2') format('woff2-variations');
    font-weight: 100 1000;
    font-stretch: 25% 151%;
  }

  .super-bold {
    font-weight: 1000;
  }
}

For older browsers, text with the class .super-bold will get rendered in the normal bold, as that's the only bold font we have available. When variable fonts are supported, we can actually use the heaviest weight of 1000.

The @supports rule is not supported by Internet Explorer, so this browser would not show any styling. If this is a problem, you could always use one of the oldschool hacks to target relevant older browsers.

If you are using the Google Fonts API, it will take care of loading the proper fonts for your visitor's browsers. Say you request the font Oswald in weight range 200 to 700, like so:

<link href="https://fonts.googleapis.com/css2?family=Oswald:wght@200..700&display=swap" rel="stylesheet">

Modern browsers that can handle variable fonts will get the variable font, and will have every weight between 200 and 700 available. Older browsers will get served individual static fonts for every weight. In this case, this means they'll download 6 font files: one for weight 200, one for weight 300, and so on.

Thanks #

This article would have only been made possible with the help of the following people:

Mustafa Kurtuldu, UX designer and design advocate at Google
Roel Nieskens, UX designer/developer and typography expert at Kabisa
Dave Crossland, Program Manager, Google Fonts
David Berlow, type designer and typographer at Font Bureau
Laurence Penney, developer of axis-praxis.org
Mandy Michael, developer of variablefonts.dev

Hero image by Bruno Martins on Unsplash.

Offline UX design guidelines

2016-11-10T00:00:00Z

This article provides design guidelines on how to create a great experience on both slow networks and offline.

The quality of a network connection can be affected by a number of factors such as:

Poor coverage of a provider.
Extreme weather conditions.
Power outages.
Entering into permanent "dead zones" such as in buildings with walls that block network connections.
Entering into temporary "dead zones" such as when traveling on a train and going through a tunnel.
Time-boxed internet connections such as those in airports or hotels.
Cultural practices that require limited or no internet access at specific times or days.

Your goal is to provide a good experience that lessens the impact of changes in connectivity.

Decide what to show your users when they have a bad network connection #

The first question that must be asked is what does success and failure of a network connection look like? A successful connection is your app's normal online experience. The failure of a connection, however, can be both the offline state of your app as well how the app behaves when there is a laggy network.

When thinking about the success or failure of a network connection you need to ask yourself these important UX questions:

How long do you wait to determine the success or failure of a connection?
What can you do while success or failure is being determined?
What should you do in the event of failure?
How do you inform the user of the above?

Inform users of their current state and change of state #

Inform the user of both the actions they can still take when they have a network failure and the current state of the application. For example, a notification could say:

You seem to have a bad network connection. Not to worry! Messages will be sent when the network is restored.

Clearly inform the user when a change in state occurs as soon as possible.

The Google I/O app used a "toast" to let the user know when they were offline.

Inform users when the network connection improves or is restored #

How you inform the user that their network connection has improved depends on your application. Apps such as a stock market app that prioritize current information should auto-update and notify the user as soon as possible.

It is recommended that you let the user know that your web app has been updated "in the background" by using a visual cue such as, for example, a material design toast element. This involves detecting both the the presence of a service worker and an update to its managed content. You can see a code example of this function at work here.

One example of this would be Chrome Platform Status which posts a note to the user when the app has been updated.

Some apps, such as the weather app, need to auto-update because old data is not useful.

Apps such as Chrome Status let the user know when content has been updated via a toast.

You may also show the last time the app was updated at all times in a prominent space. This would be useful for a currency converter app, for example.

Material Money caches rates…

…and notifies the user when the app has been updated.

Applications such as news apps could show a simple tap-to-update notification informing the user of new content. Auto-updating would cause users to lose their place.

Tailpiece, an online newspaper, will auto-download the latest news…

…but allow users to refresh manually so they do not lose their place in an article.

Update the UI to reflect the current contextual state #

Each bit of UI may have its own context and functionality that will change depending on if it requires a successful connection. One example would be an e‑commerce site that can be browsed offline. The Buy button and pricing would be disabled until a connection is reestablished.

Other forms of contextual states could include data. For example, the financial application Robinhood allows users to buy stock and uses color and graphics to notify the user when the market is open. The whole interface turns white and then grays out when the market closes. When the value of stock increases or decreases, each individual stock widget turns green or red depending on its state.

Educate the user so they understand what the offline model is #

Offline is a new mental model for everyone. You need to educate your users about what changes will occur when they don't have a connection. Inform them of where large data is saved and give them settings to change the default behavior. Make sure you use multiple UI design components such as informative language, icons, notifications, color, and imagery to convey these ideas collectively rather than relying on a single design choice, such as an icon on its own, to tell the whole story.

Provide an offline experience by default #

If your app doesn't require much data, then cache that data by default. Users can become increasingly frustrated if they can only access their data with a network connection. Try to make the experience as stable as possible. An unstable connection will make your app feel untrustworthy, where an app that lessens the impact of a network failure will feel magical to the user.

News sites could benefit from auto-downloading and auto-saving the latest news so a user could read today's news without a connection, perhaps downloading the text without the article images. Also, adapt to the user's behavior. For example, if the sports section is what they typically view, make that the priority download.

If the device is offline, Tailpiece will notify the user with a status message…

…letting them know that they can at least partially still use the app.

Inform the user when the app is ready for offline consumption #

When a web app first loads you need to indicate to the user if it is ready for offline use. Do this with a widget that provides brief feedback about an operation through a message at the bottom of the screen such as, for example, when a section has been synced or a data file has downloaded.

Again think of the language you are using to make sure it is fit for your audience. Ensure the messaging is the same in all instances where it's used. The term offline is generally misunderstood by a non-technical audience so use action-based language that your audience can relate to.

The Google I/O 2016 app notified the user when the app is ready for offline use…

…and so does the Chrome Platform Status site, which includes information about the occupied storage.

Make 'save for offline' an obvious part of the interface for data-heavy apps #

If an application uses large amounts of data, make sure that there is a switch or pin to add an item for offline use rather than auto-downloading, unless a user has specifically asked for this behavior via a settings menu. Make sure that the pin or download UI is not obscured by other UI elements and that the feature is obvious to the user.

One example would be a music player that requires large data files. The user is aware of the associated data cost, but may also want to use the player offline. Downloading music for later use requires the user to plan ahead, so education about this may be required during onboarding.

Clarify what is available offline #

Be clear as to the option you are providing. You may need to show a tab or setting that shows an "offline library" or content index, so the user can easily see what they have stored on their phone and what needs to be saved. Make sure the settings are concise and be clear where the data will be stored and who has access to it.

Show the actual cost of an action #

Many users equate offline capability with 'downloading'. Users in countries where network connections regularly fail or aren't available often share content with other users, or save content for offline use when they have connectivity.

Users on data plans may avoid downloading large files for fear of cost, so you may also want to display an associated cost so users can make an active comparison for a specific file or task. For example, if the music app above could detect if the user is on a data plan and show the file size so that users can see the cost of a file.

Help prevent hacked experiences #

Often users hack an experience without realizing they are doing it. For example before cloud-based file sharing web apps, it was common for users to save large files and attach them to emails so they could carry on editing from a different device. It is important not to be pulled into their hacked experience but rather look at what they are trying to achieve. In other words instead of thinking of how you can make attaching a large file more user-friendly, solve the problem of sharing large files across multiple devices.

Make experiences transferable from one device to another #

When building for flaky networks, try to sync as soon as the connection improves so that the experience is transferable. For example, imagine a travel app losing a network connection mid-way through a booking. When the connection is reestablished, the app syncs with the user's account allowing them to continue their booking on their desktop device. Not being able to transfer experiences can be incredibly jarring to users.

Inform the user of the current state of their data. For example, you could show whether the app has synced. Educate them where possible but try not to overburden them with messaging.

Create inclusive design experiences #

When designing seek to be inclusive by providing meaningful design devices, simple language, standard iconography, and meaningful imagery that will guide the user to complete the action or task rather than hinder their progress.

Use simple, concise language #

Good UX is not just about a well designed interface. It includes the path a user takes as well as the words used in the app. Avoid tech jargon when explaining the state of the app or individual UI components. Consider that the phrase "app offline" might not convey to the user the current state of the app.

Don't

Avoid terms that aren't intelligible to non-technical users.

Use language and imagery that describe the action.

Use multiple design devices to create accessible user experiences #

Use language, color, and visual components to demonstrate a change of state or current status. Solely using color to demonstrate state may not be noticed by the user and may be inaccessible to users with visual disabilities. Also, the instinct for designers is to use grayed UI to represent offline, but this can have a loaded meaning on the web. Grayed UI such as input elements on a form also means that an element is disabled. This can cause confusion if you only use color to depict state.

To prevent misunderstandings, express different states to the user in multiple ways, for example with color, labels, and UI components.

Don't

Avoid using color as the sole means to describe what is happening.

Use a mixture of design elements to convey meaning.

Use icons that convey meaning #

Make sure that information is conveyed correctly with meaningful text labels as well as icons. Icons alone can be problematic, since the concept of offline on the web is relatively new. Users may misunderstand icons used on their own. For example, using a floppy disc for save makes sense to an older generation but young users who have never seen a floppy disc may be confused by the metaphor. Likewise, the 'hamburger' menu icon has been known to confuse users when presented without a label.

When introducing an offline icon try to remain consistent with industry standard visuals (when they exist) as well as providing a text label and description. For example, saving for offline might be a typical download icon or perhaps if the action involves syncing it could be a syncing icon. Some actions may be interpreted as saving for offline rather than demonstrating a network's status. Think of the action you are trying to convey rather than presenting the user with an abstract concept. For example, save or download data would be action-based.

Offline can mean a number of things depending on the context, such as download, export, pin, etc. For more inspiration checkout the Material Design icon set.

Use skeleton layouts with other feedback mechanisms #

A skeleton layout is essentially a wireframe version of your app that displays while content is being loaded. This helps demonstrate to the user that content is about to be loaded. Consider also using a preloader UI as well, with a text label informing the user that the app is loading. One example would be to pulsate the wireframe content giving the app the feeling that it is alive and loading. This reassures the user that something is happening and helps prevent resubmissions or refreshes of your app.

A skeleton placeholder layout is shown during the download of the article…

…that gets replaced with the real contents once the download finishes.

Don't block content #

In some applications, a user might trigger an action such as creating a new document. Some apps will try to connect to a server in order to sync the new document and to demonstrate this they display an intrusive loading modal dialog that covers the entire screen. This may work fine if the user has a stable network connection, but if the network is unstable they won't be able to escape from this action and the UI effectively blocks them from doing anything else. Network requests that block content should be avoided. Allow the user to continue to browse your app and queue tasks that will be performed and synced once the connection has improved.

Demonstrate the state of an action by providing your users with feedback. For example, if a user is editing a doc, consider changing the feedback design so it is visibly different from when they are online but still shows that their file was "saved" and will sync when they have a network connection. This will educate the user about the different states available and reassure them that their task or action has been stored. This has the added benefit of the user growing more confident using your application.

Design for the next billion #

In many regions, low-end devices are commonplace, connectivity is unreliable and, for many users, data is unaffordable. You will need to earn user trust by being transparent and frugal with data. Think about ways to help users on poor connections and simplify the interface to help speed up tasks. Always try to ask users before downloading data-heavy content.

Offer low bandwidth options for users on laggy connections. So if the network connection is slow, provide small assets. Offer an option to choose high or low quality assets.

Conclusion #

Education is key to offline UX as users are unfamiliar with these concepts. Try to create associations with things that are familiar, e.g downloading for later use is the same as offlining data.

When designing for unstable network connections, remember these guidelines:

Design for the success, failure, and instability of a network connection.
Data may be expensive, so be considerate to the user.
For most users globally, the tech environment is almost exclusively mobile.
Low-end devices are commonplace, with limited storage, memory, and processing power, and small displays and lower touchscreen quality. Make sure performance is a part of your design process.
Allow users to browse your application when they are offline.
Inform users of their current state and of changes in states.
Try to provide offline by default if your app doesn't require much data.
If the app is data-heavy, educate users about how they can download for offline use.
Make experiences transferable between devices.
Use language, icons, imagery, typography, and color together to express ideas to the user.
Provide reassurance and feedback to help the user.

Read files in JavaScript

2010-06-18T00:00:00Z

Selecting and interacting with files on the user's local device is one of the most commonly used features of the web. It allows users to select files and upload them to a server, for example, uploading photos, or submitting tax documents, etc. But, it also allows sites to read and manipulate them without ever having to transfer the data across the network.

The modern File System Access API #

The File System Access API provides an easy way to both read from and write to files and directories on the user's local system. It's currently available in most Chromium-based browsers such as Chrome or Edge. To learn more about it, see the File System Access API article.

Since the File System Access API is not compatible with all browsers yet, check out browser-fs-access, a helper library that uses the new API wherever it is available, but falls back to legacy approaches when it is not.

Working with files, the classic way #

This guide shows you how to:

Select files
- Using the HTML input element
- Using a drag-and-drop zone
Read file metadata
Read a file's content

Select files #

HTML input element #

The easiest way for users to select files is using the <input type="file"> element, which is supported in every major browser. When clicked, it lets a user select a file, or multiple files if the multiple attribute is included, using their operating system's built-in file selection UI. When the user finishes selecting a file or files, the element's change event fires. You can access the list of files from event.target.files, which is a FileList object. Each item in the FileList is a File object.

<!-- The `multiple` attribute lets users select multiple files. -->
<input type="file" id="file-selector" multiple>
<script>
  const fileSelector = document.getElementById('file-selector');
  fileSelector.addEventListener('change', (event) => {
    const fileList = event.target.files;
    console.log(fileList);
  });
</script>

This example lets a user select multiple files using their operating system's built-in file selection UI and then logs each selected file to the console.

Limit the types of files users can select #

In some cases, you may want to limit the types of files users can select. For example, an image editing app should only accept images, not text files. To do that, add an accept attribute to the input element to specify which file types are accepted.

<input type="file" id="file-selector" accept=".jpg, .jpeg, .png">

Custom drag-and-drop #

In some browsers, the <input type="file"> element is also a drop target, allowing users to drag-and-drop files into your app. But, the drop target is small, and can be hard to use. Instead, once you've provided the core functionality using an <input type="file"> element, you can provide a large, custom drag-and-drop surface.

Choose your drop zone #

Your drop surface depends on the design of your application. You may only want part of the window to be a drop surface, or potentially the entire window.

Squoosh makes the entire window a drop zone.

Squoosh allows the user to drag and drop an image anywhere into the window, and clicking select an image invokes the <input type="file"> element. Whatever you choose as your drop zone, make sure it's clear to the user that they can drag and drop files onto that surface.

Define the drop zone #

To enable an element to be a drag-and-drop zone, you'll need to listen for two events, dragover and drop. The dragover event updates the browser UI to visually indicate that the drag-and-drop action is creating a copy of the file. The drop event is fired after the user drops the files onto the surface. Similar to the input element, you can access the list of files from event.dataTransfer.files, which is a FileList object. Each item in the FileList is a File object.

const dropArea = document.getElementById('drop-area');

dropArea.addEventListener('dragover', (event) => {
  event.stopPropagation();
  event.preventDefault();
  // Style the drag-and-drop as a "copy file" operation.
  event.dataTransfer.dropEffect = 'copy';
});

dropArea.addEventListener('drop', (event) => {
  event.stopPropagation();
  event.preventDefault();
  const fileList = event.dataTransfer.files;
  console.log(fileList);
});

event.stopPropagation() and event.preventDefault() stop the browser's default behavior and allow your code to run instead. Without them, the browser would otherwise navigate away from your page and open the files the user dropped into the browser window.

Check out Custom drag-and-drop for a live demonstration.

What about directories? #

Unfortunately, today there isn't a good way to access a directory.

The webkitdirectory attribute on the <input type="file"> element allows the user to choose a directory or directories. It's supported in most major browsers except for Firefox for Android.

If drag-and-drop is enabled, a user may try to drag a directory into the drop zone. When the drop event is fired, it will include a File object for the directory, but does not provide access any of the files within the directory.

Read file metadata #

The File object contains metadata about the file. Most browsers provide the file name, the size of the file, and the MIME type, though depending on the platform, different browsers may provide different, or additional information.

function getMetadataForFileList(fileList) {
  for (const file of fileList) {
    // Not supported in Safari for iOS.
    const name = file.name ? file.name : 'NOT SUPPORTED';
    // Not supported in Firefox for Android or Opera for Android.
    const type = file.type ? file.type : 'NOT SUPPORTED';
    // Unknown cross-browser support.
    const size = file.size ? file.size : 'NOT SUPPORTED';
    console.log({file, name, type, size});
  }
}

You can see this in action in the input-type-file demo.

Read a file's content #

To read a file, use FileReader, which enables you to read the content of a File object into memory. You can instruct FileReader to read a file as an array buffer, a data URL, or text.

function readImage(file) {
  // Check if the file is an image.
  if (file.type && !file.type.startsWith('image/')) {
    console.log('File is not an image.', file.type, file);
    return;
  }

  const reader = new FileReader();
  reader.addEventListener('load', (event) => {
    img.src = event.target.result;
  });
  reader.readAsDataURL(file);
}

The example above reads a File provided by the user, then converts it to a data URL, and uses that data URL to display the image in an img element. Check out the read-image-file demo to see how to verify that the user has selected an image file.

Monitor the progress of a file read #

When reading large files, it may be helpful to provide some UX to indicate how far the read has progressed. For that, use the progress event provided by FileReader. The progress event provides two properties, loaded (the amount read) and total (the amount to read).

function readFile(file) {
  const reader = new FileReader();
  reader.addEventListener('load', (event) => {
    const result = event.target.result;
    // Do something with result
  });

  reader.addEventListener('progress', (event) => {
    if (event.loaded && event.total) {
      const percent = (event.loaded / event.total) * 100;
      console.log(`Progress: ${Math.round(percent)}`);
    }
  });
  reader.readAsDataURL(file);
}

Hero image by Vincent Botta from Unsplash

Thomas Steiner on web.dev

Compiling mkbitmap to WebAssembly

About mkbitmap #

Get the code #

Compile and install locally #

Compile mkbitmap to WebAssembly #

mkbitmap with WebAssembly in the browser #

Prevent automatic execution #

Create a modular build with some more build flags #

Manually execute the main function #

Redirect the standard output #

Get the input file into the memory file system #

First actual execution #

Get the output file out of the memory file system #

Add an interactive UI #

Conclusion #

Acknowledgements #

What is WebAssembly and where did it come from?

Assembly language #

asm.js #

WebAssembly #

Textual representation #

Binary representation #

Compiling to WebAssembly #

Learn more #

Acknowledgements #

The origin private file system

Browser support #

Motivation #

Traditional way of working with files on the web #

Modern way of working with files on the web #

Restrictions of working with files on the web #

Files as the foundation of processing #

The user-visible versus the origin private file system #

Specifics of the origin private file system #

Getting access to the root directory #

Main thread or Web Worker #

Using the origin private file system on the main thread #

Creating new files and folders #

Accessing existing files and folders #

Getting the file associated with a file handle for reading #

Writing to a file by streaming #

Deleting files and folders #

Moving and renaming files and folders #

Resolving the path of a file or folder #

Checking if two file or folder handles point to the same file or folder #

Listing the contents of a folder #

Recursively listing the contents of a folder and all subfolders #

Using the origin private file system in a Web Worker #

Getting a synchronous access handle #

Synchronous in-place file methods #

Copying a file from the origin private file system to the user-visible file system #

Debugging the origin private file system #

Demo #

Conclusions #

Related links #

Acknowledgements #

PWAs in app stores

Prerequisites #

Packaging #

Submitting #

Example #

JavaScript import maps are now supported cross-browser

Further reading #

Acknowledgements #

Push notifications are now supported cross-browser

Further reading #

Acknowledgements #

How to use multiple screens

The modern way #

Using window.getScreenDetails() #

The classic way #

Using the window.screen method #

Progressive enhancement #

Further reading #

Demo #

How to let the user share the website they are on

The modern way #

Using the Web Share API #

The classic way #

About `mkbitmap` #

Compile `mkbitmap` to WebAssembly #

`mkbitmap` with WebAssembly in the browser #

Using `window.getScreenDetails()` #

Using `navigator.clipboard.writeText()` #

Using the File System Access API's `DataTransferItem.getAsFileSystemHandle()` method #

Using the non-standard `DataTransferItem.webkitGetAsEntry()` method #

Using the File System Access API's `DataTransferItem.getAsFileSystemHandle()` method #

Using the classic `DataTransferItem.getAsFile()` method #

Using the File System Access API's `showDirectoryPicker()` method #

Using the `<input type="file" webkitdirectory>` element #

Using the File System Access API's `showOpenFilePicker()` method #

Using the `<input type="file">` element #

Using the File System Access API's `showSaveFilePicker()` method #

Using the `<a download>` element #