Eric Bidelman on web.dev

Network Error Logging (NEL)

2022-01-31T00:00:00Z

Introduction #

Network Error Logging (NEL) is a mechanism for collecting client-side network errors from an origin.

It uses the NEL HTTP response header to tell the browser to collect network errors, then integrates with the Reporting API to report the errors to a server.

Overview of the legacy Reporting API #

The legacy `Report-To` Header #

To use the legacy Reporting API, you'll need to set a Report-To HTTP response header. Its value is an object which describes an endpoint group for the browser to report errors to:

Report-To:
{
    "max_age": 10886400,
    "endpoints": [{
    "url": "https://analytics.provider.com/browser-errors"
    }]
}

If your endpoint URL lives on a different origin than your site, the endpoint should support CORS preflight requests. (e.g. Access-Control-Allow-Origin: *; Access-Control-Allow-Methods: GET,PUT,POST,DELETE,OPTIONS; Access-Control-Allow-Headers: Content-Type, Authorization, Content-Length, X-Requested-With).

In the example, sending this response header with your main page configures the browser to report browser-generated warnings to the endpoint https://analytics.provider.com/browser-errors for max_age seconds. It's important to note that all subsequent HTTP requests made by the page (for images, scripts, etc.) are ignored. Configuration is setup during the response of the main page.

Explanation of header fields #

Each endpoint configuration contains a group name, max_age, and endpoints array. You can also choose whether to consider subdomains when reporting errors by using the include_subdomains field.

Field	Type	Description
`group`	string	Optional. If a `group` name is not specified, the endpoint is given a name of "default".
`max_age`	number	Required. A non-negative integer that defines the lifetime of the endpoint in seconds. A value of "0" will cause the endpoint group to be removed from the user agent’s reporting cache.
`endpoints`	Array<Object>	Required. An array of JSON objects that specify the actual URL of your report collector.
`include_subdomains`	boolean	Optional. A boolean that enables the endpoint group for all subdomains of the current origin's host. If omitted or anything other than "true", the subdomains are not reported to the endpoint.

The group name is a unique name used to associate a string with an endpoint. Use this name in other places that integrate with the Reporting API to refer to a specific endpoint group.

The max-age field is also required and specifies how long the browser should use the endpoint and report errors to it.

The endpoints field is an array to provide failover and load balancing features. See the section on Failover and load balancing. It's important to note that the browser will select only one endpoint, even if the group lists several collectors in endpoints. If you want to send a report to several servers at once, your backend will need to forward the reports.

How does the browser send reports? #

The browser periodically batches reports and sends them to the reporting endpoints that you configure.

To send reports, the browser issues a POST request with Content-Type: application/reports+json and a body containing the array of warnings/errors which were captured.

When does the browser send reports? #

Reports are delivered out-of-band from your app, meaning the browser controls when reports are sent to your server(s).

The browser attempts to deliver queued reports at the most opportune time. This may be as soon as they're ready (in order to provide timely feedback to the developer) but the browser can also delay delivery if it's busy processing higher priority work, or if the user is on a slow and/or congested network at the time. The browser may also prioritize sending reports about a particular origin first, if the user is a frequent visitor.

There's little to no performance concern (e.g. network contention with your app) when using the Reporting API. There's also no way to control when the browser sends queued reports.

Configuring multiple endpoints #

A single response can configure several endpoints at once by sending multiple Report-To headers:

Report-To: {
             "group": "default",
             "max_age": 10886400,
             "endpoints": [{
               "url": "https://example.com/browser-reports"
             }]
           }
Report-To: {
             "group": "network-errors-endpoint",
             "max_age": 10886400,
             "endpoints": [{
               "url": "https://example.com/network-errors"
             }]
           }

or by combining them into a single HTTP header:

Report-To: {
             "group": "network-errors-endpoint",
             "max_age": 10886400,
             "endpoints": [{
               "url": "https://example.com/network-errors"
             }]
           },
           {
             "max_age": 10886400,
             "endpoints": [{
               "url": "https://example.com/browser-errors"
             }]
           }

Once you've sent the Report-To header, the browser caches the endpoints according to their max_age values, and sends all of those nasty console warnings/errors to your URLs.

Failover and load balancing #

Most of the time you'll be configuring one URL collector per group. However, since reporting can generate a good deal of traffic, the spec includes failover and load-balancing features inspired by the DNS SRV record.

The browser will do its best to deliver a report to at most one endpoint in a group. Endpoints can be assigned a weight to distribute load, with each endpoint receiving a specified fraction of reporting traffic. Endpoints can also be assigned a priority to set up fallback collectors.

Fallback collectors are only tried when uploads to primary collectors fail.

Example: Create a fallback collector at https://backup.com/reports:

Report-To: {
             "group": "endpoint-1",
             "max_age": 10886400,
             "endpoints": [
               {"url": "https://example.com/reports", "priority": 1},
               {"url": "https://backup.com/reports", "priority": 2}
             ]
           }

Setting up Network Error Logging #

Setup #

To use NEL, set up the Report-To header with a collector that uses a named group:

Report-To: {
    ...
  }, {
    "group": "network-errors",
    "max_age": 2592000,
    "endpoints": [{
      "url": "https://analytics.provider.com/networkerrors"
    }]
  }

Next, send the NEL response header to start collecting errors. Since NEL is opt-in for an origin, you only need to send the header once. Both NEL and Report-To will apply to future requests to the same origin and will continue to collect errors according to the max_age value that was used to set up the collector.

The header value should be a JSON object that contains a max_age and report_to field. Use the latter to reference the group name of your network errors collector:

GET /index.html HTTP/1.1
NEL: {"report_to": "network-errors", "max_age": 2592000}

Subresources #

Example: If example.com loads foobar.com/cat.gif and that resource fails to load:

foobar.com's NEL collector is notified
example.com's NEL collector is not notified

The rule of thumb is that NEL reproduces server-side logs, just generated on the client.

Since example.com has no visibility into foobar.com's server logs, it also has no visibility into its NEL reports.

Debugging report configurations #

If you don't see reports showing up on your server, head over to chrome://net-export/. That page is useful for verifying things are configured correctly and reports are being sent out properly.

What about ReportingObserver? #

ReportingObserver is a related, but different reporting mechanism. It's based on JavaScript calls. It's not suited for network error logging, as network errors can't be intercepted via JavaScript.

Example server #

Below is an example Node server that uses Express. It shows how to configure reporting for network errors, and creates a dedicated handler to capture the result.

const express = require('express');

const app = express();
app.use(
  express.json({
    type: ['application/json', 'application/reports+json'],
  }),
);
app.use(express.urlencoded());

app.get('/', (request, response) => {
  // Note: report_to and not report-to for NEL.
  response.set('NEL', `{"report_to": "network-errors", "max_age": 2592000}`);

  // The Report-To header tells the browser where to send network errors.
  // The default group (first example below) captures interventions and
  // deprecation reports. Other groups, like the network-error group, are referenced by their "group" name.
  response.set(
    'Report-To',
    `{
    "max_age": 2592000,
    "endpoints": [{
      "url": "https://reporting-observer-api-demo.glitch.me/reports"
    }],
  }, {
    "group": "network-errors",
    "max_age": 2592000,
    "endpoints": [{
      "url": "https://reporting-observer-api-demo.glitch.me/network-reports"
    }]
  }`,
  );

  response.sendFile('./index.html');
});

function echoReports(request, response) {
  // Record report in server logs or otherwise process results.
  for (const report of request.body) {
    console.log(report.body);
  }
  response.send(request.body);
}

app.post('/network-reports', (request, response) => {
  console.log(`${request.body.length} Network error reports:`);
  echoReports(request, response);
});

const listener = app.listen(process.env.PORT, () => {
  console.log(`Your app is listening on port ${listener.address().port}`);
});

Custom Elements v1 - Reusable Web Components

2017-08-14T00:00:00Z

With Custom Elements, web developers can create new HTML tags, beef-up existing HTML tags, or extend the components other developers have authored. The API is the foundation of web components. It brings a web standards-based way to create reusable components using nothing more than vanilla JS/HTML/CSS. The result is less code, modular code, and more reuse in our apps.

Introduction #

The browser gives us an excellent tool for structuring web applications. It's called HTML. You may have heard of it! It's declarative, portable, well supported, and easy to work with. Great as HTML may be, its vocabulary and extensibility are limited. The HTML living standard has always lacked a way to automatically associate JS behavior with your markup… until now.

Custom elements are the answer to modernizing HTML, filling in the missing pieces, and bundling structure with behavior. If HTML doesn't provide the solution to a problem, we can create a custom element that does. Custom elements teach the browser new tricks while preserving the benefits of HTML.

Defining a new element #

To define a new HTML element we need the power of JavaScript!

The customElements global is used for defining a custom element and teaching the browser about a new tag. Call customElements.define() with the tag name you want to create and a JavaScript class that extends the base HTMLElement.

Example - defining a mobile drawer panel, <app-drawer>:

class AppDrawer extends HTMLElement {...}
window.customElements.define('app-drawer', AppDrawer);

// Or use an anonymous class if you don't want a named constructor in current scope.
window.customElements.define('app-drawer', class extends HTMLElement {...});

Example usage:

<app-drawer></app-drawer>

It's important to remember that using a custom element is no different than using a <div> or any other element. Instances can be declared on the page, created dynamically in JavaScript, event listeners can be attached, etc. Keep reading for more examples.

Defining an element's JavaScript API #

The functionality of a custom element is defined using an ES2015 class which extends HTMLElement. Extending HTMLElement ensures the custom element inherits the entire DOM API and means any properties/methods that you add to the class become part of the element's DOM interface. Essentially, use the class to create a public JavaScript API for your tag.

Example - defining the DOM interface of <app-drawer>:

class AppDrawer extends HTMLElement {

  // A getter/setter for an open property.
  get open() {
    return this.hasAttribute('open');
  }

  set open(val) {
    // Reflect the value of the open property as an HTML attribute.
    if (val) {
      this.setAttribute('open', '');
    } else {
      this.removeAttribute('open');
    }
    this.toggleDrawer();
  }

  // A getter/setter for a disabled property.
  get disabled() {
    return this.hasAttribute('disabled');
  }

  set disabled(val) {
    // Reflect the value of the disabled property as an HTML attribute.
    if (val) {
      this.setAttribute('disabled', '');
    } else {
      this.removeAttribute('disabled');
    }
  }

  // Can define constructor arguments if you wish.
  constructor() {
    // If you define a constructor, always call super() first!
    // This is specific to CE and required by the spec.
    super();

    // Setup a click listener on <app-drawer> itself.
    this.addEventListener('click', e => {
      // Don't toggle the drawer if it's disabled.
      if (this.disabled) {
        return;
      }
      this.toggleDrawer();
    });
  }

  toggleDrawer() {
    // ...
  }
}

customElements.define('app-drawer', AppDrawer);

In this example, we're creating a drawer that has an open property, disabled property, and a toggleDrawer() method. It also reflects properties as HTML attributes.

A neat feature of custom elements is that this inside a class definition refers to the DOM element itself i.e. the instance of the class. In our example, this refers to <app-drawer>. This (😉) is how the element can attach a click listener to itself! And you're not limited to event listeners. The entire DOM API is available inside element code. Use this to access the element's properties, inspect its children (this.children), query nodes (this.querySelectorAll('.items')), etc.

Rules on creating custom elements #

The name of a custom element must contain a dash (-). So <x-tags>, <my-element>, and <my-awesome-app> are all valid names, while <tabs> and <foo_bar> are not. This requirement is so the HTML parser can distinguish custom elements from regular elements. It also ensures forward compatibility when new tags are added to HTML.
You can't register the same tag more than once. Attempting to do so will throw a DOMException. Once you've told the browser about a new tag, that's it. No take backs.
Custom elements cannot be self-closing because HTML only allows a few elements to be self-closing. Always write a closing tag (<app-drawer></app-drawer>).

Custom element reactions #

A custom element can define special lifecycle hooks for running code during interesting times of its existence. These are called custom element reactions.

Name	Called when
`constructor`	An instance of the element is created or upgraded. Useful for initializing state, setting up event listeners, or creating a shadow dom. See the spec for restrictions on what you can do in the `constructor`.
`connectedCallback`	Called every time the element is inserted into the DOM. Useful for running setup code, such as fetching resources or rendering. Generally, you should try to delay work until this time.
`disconnectedCallback`	Called every time the element is removed from the DOM. Useful for running clean up code.
`attributeChangedCallback(attrName, oldVal, newVal)`	Called when an observed attribute has been added, removed, updated, or replaced. Also called for initial values when an element is created by the parser, or upgraded. Note: only attributes listed in the `observedAttributes` property will receive this callback.
`adoptedCallback`	The custom element has been moved into a new `document` (e.g. someone called `document.adoptNode(el)`).

Note: The browser calls the attributeChangedCallback() for any attributes listed in the observedAttributes array (see Observing changes to attributes). Essentially, this is a performance optimization. When users change a common attribute like style or class, you don't want to be spammed with tons of callbacks.

Reaction callbacks are synchronous. If someone calls el.setAttribute() on your element, the browser will immediately call attributeChangedCallback(). Similarly, you'll receive a disconnectedCallback() right after your element is removed from the DOM (e.g. the user calls el.remove()).

Example: adding custom element reactions to <app-drawer>:

class AppDrawer extends HTMLElement {
  constructor() {
    super(); // always call super() first in the constructor.
    // ...
  }

  connectedCallback() {
    // ...
  }

  disconnectedCallback() {
    // ...
  }

  attributeChangedCallback(attrName, oldVal, newVal) {
    // ...
  }
}

Define reactions if/when it make senses. If your element is sufficiently complex and opens a connection to IndexedDB in connectedCallback(), do the necessary cleanup work in disconnectedCallback(). But be careful! You can't rely on your element being removed from the DOM in all circumstances. For example, disconnectedCallback() will never be called if the user closes the tab.

Properties and attributes #

Reflecting properties to attributes #

It's common for HTML properties to reflect their value back to the DOM as an HTML attribute. For example, when the values of hidden or id are changed in JS:

div.id = 'my-id';
div.hidden = true;

the values are applied to the live DOM as attributes:

<div id="my-id" hidden>

This is called "reflecting properties to attributes". Almost every property in HTML does this. Why? Attributes are also useful for configuring an element declaratively and certain APIs like accessibility and CSS selectors rely on attributes to work.

Reflecting a property is useful anywhere you want to keep the element's DOM representation in sync with its JavaScript state. One reason you might want to reflect a property is so user-defined styling applies when JS state changes.

Recall our <app-drawer>. A consumer of this component may want to fade it out and/or prevent user interaction when it's disabled:

app-drawer[disabled] {
  opacity: 0.5;
  pointer-events: none;
}

When the disabled property is changed in JS, we want that attribute to be added to the DOM so the user's selector matches. The element can provide that behavior by reflecting the value to an attribute of the same name:

get disabled() {
  return this.hasAttribute('disabled');
}

set disabled(val) {
  // Reflect the value of `disabled` as an attribute.
  if (val) {
    this.setAttribute('disabled', '');
  } else {
    this.removeAttribute('disabled');
  }
  this.toggleDrawer();
}

Observing changes to attributes #

HTML attributes are a convenient way for users to declare initial state:

<app-drawer open disabled></app-drawer>

Elements can react to attribute changes by defining a attributeChangedCallback. The browser will call this method for every change to attributes listed in the observedAttributes array.

class AppDrawer extends HTMLElement {
  // ...

  static get observedAttributes() {
    return ['disabled', 'open'];
  }

  get disabled() {
    return this.hasAttribute('disabled');
  }

  set disabled(val) {
    if (val) {
      this.setAttribute('disabled', '');
    } else {
      this.removeAttribute('disabled');
    }
  }

  // Only called for the disabled and open attributes due to observedAttributes
  attributeChangedCallback(name, oldValue, newValue) {
    // When the drawer is disabled, update keyboard/screen reader behavior.
    if (this.disabled) {
      this.setAttribute('tabindex', '-1');
      this.setAttribute('aria-disabled', 'true');
    } else {
      this.setAttribute('tabindex', '0');
      this.setAttribute('aria-disabled', 'false');
    }
    // TODO: also react to the open attribute changing.
  }
}

In the example, we're setting additional attributes on the <app-drawer> when a disabled attribute is changed. Although we're not doing it here, you could also use the attributeChangedCallback to keep a JS property in sync with its attribute.

Element upgrades #

Progressively enhanced HTML #

We've already learned that custom elements are defined by calling customElements.define(). But this doesn't mean you have to define + register a custom element all in one go.

Custom elements can be used before their definition is registered.

Progressive enhancement is a feature of custom elements. In other words, you can declare a bunch of <app-drawer> elements on the page and never invoke customElements.define('app-drawer', ...) until much later. This is because the browser treats potential custom elements differently thanks to unknown tags. The process of calling define() and endowing an existing element with a class definition is called "element upgrades".

To know when a tag name becomes defined, you can use window.customElements.whenDefined(). It returns a Promise that resolves when the element becomes defined.

customElements.whenDefined('app-drawer').then(() => {
  console.log('app-drawer defined');
});

Example - delay work until a set of child elements are upgraded

<share-buttons>
  <social-button type="twitter"><a href="...">Twitter</a></social-button>
  <social-button type="fb"><a href="...">Facebook</a></social-button>
  <social-button type="plus"><a href="...">G+</a></social-button>
</share-buttons>

// Fetch all the children of <share-buttons> that are not defined yet.
let undefinedButtons = buttons.querySelectorAll(':not(:defined)');

let promises = [...undefinedButtons].map((socialButton) => {
  return customElements.whenDefined(socialButton.localName);
});

// Wait for all the social-buttons to be upgraded.
Promise.all(promises).then(() => {
  // All social-button children are ready.
});

Element-defined content #

Custom elements can manage their own content by using the DOM APIs inside element code. Reactions come in handy for this.

Example - create an element with some default HTML:

customElements.define('x-foo-with-markup', class extends HTMLElement {
  connectedCallback() {
    this.innerHTML = "<b>I'm an x-foo-with-markup!</b>";
  }
  // ...
});

Declaring this tag will produce:

<x-foo-with-markup>
  <b>I'm an x-foo-with-markup!</b>
</x-foo-with-markup>

Creating an element that uses Shadow DOM #

Note: I'm not going to cover the features of Shadow DOM in this article, but it's a powerful API to combine with custom elements. By itself, Shadow DOM is composition tool. When it's used in conjunction with custom elements, magical things happen.

Shadow DOM provides a way for an element to own, render, and style a chunk of DOM that's separate from the rest of the page. Heck, you could even hide away an entire app within a single tag:

<!-- chat-app's implementation details are hidden away in Shadow DOM. -->
<chat-app></chat-app>

To use Shadow DOM in a custom element, call this.attachShadow inside your constructor:

let tmpl = document.createElement('template');
tmpl.innerHTML = `
  <style>:host { ... }</style> <!-- look ma, scoped styles -->
  <b>I'm in shadow dom!</b>
  <slot></slot>
`;

customElements.define('x-foo-shadowdom', class extends HTMLElement {
  constructor() {
    super(); // always call super() first in the constructor.

    // Attach a shadow root to the element.
    let shadowRoot = this.attachShadow({mode: 'open'});
    shadowRoot.appendChild(tmpl.content.cloneNode(true));
  }
  // ...
});

Example usage:

<x-foo-shadowdom>
  <p><b>User's</b> custom text</p>
</x-foo-shadowdom>

<!-- renders as -->
<x-foo-shadowdom>
  #shadow-root
  <b>I'm in shadow dom!</b>
  <slot></slot> <!-- slotted content appears here -->
</x-foo-shadowdom>

User's custom text

Creating elements from a `<template>` #

For those unfamiliar, the <template> element allows you to declare fragments of the DOM which are parsed, inert at page load, and can be activated later at runtime. It's another API primitive in the web components family. Templates are an ideal placeholder for declaring the structure of a custom element.

Example: registering an element with Shadow DOM content created from a <template>:

<template id="x-foo-from-template">
  <style>
    p { color: green; }
  </style>
  <p>I'm in Shadow DOM. My markup was stamped from a &lt;template&gt;.</p>
</template>

<script>
  let tmpl = document.querySelector('#x-foo-from-template');
  // If your code is inside of an HTML Import you'll need to change the above line to:
  // let tmpl = document.currentScript.ownerDocument.querySelector('#x-foo-from-template');

  customElements.define('x-foo-from-template', class extends HTMLElement {
    constructor() {
      super(); // always call super() first in the constructor.
      let shadowRoot = this.attachShadow({mode: 'open'});
      shadowRoot.appendChild(tmpl.content.cloneNode(true));
    }
    // ...
  });
</script>

These few lines of code pack a punch. Let's understand the key things going on:

We're defining a new element in HTML: <x-foo-from-template>
The element's Shadow DOM is created from a <template>
The element's DOM is local to the element thanks to the Shadow DOM
The element's internal CSS is scoped to the element thanks to the Shadow DOM

Styling a custom element #

Even if your element defines its own styling using Shadow DOM, users can style your custom element from their page. These are called "user-defined styles".

<!-- user-defined styling -->
<style>
  app-drawer {
    display: flex;
  }
  panel-item {
    transition: opacity 400ms ease-in-out;
    opacity: 0.3;
    flex: 1;
    text-align: center;
    border-radius: 50%;
  }
  panel-item:hover {
    opacity: 1.0;
    background: rgb(255, 0, 255);
    color: white;
  }
  app-panel > panel-item {
    padding: 5px;
    list-style: none;
    margin: 0 7px;
  }
</style>

<app-drawer>
  <panel-item>Do</panel-item>
  <panel-item>Re</panel-item>
  <panel-item>Mi</panel-item>
</app-drawer>

You might be asking yourself how CSS specificity works if the element has styles defined within Shadow DOM. In terms of specificity, user styles win. They'll always override element-defined styling. See the section on Creating an element that uses Shadow DOM.

Pre-styling unregistered elements #

Before an element is upgraded you can target it in CSS using the :defined pseudo-class. This is useful for pre-styling a component. For example, you may wish to prevent layout or other visual FOUC by hiding undefined components and fading them in when they become defined.

Example - hide <app-drawer> before it's defined:

app-drawer:not(:defined) {
  /* Pre-style, give layout, replicate app-drawer's eventual styles, etc. */
  display: inline-block;
  height: 100vh;
  opacity: 0;
  transition: opacity 0.3s ease-in-out;
}

After <app-drawer> becomes defined, the selector (app-drawer:not(:defined)) no longer matches.

Extending elements #

The Custom Elements API is useful for creating new HTML elements, but it's also useful for extending other custom elements or even the browser's built-in HTML.

Extending a custom element #

Extending another custom element is done by extending its class definition.

Example - create <fancy-app-drawer> that extends <app-drawer>:

class FancyDrawer extends AppDrawer {
  constructor() {
    super(); // always call super() first in the constructor. This also calls the extended class' constructor.
    // ...
  }

  toggleDrawer() {
    // Possibly different toggle implementation?
    // Use ES2015 if you need to call the parent method.
    // super.toggleDrawer()
  }

  anotherMethod() {
    // ...
  }
}

customElements.define('fancy-app-drawer', FancyDrawer);

Extending native HTML elements #

Let's say you wanted to create a fancier <button>. Instead of replicating the behavior and functionality of <button>, a better option is to progressively enhance the existing element using custom elements.

A customized built-in element is a custom element that extends one of the browser's built-in HTML tags. The primary benefit of extending an existing element is to gain all of its features (DOM properties, methods, accessibility). There's no better way to write a progressive web app than to progressively enhance existing HTML elements.

Note: Only Chrome 67 supports customized built-in elements (status) right now. Edge and Firefox will implement it, but Safari has chosen not to implement it. This is unfortunate for accessibility and progressive enhancement. If you think extending native HTML elements is useful, voice your thoughts on 509 and 662 on Github.

To extend an element, you'll need to create a class definition that inherits from the correct DOM interface. For example, a custom element that extends <button> needs to inherit from HTMLButtonElement instead of HTMLElement. Similarly, an element that extends <img> needs to extend HTMLImageElement.

Example - extending <button>:

// See https://html.spec.whatwg.org/multipage/indices.html#element-interfaces
// for the list of other DOM interfaces.
class FancyButton extends HTMLButtonElement {
  constructor() {
    super(); // always call super() first in the constructor.
    this.addEventListener('click', e => this.drawRipple(e.offsetX, e.offsetY));
  }

  // Material design ripple animation.
  drawRipple(x, y) {
    let div = document.createElement('div');
    div.classList.add('ripple');
    this.appendChild(div);
    div.style.top = `${y - div.clientHeight/2}px`;
    div.style.left = `${x - div.clientWidth/2}px`;
    div.style.backgroundColor = 'currentColor';
    div.classList.add('run');
    div.addEventListener('transitionend', (e) => div.remove());
  }
}

customElements.define('fancy-button', FancyButton, {extends: 'button'});

Notice that the call to define() changes slightly when extending a native element. The required third parameter tells the browser which tag you're extending. This is necessary because many HTML tags share the same DOM interface. <section>, <address>, and <em> (among others) all share HTMLElement; both <q> and <blockquote> share HTMLQuoteElement; etc… Specifying {extends: 'blockquote'} lets the browser know you're creating a souped-up <blockquote> instead of a <q>. See the HTML spec for the full list of HTML's DOM interfaces.

Note: Extending HTMLButtonElement endows our fancy button with all the DOM properties/methods of <button>. That checks off a bunch of stuff we don't have to implement ourselves: disabled property, click() method, keydown listeners, tabindex management. Instead, our focus can be progressively enhancing <button> with custom functionality, namely, the drawRipple() method. Less code, more reuse!

Consumers of a customized built-in element can use it in several ways. They can declare it by adding the is="" attribute on the native tag:

<!-- This <button> is a fancy button. -->
<button is="fancy-button" disabled>Fancy button!</button>

create an instance in JavaScript:

// Custom elements overload createElement() to support the is="" attribute.
let button = document.createElement('button', {is: 'fancy-button'});
button.textContent = 'Fancy button!';
button.disabled = true;
document.body.appendChild(button);

or use the new operator:

let button = new FancyButton();
button.textContent = 'Fancy button!';
button.disabled = true;

Here's another example that extends <img>.

Example - extending <img>:

customElements.define('bigger-img', class extends Image {
  // Give img default size if users don't specify.
  constructor(width=50, height=50) {
    super(width * 10, height * 10);
  }
}, {extends: 'img'});

Users declare this component as:

<!-- This <img> is a bigger img. -->
<img is="bigger-img" width="15" height="20">

or create an instance in JavaScript:

const BiggerImage = customElements.get('bigger-img');
const image = new BiggerImage(15, 20); // pass constructor values like so.
console.assert(image.width === 150);
console.assert(image.height === 200);

Misc details #

Unknown elements vs. undefined custom elements #

HTML is lenient and flexible to work with. For example, declare <randomtagthatdoesntexist> on a page and the browser is perfectly happy accepting it. Why do non-standard tags work? The answer is the HTML specification allows it. Elements that are not defined by the specification get parsed as HTMLUnknownElement.

The same is not true for custom elements. Potential custom elements are parsed as an HTMLElement if they're created with a valid name (includes a "-"). You can check this in a browser that supports custom elements. Fire up the Console: Ctrl+Shift+J (or Cmd+Opt+J on Mac) and paste in the following lines of code:

// "tabs" is not a valid custom element name
document.createElement('tabs') instanceof HTMLUnknownElement === true

// "x-tabs" is a valid custom element name
document.createElement('x-tabs') instanceof HTMLElement === true

API reference #

The customElements global defines useful methods for working with custom elements.

define(tagName, constructor, options)

Defines a new custom element in the browser.

Example

customElements.define('my-app', class extends HTMLElement { ... });
customElements.define(
    'fancy-button', class extends HTMLButtonElement { ... }, {extends: 'button'});

get(tagName)

Given a valid custom element tag name, returns the element's constructor. Returns undefined if no element definition has been registered.

Example

let Drawer = customElements.get('app-drawer');
let drawer = new Drawer();

whenDefined(tagName)

Returns a Promise that resolves when the custom element is defined. If the element is already defined, resolve immediately. Rejects if the tag name is not a valid custom element name.

Example

customElements.whenDefined('app-drawer').then(() => {
  console.log('ready!');
});

History and browser support #

If you've been following web components for the last couple of years, you'll know that Chrome 36+ implemented a version of the Custom Elements API that uses document.registerElement() instead of customElements.define(). That's now considered a deprecated version of the standard, called v0. customElements.define() is the new hotness and what browser vendors are starting to implement. It's called Custom Elements v1.

If you happen to be interested in the old v0 spec, check out the html5rocks article.

Browser support #

Chrome 54 (status), Safari 10.1 (status), and Firefox 63 (status) have Custom Elements v1. Edge has begun development.

To feature detect custom elements, check for the existence of window.customElements:

const supportsCustomElementsV1 = 'customElements' in window;

Polyfill #

Until browser support is widely available, there's a standalone polyfill available for Custom Elements v1. However, we recommend using the webcomponents.js loader to optimally load the web components polyfills. The loader uses feature detection to asynchronously load only the necessary pollyfills required by the browser.

Install it:

npm install --save @webcomponents/webcomponentsjs

Usage:

<!-- Use the custom element on the page. -->
<my-element></my-element>

<!-- Load polyfills; note that "loader" will load these async -->
<script src="node_modules/@webcomponents/webcomponentsjs/webcomponents-loader.js" defer></script>

<!-- Load a custom element definitions in `waitFor` and return a promise -->
<script type="module">
  function loadScript(src) {
    return new Promise(function(resolve, reject) {
      const script = document.createElement('script');
      script.src = src;
      script.onload = resolve;
      script.onerror = reject;
      document.head.appendChild(script);
    });
  }

  WebComponents.waitFor(() => {
    // At this point we are guaranteed that all required polyfills have
    // loaded, and can use web components APIs.
    // Next, load element definitions that call `customElements.define`.
    // Note: returning a promise causes the custom elements
    // polyfill to wait until all definitions are loaded and then upgrade
    // the document in one batch, for better performance.
    return loadScript('my-element.js');
  });
</script>

Conclusion #

Custom elements give us a new tool for defining new HTML tags in the browser and creating reusable components. Combine them with the other new platform primitives like Shadow DOM and <template>, and we start to realize the grand picture of Web Components:

Cross-browser (web standard) for creating and extending reusable components.
Requires no library or framework to get started. Vanilla JS/HTML FTW!
Provides a familiar programming model. It's just DOM/CSS/HTML.
Works well with other new web platform features (Shadow DOM, <template>, CSS custom properties, etc.)
Tightly integrated with the browser's DevTools.
Leverage existing accessibility features.

Shadow DOM v1 - Self-Contained Web Components

2016-08-01T00:00:00Z

Summary #

Shadow DOM removes the brittleness of building web apps. The brittleness comes from the global nature of HTML, CSS, and JS. Over the years we've invented an exorbitant number of tools to circumvent the issues. For example, when you use a new HTML id/class, there's no telling if it will conflict with an existing name used by the page. Subtle bugs creep up, CSS specificity becomes a huge issue (!important all the things!), style selectors grow out of control, and performance can suffer. The list goes on.

Shadow DOM fixes CSS and DOM. It introduces scoped styles to the web platform. Without tools or naming conventions, you can bundle CSS with markup, hide implementation details, and author self-contained components in vanilla JavaScript.

Introduction #

Note: Already familiar with Shadow DOM? This article describes the new Shadow DOM v1 spec. If you've been using Shadow DOM, chances are you're familiar with the v0 version that shipped in Chrome 35, and the webcomponents.js polyfills. The concepts are the same, but the v1 spec has important API differences. It's also the version that all major browsers have agreed to implement, with implementations already in Safari, Chrome and Firefox. Keep reading to see what's new or check out the section on History and browser support for more info.

Shadow DOM is one of the three Web Component standards: HTML Templates, Shadow DOM and Custom elements. HTML Imports used to be part of the list but now are considered deprecated.

You don't have to author web components that use shadow DOM. But when you do, you take advantage of its benefits (CSS scoping, DOM encapsulation, composition) and build reusable custom elements, which are resilient, highly configurable, and extremely reusable. If custom elements are the way to create a new HTML (with a JS API), shadow DOM is the way you provide its HTML and CSS. The two APIs combine to make a component with self-contained HTML, CSS, and JavaScript.

Shadow DOM is designed as a tool for building component-based apps. Therefore, it brings solutions for common problems in web development:

Isolated DOM: A component's DOM is self-contained (e.g. document.querySelector() won't return nodes in the component's shadow DOM).
Scoped CSS: CSS defined inside shadow DOM is scoped to it. Style rules don't leak out and page styles don't bleed in.
Composition: Design a declarative, markup-based API for your component.
Simplifies CSS - Scoped DOM means you can use simple CSS selectors, more generic id/class names, and not worry about naming conflicts.
Productivity - Think of apps in chunks of DOM rather than one large (global) page.

`fancy-tabs` demo #

Throughout this article, I'll be referring to a demo component (<fancy-tabs>) and referencing code snippets from it. If your browser supports the APIs, you should see a live demo of it just below. Otherwise, check out the full source on Github.

View source on Github

What is shadow DOM? #

Background on DOM #

HTML powers the web because it's easy to work with. By declaring a few tags, you can author a page in seconds that has both presentation and structure. However, by itself HTML isn't all that useful. It's easy for humans to understand a text- based language, but machines need something more. Enter the Document Object Model, or DOM.

When the browser loads a web page it does a bunch of interesting stuff. One of the things it does is transform the author's HTML into a live document. Basically, to understand the page's structure, the browser parses HTML (static strings of text) into a data model (objects/nodes). The browser preserves the HTML's hierarchy by creating a tree of these nodes: the DOM. The cool thing about DOM is that it's a live representation of your page. Unlike the static HTML we author, the browser-produced nodes contain properties, methods, and best of all… can be manipulated by programs! That's why we're able to create DOM elements directly using JavaScript:

const header = document.createElement('header');
const h1 = document.createElement('h1');
h1.textContent = 'Hello DOM';
header.appendChild(h1);
document.body.appendChild(header);

produces the following HTML markup:

<body>
    <header>
    <h1>Hello DOM</h1>
    </header>
</body>

All that is well and good. Then what the heck is shadow DOM?

DOM… in the shadows #

Shadow DOM is just normal DOM with two differences: 1) how it's created/used and 2) how it behaves in relation to the rest of the page. Normally, you create DOM nodes and append them as children of another element. With shadow DOM, you create a scoped DOM tree that's attached to the element, but separate from its actual children. This scoped subtree is called a shadow tree. The element it's attached to is its shadow host. Anything you add in the shadows becomes local to the hosting element, including <style>. This is how shadow DOM achieves CSS style scoping.

Creating shadow DOM #

A shadow root is a document fragment that gets attached to a “host” element. The act of attaching a shadow root is how the element gains its shadow DOM. To create shadow DOM for an element, call element.attachShadow():

const header = document.createElement('header');
const shadowRoot = header.attachShadow({mode: 'open'});
shadowRoot.innerHTML = '<h1>Hello Shadow DOM</h1>'; // Could also use appendChild().

// header.shadowRoot === shadowRoot
// shadowRoot.host === header

I'm using .innerHTML to fill the shadow root, but you could also use other DOM APIs. This is the web. We have choice.

The spec defines a list of elements that can't host a shadow tree. There are several reasons an element might be on the list:

The browser already hosts its own internal shadow DOM for the element (<textarea>, <input>).
It doesn't make sense for the element to host a shadow DOM (<img>).

For example, this doesn't work:

    document.createElement('input').attachShadow({mode: 'open'});
    // Error. `<input>` cannot host shadow dom.

Creating shadow DOM for a custom element #

Shadow DOM is particularly useful when creating custom elements. Use shadow DOM to compartmentalize an element's HTML, CSS, and JS, thus producing a "web component".

Example - a custom element attaches shadow DOM to itself, encapsulating its DOM/CSS:

// Use custom elements API v1 to register a new HTML tag and define its JS behavior
// using an ES6 class. Every instance of <fancy-tab> will have this same prototype.
customElements.define('fancy-tabs', class extends HTMLElement {
    constructor() {
    super(); // always call super() first in the constructor.

    // Attach a shadow root to <fancy-tabs>.
    const shadowRoot = this.attachShadow({mode: 'open'});
    shadowRoot.innerHTML = `
        <style>#tabs { ... }</style> <!-- styles are scoped to fancy-tabs! -->
        <div id="tabs">...</div>
        <div id="panels">...</div>
    `;
    }
    ...
});

There are a couple of interesting things going on here. The first is that the custom element creates its own shadow DOM when an instance of <fancy-tabs> is created. That's done in the constructor(). Secondly, because we're creating a shadow root, the CSS rules inside the <style> will be scoped to <fancy-tabs>.

Note: When you try to run this example, you'll probably notice that nothing renders. The user's markup seemingly disappears! That's because the element's shadow DOM is rendered in place of its children. If you want to display the children, you need to tell the browser where to render them by placing a <slot> element in your shadow DOM. More on that later.

Composition and slots #

Composition is one of the least understood features of shadow DOM, but it's arguably the most important.

In our world of web development, composition is how we construct apps, declaratively out of HTML. Different building blocks (<div>s, <header>s, <form>s, <input>s) come together to form apps. Some of these tags even work with each other. Composition is why native elements like <select>, <details>, <form>, and <video> are so flexible. Each of those tags accepts certain HTML as children and does something special with them. For example, <select> knows how to render <option> and <optgroup> into dropdown and multi-select widgets. The <details> element renders <summary> as a expandable arrow. Even <video> knows how to deal with certain children: <source> elements don't get rendered, but they do affect the video's behavior. What magic!

Terminology: light DOM vs. shadow DOM #

Shadow DOM composition introduces a bunch of new fundamentals in web development. Before getting into the weeds, let's standardize on some terminology so we're speaking the same lingo.

Light DOM

The markup a user of your component writes. This DOM lives outside the component's shadow DOM. It is the element's actual children.

<better-button>
    <!-- the image and span are better-button's light DOM -->
    <img src="gear.svg" slot="icon">
    <span>Settings</span>
</better-button>

Shadow DOM

The DOM a component author writes. Shadow DOM is local to the component and defines its internal structure, scoped CSS, and encapsulates your implementation details. It can also define how to render markup that's authored by the consumer of your component.

#shadow-root
    <style>...</style>
    <slot name="icon"></slot>
    <span id="wrapper">
    <slot>Button</slot>
    </span>

Flattened DOM tree

The result of the browser distributing the user's light DOM into your shadow DOM, rendering the final product. The flattened tree is what you ultimately see in the DevTools and what's rendered on the page.

<better-button>
    #shadow-root
    <style>...</style>
    <slot name="icon">
        <img src="gear.svg" slot="icon">
    </slot>
    <span id="wrapper">
        <slot>
        <span>Settings</span>
        </slot>
    </span>
</better-button>

The <slot> element #

Shadow DOM composes different DOM trees together using the <slot> element. Slots are placeholders inside your component that users can fill with their own markup. By defining one or more slots, you invite outside markup to render in your component's shadow DOM. Essentially, you're saying "Render the user's markup over here".

Note: Slots are a way of creating a "declarative API" for a web component. They mix-in the user's DOM to help render the overall component, thus, composing different DOM trees together.

Elements are allowed to "cross" the shadow DOM boundary when a <slot> invites them in. These elements are called distributed nodes. Conceptually, distributed nodes can seem a bit bizarre. Slots don't physically move DOM; they render it at another location inside the shadow DOM.

A component can define zero or more slots in its shadow DOM. Slots can be empty or provide fallback content. If the user doesn't provide light DOM content, the slot renders its fallback content.

<!-- Default slot. If there's more than one default slot, the first is used. -->
<slot></slot>

<slot>fallback content</slot> <!-- default slot with fallback content -->

<slot> <!-- default slot entire DOM tree as fallback -->
    <h2>Title</h2>
    <summary>Description text</summary>
</slot>

You can also create named slots. Named slots are specific holes in your shadow DOM that users reference by name.

Example - the slots in <fancy-tabs>'s shadow DOM:

#shadow-root
    <div id="tabs">
    <slot id="tabsSlot" name="title"></slot> <!-- named slot -->
    </div>
    <div id="panels">
    <slot id="panelsSlot"></slot>
    </div>

Component users declare <fancy-tabs> like so:

<fancy-tabs>
    <button slot="title">Title</button>
    <button slot="title" selected>Title 2</button>
    <button slot="title">Title 3</button>
    <section>content panel 1</section>
    <section>content panel 2</section>
    <section>content panel 3</section>
</fancy-tabs>

<!-- Using <h2>'s and changing the ordering would also work! -->
<fancy-tabs>
    <h2 slot="title">Title</h2>
    <section>content panel 1</section>
    <h2 slot="title" selected>Title 2</h2>
    <section>content panel 2</section>
    <h2 slot="title">Title 3</h2>
    <section>content panel 3</section>
</fancy-tabs>

And if you're wondering, the flattened tree looks something like this:

<fancy-tabs>
    #shadow-root
    <div id="tabs">
        <slot id="tabsSlot" name="title">
        <button slot="title">Title</button>
        <button slot="title" selected>Title 2</button>
        <button slot="title">Title 3</button>
        </slot>
    </div>
    <div id="panels">
        <slot id="panelsSlot">
        <section>content panel 1</section>
        <section>content panel 2</section>
        <section>content panel 3</section>
        </slot>
    </div>
</fancy-tabs>

Notice our component is able to handle different configurations, but the flattened DOM tree remains the same. We can also switch from <button> to <h2>. This component was authored to handle different types of children… just like <select> does!

Styling #

There are many options for styling web components. A component that uses shadow DOM can be styled by the main page, define its own styles, or provide hooks (in the form of CSS custom properties) for users to override defaults.

Component-defined styles #

Hands down the most useful feature of shadow DOM is scoped CSS:

CSS selectors from the outer page don't apply inside your component.
Styles defined inside don't bleed out. They're scoped to the host element.

CSS selectors used inside shadow DOM apply locally to your component. In practice, this means we can use common id/class names again, without worrying about conflicts elsewhere on the page. Simpler CSS selectors are a best practice inside Shadow DOM. They're also good for performance.

Example - styles defined in a shadow root are local

#shadow-root
    <style>
    #panels {
        box-shadow: 0 2px 2px rgba(0, 0, 0, .3);
        background: white;
        ...
    }
    #tabs {
        display: inline-flex;
        ...
    }
    </style>
    <div id="tabs">
    ...
    </div>
    <div id="panels">
    ...
    </div>

Stylesheets are also scoped to the shadow tree:

#shadow-root
    <link rel="stylesheet" href="styles.css">
    <div id="tabs">
    ...
    </div>
    <div id="panels">
    ...
    </div>

Ever wonder how the <select> element renders a multi-select widget (instead of a dropdown) when you add the multiple attribute:

<select multiple>
  <option>Do</option>
  <option selected>Re</option>
  <option>Mi</option>
  <option>Fa</option>
  <option>So</option>
</select>

<select> is able to style itself differently based on the attributes you declare on it. Web components can style themselves too, by using the :host selector.

Example - a component styling itself

<style>
:host {
    display: block; /* by default, custom elements are display: inline */
    contain: content; /* CSS containment FTW. */
}
</style>

One gotcha with :host is that rules in the parent page have higher specificity than :host rules defined in the element. That is, outside styles win. This allows users to override your top-level styling from the outside. Also, :host only works in the context of a shadow root, so you can't use it outside of shadow DOM.

The functional form of :host(<selector>) allows you to target the host if it matches a <selector>. This is a great way for your component to encapsulate behaviors that react to user interaction or state or style internal nodes based on the host.

<style>
:host {
    opacity: 0.4;
    will-change: opacity;
    transition: opacity 300ms ease-in-out;
}
:host(:hover) {
    opacity: 1;
}
:host([disabled]) { /* style when host has disabled attribute. */
    background: grey;
    pointer-events: none;
    opacity: 0.4;
}
:host(.blue) {
    color: blue; /* color host when it has class="blue" */
}
:host(.pink) > #tabs {
    color: pink; /* color internal #tabs node when host has class="pink". */
}
</style>

Styling based on context #

:host-context(<selector>) matches the component if it or any of its ancestors matches <selector>. A common use for this is theming based on a component's surroundings. For example, many people do theming by applying a class to <html> or <body>:

<body class="darktheme">
    <fancy-tabs>
    ...
    </fancy-tabs>
</body>

:host-context(.darktheme) would style <fancy-tabs> when it's a descendant of .darktheme:

:host-context(.darktheme) {
    color: white;
    background: black;
}

:host-context() can be useful for theming, but an even better approach is to create style hooks using CSS custom properties.

Styling distributed nodes #

::slotted(<compound-selector>) matches nodes that are distributed into a <slot>.

Let's say we've created a name badge component:

<name-badge>
    <h2>Eric Bidelman</h2>
    <span class="title">
    Digital Jedi, <span class="company">Google</span>
    </span>
</name-badge>

The component's shadow DOM can style the user's <h2> and .title:

<style>
::slotted(h2) {
    margin: 0;
    font-weight: 300;
    color: red;
}
::slotted(.title) {
    color: orange;
}
/* DOESN'T WORK (can only select top-level nodes).
::slotted(.company),
::slotted(.title .company) {
    text-transform: uppercase;
}
*/
</style>
<slot></slot>

If you remember from before, <slot>s do not move the user's light DOM. When nodes are distributed into a <slot>, the <slot> renders their DOM but the nodes physically stay put. Styles that applied before distribution continue to apply after distribution. However, when the light DOM is distributed, it can take on additional styles (ones defined by the shadow DOM).

Another, more in-depth example from <fancy-tabs>:

const shadowRoot = this.attachShadow({mode: 'open'});
shadowRoot.innerHTML = `
    <style>
    #panels {
        box-shadow: 0 2px 2px rgba(0, 0, 0, .3);
        background: white;
        border-radius: 3px;
        padding: 16px;
        height: 250px;
        overflow: auto;
    }
    #tabs {
        display: inline-flex;
        -webkit-user-select: none;
        user-select: none;
    }
    #tabsSlot::slotted(*) {
        font: 400 16px/22px 'Roboto';
        padding: 16px 8px;
        ...
    }
    #tabsSlot::slotted([aria-selected="true"]) {
        font-weight: 600;
        background: white;
        box-shadow: none;
    }
    #panelsSlot::slotted([aria-hidden="true"]) {
        display: none;
    }
    </style>
    <div id="tabs">
    <slot id="tabsSlot" name="title"></slot>
    </div>
    <div id="panels">
    <slot id="panelsSlot"></slot>
    </div>
`;

In this example, there are two slots: a named slot for the tab titles, and a slot for the tab panel content. When the user selects a tab, we bold their selection and reveal its panel. That's done by selecting distributed nodes that have the selected attribute. The custom element's JS (not shown here) adds that attribute at the correct time.

Styling a component from the outside #

There are a couple of ways to style a component from the outside. The easiest way is to use the tag name as a selector:

fancy-tabs {
    width: 500px;
    color: red; /* Note: inheritable CSS properties pierce the shadow DOM boundary. */
}
fancy-tabs:hover {
    box-shadow: 0 3px 3px #ccc;
}

Outside styles always win over styles defined in shadow DOM. For example, if the user writes the selector fancy-tabs { width: 500px; }, it will trump the component's rule: :host { width: 650px;}.

Styling the component itself will only get you so far. But what happens if you want to style the internals of a component? For that, we need CSS custom properties.

Creating style hooks using CSS custom properties #

Users can tweak internal styles if the component's author provides styling hooks using CSS custom properties. Conceptually, the idea is similar to <slot>. You create "style placeholders" for users to override.

Example - <fancy-tabs> allows users to override the background color:

<!-- main page -->
<style>
    fancy-tabs {
    margin-bottom: 32px;
    --fancy-tabs-bg: black;
    }
</style>
<fancy-tabs background>...</fancy-tabs>

Inside its shadow DOM:

:host([background]) {
    background: var(--fancy-tabs-bg, #9E9E9E);
    border-radius: 10px;
    padding: 10px;
}

In this case, the component will use black as the background value since the user provided it. Otherwise, it would default to #9E9E9E.

Note: As the component author, you're responsible for letting developers know about CSS custom properties they can use. Consider it part of your component's public interface. Make sure to document styling hooks!

Advanced topics #

Creating closed shadow roots (should avoid) #

There's another flavor of shadow DOM called "closed" mode. When you create a closed shadow tree, outside JavaScript won't be able to access the internal DOM of your component. This is similar to how native elements like <video> work. JavaScript cannot access the shadow DOM of <video> because the browser implements it using a closed-mode shadow root.

Example - creating a closed shadow tree:

const div = document.createElement('div');
const shadowRoot = div.attachShadow({mode: 'closed'}); // close shadow tree
// div.shadowRoot === null
// shadowRoot.host === div

Other APIs are also affected by closed-mode:

Element.assignedSlot / TextNode.assignedSlot returns null
Event.composedPath() for events associated with elements inside the shadow DOM, returns []

Here's my summary of why you should never create web components with {mode: 'closed'}:

Artificial sense of security. There's nothing stopping an attacker from hijacking Element.prototype.attachShadow.

Closed mode prevents your custom element code from accessing its own shadow DOM. That's complete fail. Instead, you'll have to stash a reference for later if you want to use things like querySelector(). This completely defeats the original purpose of closed mode!

    customElements.define('x-element', class extends HTMLElement {
    constructor() {
    super(); // always call super() first in the constructor.
    this._shadowRoot = this.attachShadow({mode: 'closed'});
    this._shadowRoot.innerHTML = '<div class="wrapper"></div>';
    }
    connectedCallback() {
    // When creating closed shadow trees, you'll need to stash the shadow root
    // for later if you want to use it again. Kinda pointless.
    const wrapper = this._shadowRoot.querySelector('.wrapper');
    }
    ...
});

Closed mode makes your component less flexible for end users. As you build web components, there will come a time when you forget to add a feature. A configuration option. A use case the user wants. A common example is forgetting to include adequate styling hooks for internal nodes. With closed mode, there's no way for users to override defaults and tweak styles. Being able to access the component's internals is super helpful. Ultimately, users will fork your component, find another, or create their own if it doesn't do what they want :(

Working with slots in JS #

The shadow DOM API provides utilities for working with slots and distributed nodes. These come in handy when authoring a custom element.

slotchange event #

The slotchange event fires when a slot's distributed nodes changes. For example, if the user adds/removes children from the light DOM.

const slot = this.shadowRoot.querySelector('#slot');
slot.addEventListener('slotchange', e => {
    console.log('light dom children changed!');
});

To monitor other types of changes to light DOM, you can setup a MutationObserver in your element's constructor.

What elements are being rendering in a slot? #

Sometimes it's useful to know what elements are associated with a slot. Call slot.assignedNodes() to find which elements the slot is rendering. The {flatten: true} option will also return a slot's fallback content (if no nodes are being distributed).

As an example, let's say your shadow DOM looks like this:

<slot><b>fallback content</b></slot>

Usage	Call	Result
<my-component>component text</my-component>	`slot.assignedNodes();`	`[component text]`
<my-component></my-component>	`slot.assignedNodes();`	`[]`
<my-component></my-component>	`slot.assignedNodes({flatten: true});`	`[<b>fallback content</b>]`

What slot is an element assigned to? #

Answering the reverse question is also possible. element.assignedSlot tells you which of the component slots your element is assigned to.

The Shadow DOM event model #

When an event bubbles up from shadow DOM it's target is adjusted to maintain the encapsulation that shadow DOM provides. That is, events are re-targeted to look like they've come from the component rather than internal elements within your shadow DOM. Some events do not even propagate out of shadow DOM.

The events that do cross the shadow boundary are:

Focus Events: blur, focus, focusin, focusout
Mouse Events: click, dblclick, mousedown, mouseenter, mousemove, etc.
Wheel Events: wheel
Input Events: beforeinput, input
Keyboard Events: keydown, keyup
Composition Events: compositionstart, compositionupdate, compositionend
DragEvent: dragstart, drag, dragend, drop, etc.

Tips

If the shadow tree is open, calling event.composedPath() will return an array of nodes that the event traveled through.

Using custom events #

Custom DOM events which are fired on internal nodes in a shadow tree do not bubble out of the shadow boundary unless the event is created using the composed: true flag:

// Inside <fancy-tab> custom element class definition:
selectTab() {
    const tabs = this.shadowRoot.querySelector('#tabs');
    tabs.dispatchEvent(new Event('tab-select', {bubbles: true, composed: true}));
}

If composed: false (default), consumers won't be able to listen for the event outside of your shadow root.

<fancy-tabs></fancy-tabs>
<script>
    const tabs = document.querySelector('fancy-tabs');
    tabs.addEventListener('tab-select', e => {
    // won't fire if `tab-select` wasn't created with `composed: true`.
    });
</script>

Handling focus #

If you recall from shadow DOM's event model, events that are fired inside shadow DOM are adjusted to look like they come from the hosting element. For example, let's say you click an <input> inside a shadow root:

<x-focus>
    #shadow-root
    <input type="text" placeholder="Input inside shadow dom">

The focus event will look like it came from <x-focus>, not the <input>. Similarly, document.activeElement will be <x-focus>. If the shadow root was created with mode:'open' (see closed mode), you'll also be able access the internal node that gained focus:

document.activeElement.shadowRoot.activeElement // only works with open mode.

If there are multiple levels of shadow DOM at play (say a custom element within another custom element), you need to recursively drill into the shadow roots to find the activeElement:

function deepActiveElement() {
    let a = document.activeElement;
    while (a && a.shadowRoot && a.shadowRoot.activeElement) {
    a = a.shadowRoot.activeElement;
    }
    return a;
}

Another option for focus is the delegatesFocus: true option, which expands the focus behavior of element's within a shadow tree:

If you click a node inside shadow DOM and the node is not a focusable area, the first focusable area becomes focused.
When a node inside shadow DOM gains focus, :focus applies to the host in addition to the focused element.

Example - how delegatesFocus: true changes focus behavior

<style>
    :focus {
    outline: 2px solid red;
    }
</style>

<x-focus></x-focus>

<script>
customElements.define('x-focus', class extends HTMLElement {
    constructor() {
    super(); // always call super() first in the constructor.

    const root = this.attachShadow({mode: 'open', delegatesFocus: true});
    root.innerHTML = `
        <style>
        :host {
            display: flex;
            border: 1px dotted black;
            padding: 16px;
        }
        :focus {
            outline: 2px solid blue;
        }
        </style>
        <div>Clickable Shadow DOM text</div>
        <input type="text" placeholder="Input inside shadow dom">`;

    // Know the focused element inside shadow DOM:
    this.addEventListener('focus', function(e) {
        console.log('Active element (inside shadow dom):',
                    this.shadowRoot.activeElement);
    });
    }
});
</script>

Result

Above is the result when <x-focus> is focused (user click, tabbed into, focus(), etc.), "Clickable Shadow DOM text" is clicked, or the internal <input> is focused (including autofocus).

If you were to set delegatesFocus: false, here's what you would see instead:

delegatesFocus: false and the internal <input> is focused.

delegatesFocus: false and <x-focus> gains focus (e.g. it has tabindex="0").

delegatesFocus: false and "Clickable Shadow DOM text" is clicked (or other empty area within the element's shadow DOM is clicked).

Tips & Tricks #

Over the years I've learned a thing or two about authoring web components. I think you'll find some of these tips useful for authoring components and debugging shadow DOM.

Use CSS containment #

Typically, a web component's layout/style/paint is fairly self-contained. Use CSS containment in :host for a perf win:

<style>
:host {
    display: block;
    contain: content; /* Boom. CSS containment FTW. */
}
</style>

Resetting inheritable styles #

Inheritable styles (background, color, font, line-height, etc.) continue to inherit in shadow DOM. That is, they pierce the shadow DOM boundary by default. If you want to start with a fresh slate, use all: initial; to reset inheritable styles to their initial value when they cross the shadow boundary.

<style>
    div {
    padding: 10px;
    background: red;
    font-size: 25px;
    text-transform: uppercase;
    color: white;
    }
</style>

<div>
    <p>I'm outside the element (big/white)</p>
    <my-element>Light DOM content is also affected.</my-element>
    <p>I'm outside the element (big/white)</p>
</div>

<script>
const el = document.querySelector('my-element');
el.attachShadow({mode: 'open'}).innerHTML = `
    <style>
    :host {
        all: initial; /* 1st rule so subsequent properties are reset. */
        display: block;
        background: white;
    }
    </style>
    <p>my-element: all CSS properties are reset to their
        initial value using <code>all: initial</code>.</p>
    <slot></slot>
`;
</script>

Finding all the custom elements used by a page #

Sometimes it's useful to find custom elements used on the page. To do so, you need to recursively traverse the shadow DOM of all elements used on the page.

const allCustomElements = [];

function isCustomElement(el) {
    const isAttr = el.getAttribute('is');
    // Check for <super-button> and <button is="super-button">.
    return el.localName.includes('-') || isAttr && isAttr.includes('-');
}

function findAllCustomElements(nodes) {
    for (let i = 0, el; el = nodes[i]; ++i) {
    if (isCustomElement(el)) {
        allCustomElements.push(el);
    }
    // If the element has shadow DOM, dig deeper.
    if (el.shadowRoot) {
        findAllCustomElements(el.shadowRoot.querySelectorAll('*'));
    }
    }
}

findAllCustomElements(document.querySelectorAll('*'));

Creating elements from a <template> #

Instead of populating a shadow root using .innerHTML, we can use a declarative <template>. Templates are an ideal placeholder for declaring the structure of a web component.

See the example in "Custom elements: building reusable web components".

History & browser support #

If you've been following web components for the last couple of years, you'll know that Chrome 35+/Opera have been shipping an older version of shadow DOM for some time. Blink will continue to support both versions in parallel for some time. The v0 spec provided a different method to create a shadow root (element.createShadowRoot instead of v1's element.attachShadow). Calling the older method continues to create a shadow root with v0 semantics, so existing v0 code won't break.

If you happen to be interested in the old v0 spec, check out the html5rocks articles: 1, 2, 3. There's also a great comparison of the differences between shadow DOM v0 and v1.

Browser support #

Shadow DOM v1 is shipped in Chrome 53 (status), Opera 40, Safari 10, and Firefox 63. Edge has started development.

To feature detect shadow DOM, check for the existence of attachShadow:

const supportsShadowDOMV1 = !!HTMLElement.prototype.attachShadow;

Polyfill #

Until browser support is widely available, the shadydom and shadycss polyfills give you v1 feature. Shady DOM mimics the DOM scoping of Shadow DOM and shadycss polyfills CSS custom properties and the style scoping the native API provides.

Install the polyfills:

bower install --save webcomponents/shadydom
bower install --save webcomponents/shadycss

Use the polyfills:

function loadScript(src) {
    return new Promise(function(resolve, reject) {
    const script = document.createElement('script');
    script.async = true;
    script.src = src;
    script.onload = resolve;
    script.onerror = reject;
    document.head.appendChild(script);
    });
}

// Lazy load the polyfill if necessary.
if (!supportsShadowDOMV1) {
    loadScript('/bower_components/shadydom/shadydom.min.js')
    .then(e => loadScript('/bower_components/shadycss/shadycss.min.js'))
    .then(e => {
        // Polyfills loaded.
    });
} else {
    // Native shadow dom v1 support. Go to go!
}

See the https://github.com/webcomponents/shadycss#usage for instructions on how to shim/scope your styles.

Conclusion #

For the first time ever, we have an API primitive that does proper CSS scoping, DOM scoping, and has true composition. Combined with other web component APIs like custom elements, shadow DOM provides a way to author truly encapsulated components without hacks or using older baggage like <iframe>s.

Don't get me wrong. Shadow DOM is certainly a complex beast! But it's a beast worth learning. Spend some time with it. Learn it and ask questions!

FAQ #

Can I use Shadow DOM v1 today?

With a polyfill, yes. See Browser support.

What security features does shadow DOM provide?

Shadow DOM is not a security feature. It's a lightweight tool for scoping CSS and hiding away DOM trees in component. If you want a true security boundary, use an <iframe>.

Does a web component have to use shadow DOM?

Nope! You don't have to create web components that use shadow DOM. However, authoring custom elements that use Shadow DOM means you can take advantage of features like CSS scoping, DOM encapsulation, and composition.

What's the difference between open and closed shadow roots?

See Closed shadow roots.

Building the Google I/O 2016 Progressive Web App

2016-06-10T00:00:00Z

Summary #

Learn how we built a single page app using web components, Polymer, and material design and launched it into production on Google.com.

Results #

More engagement than the native app (4:06 min mobile web vs. Android's 2:40 min).
450ms faster first paint for return users thanks to service worker caching
84% of visitors supported Service Worker
Add to homescreen saves were up +900% compared to 2015.
3.8% users went offline but continued to generate 11k page views!
50% of signed in users enabled notifications.
536k notifications were sent to users (12% brought them back).
99% of users' browsers supported the web components polyfills

Overview #

This year, I had the pleasure of working on the Google I/O 2016 progressive web app, affectionately named "IOWA". It's mobile first, works fully offline, and heavily inspired by material design.

IOWA is a single page application (SPA) that's built using web components, Polymer, and Firebase, and has an extensive backend written in App Engine (Go). It pre-caches content using a service worker, dynamically loads new pages, gracefully transitions between views, and reuses content after first load.

In this case study, I'll go through some of the more interesting architectural decisions that we made for the frontend. If you're interested in the source code, check it out on Github.

View on Github

Building an SPA using web components #

Every page as a component #

One of the core aspects about our frontend is that it's centered around web components. In fact, every page in our SPA is a web component:

    <io-home-page date="2016-05-18T17:00:00Z" app="[[app]]"></io-home-page>
    <io-schedule-page date="2016-05-18T17:00:00Z" app="{ % templatetag openvariable % }app}}"></io-schedule-page>
    <io-attend-page></io-attend-page>
    <io-extended-page></io-extended-page>
    <io-faq-page></io-faq-page>

Why did we do this? The first reason is that this code is readable. As a first time reader, it's completely obvious what every page in our app is. The second reason is that web components have some nice properties for building an SPA. Lots of common frustrations (state management, view activation, style scoping) go away thanks to the inherent features of the <template> element, Custom Elements, and Shadow DOM. These are developer tools being built into the browser. Why not take advantage of them?

By creating a Custom Element for each page, we got a lot for free:

Page lifecycle management.
Scoped CSS/HTML particular to the page.
All the CSS/HTML/JS specific to a page is bundled and loaded together as needed.
Views are reusable. Since pages are DOM nodes, simply adding or removing them changes the view.
Future maintainers can understand our app simply by groking the markup.
Server-rendered markup can be progressively enhanced as element definitions are registered and upgraded by the browser.
Custom Elements have an inheritance model. DRY code is good code.
…lots more stuff.

We took full advantage of these benefits in IOWA. Let's dive into some of the details.

Dynamically activating pages #

The <template> element is the browser's standard way to create reusable markup. <template> has two characteristics that SPAs can take advantage of. First, anything inside of the <template> is inert until an instance of the template is created. Second, the browser parses the markup but the contents aren't reachable from the main page. It's a true, reusable chunk of markup. For example:

<template id="t">
    <div>This markup is inert and not part of the main page's DOM.</div>
    <img src="profile.png"> <!-- not loaded by the browser -->
    <video id="vid" src="vid.mp4" autoplay></video> <!-- doesn't load/start -->
    <script>alert("Not run until the template is stamped");</script>
</template>

Polymer extends the <template>'s with a few type extension custom elements, namely <template is="dom-if"> and <template is="dom-repeat">. Both are custom elements that extend <template> with extra capabilities. And thanks to the declarative nature of web components, both do exactly what you'd expect. The first component stamps markup based on a conditional. The second repeats markup for every item in a list (data model).

How is IOWA using these type extension elements?

If you recall, every page in IOWA is a web component. However, it would be silly to declare every component on first load. That would mean creating an instance of every page when the app first loads. We didn't want to hurt our initial load performance, especially as some users will only navigate to 1 or 2 pages.

Our solution was to cheat. In IOWA, we wrap each page's element in a <template is="dom-if"> so that its contents don't load on first boot. Then we activate pages when the template's name attribute matches the URL. The <lazy-pages> web component handles all this logic for us. The markup looks something like this:

<!-- Lazy pages manages the template stamping. It watches for route changes
        and sets `template.if = true` on the appropriate template. -->
<lazy-pages>
    <template is="dom-if" name="home">
    <io-home-page date="2016-05-18T17:00:00Z"></io-home-page>
    </template>

    <template is="dom-if" name="schedule">
    <io-schedule-page date="2016-05-18T17:00:00Z"></io-schedule-page>
    </template>

    <template is="dom-if" name="attend">
    <io-attend-page></io-attend-page>
    </template>
</lazy-pages>

What I like about this is that every page is parsed and ready to go when the page loads but its CSS/HTML/JS only get executed on-demand (when its parent <template> is stamped). Dynamic + lazy views using web components FTW.

Future improvements

When the page first loads, we're loading all the HTML Imports for each page at once. An obvious improvement would be to lazy load the element definitions only when they're needed. Polymer also has a nice helper for async loading HTML Imports:

Polymer.Base.importHref('io-home-page.html', (e) => { ... });

IOWA doesn't do this because a) we got lazy and b) it's unclear just how much performance boost we would have seen. Our first paint was already ~1s.

Page lifecycle management #

The Custom Elements API defines "lifecycle callbacks" for managing the state of a component. When you implement these methods, you get free hooks into the life of a component:

createdCallback() {
    // automatically called when an instance of the element is created.
}

attachedCallback() {
    // automatically called when the element is attached to the DOM.
}

detachedCallback() {
    // automatically called when the element is removed from the DOM.
}

attributeChangedCallback() {
    // automatically called when an HTML attribute changes.
}

It was easy to leverage these callbacks in IOWA. Remember, every page is a self-contained DOM node. Navigating to a "new view" in our SPA is a matter of attaching one node to the DOM and removing another.

We used the attachedCallback to perform setup work (init state, attach event listeners). When users navigate to a different page, the detachedCallback does cleanup (remove listeners, reset shared state). We also expanded the native lifecycle callbacks with several of our own:

onPageTransitionDone() {
    // page transition animations are complete.
},

onSubpageTransitionDone() {
    // sub nav/tab page transitions are complete.
}

These were useful additions for delaying work and minimizing the jank between page transitions. More on this later.

DRYing up common functionality across pages #

Inheritance is a powerful feature of Custom Elements. It provides a standard inheritance model for the web.

Unfortunately, Polymer 1.0 has yet to implement element inheritance at the time of writing. In the meantime, Polymer's Behaviors feature was just as useful. Behaviors are just mixins.

Rather than creating the same API surface on all pages, it made sense to DRY-up the codebase by creating shared mixins. For example, PageBehavior defines common properties/methods that all pages in our app need:

PageBehavior.html

let PageBehavior = {

    // Common properties all pages need.
    properties: {
    name: { type: String }, // Slug name of the page.
    ...
    },

    attached() {
    // If the page defines a `onPageTransitionDone`, call it when the router
    // fires 'page-transition-done'.
    if (this.onPageTransitionDone) {
        this.listen(document.body, 'page-transition-done', 'onPageTransitionDone');
    }

    // Update page meta data when new page is navigated to.
    document.body.id = `page-${this.name}`;
    document.title = this.title || 'Google I/O 2016';

    // Scroll to top of new page.
    if (IOWA.Elements.Scroller) {
        IOWA.Elements.Scroller.scrollTop = 0;
    }

    this.setupSubnavEffects();
    },

    detached() {
    this.unlisten(document.body, 'page-transition-done', 'onPageTransitionDone');
    this.teardownSubnavEffects();
    }
};

IOWA.IOBehaviors = IOWA.IOBehaviors || {PageBehavior: PageBehavior};

As you can see, PageBehavior performs common tasks that run when a new page is visited. Things like updating the document.title, resetting the scroll position, and setting up event listeners for scroll and sub nav effects.

Individual pages use PageBehavior by loading it as a dependency and using behaviors. They're also free to override its base properties/methods if needed. As an example, here's what our homepage "subclass" overrides:

io-home-page.html

<link rel="import" href="../bower_components/polymer/polymer.html">
<link rel="import" href="PageBehavior.html">
<!-- rest of the import dependencies used by the page. -->

<dom-module id="io-home-page">
    <template>
    <!-- PAGE'S MARKUP -->
    </template>
    <script>
    Polymer({
        is: 'io-home-page',

        behaviors: [IOBehaviors.PageBehavior], // All pages have common functionality.

        // Pages define their own title and slug for the router.
        title: 'Schedule - Google I/O 2016',
        name: 'home',

        // The home page has custom setup work when it's added navigated to.
        // Note: PageBehavior's attached also gets called.
        attached() {
        if (this.app.isPhoneSize) {
            this.listen(IOWA.Elements.ScrollContainer, 'scroll', '_onPageScroll');
        }
        },

        // The home page does its own cleanup when a new page is navigated to.
        // Note: PageBehavior's detached also gets called.
        detached() {
        this.unlisten(IOWA.Elements.ScrollContainer, 'scroll', '_onPageScroll');
        },

        // The home page can define onPageTransitionDone to do extra work
        // when page transitions are done, and thus preventing janky animations.
        onPageTransitionDone() {
        ...
        }
    });
    </script>
</dom-module>

To share styles across different components in our app, we used Polymer's shared style modules. Style modules allow you to define a chunk of CSS once and reuse it in different places throughout an app. For us, "different places" meant different components.

In IOWA, we created shared-app-styles to share colors, typography, and layout classes across pages and other components we made.

shared-app-styles.html

<link rel="import" href="../bower_components/polymer/polymer.html">
<link rel="import" href="../bower_components/iron-flex-layout/iron-flex-layout.html">
<link rel="import" href="../bower_components/paper-styles/color.html">

<dom-module id="shared-app-styles">
    <template>
    <style>
        [layout] {
        @apply(--layout);
        }
        [layout][horizontal] {
        @apply(--layout-horizontal);
        }
        .scrollable {
        @apply(--layout-scroll);
        }
        .noscroll {
        overflow: hidden;
        }
        /* Style radio buttons and tabs the same throughout the app */
        paper-tabs {
        --paper-tabs-selection-bar-color: currentcolor;
        }
        paper-radio-button {
        --paper-radio-button-checked-color: var(--paper-cyan-600);
        --paper-radio-button-checked-ink-color: var(--paper-cyan-600);
        }
        ...
    </style>
    </template>
</dom-module>

io-home-page.html

<link rel="import" href="shared-app-styles.html">
<!-- Rest of import dependencies used by the page. -->

<dom-module id="io-home-page">
    <template>
    <style include="shared-app-styles">
        :host { display: block} /* Other element styles can go here. */
    </style>
    <!-- PAGE'S MARKUP -->
    </template>
    <script>Polymer({...});</script>
</dom-module>

Here, <style include="shared-app-styles"></style> is Polymer's syntax for saying "include the styles in the module named "shared-app-styles".

By now you know that every page in our app is a Custom Element. I've said it a million times. Ok, but if every page is self-contained web component, you might be asking yourself how we share state across the app.

IOWA uses a technique similar to dependency injection (Angular) or redux (React) for sharing state. We created a global app property and hung shared sub-properties off it. app gets passed around our application by injecting it into every component that needs its data. Using Polymer's data binding features makes this easy since we can do the wiring without writing any code:

<lazy-pages>
    <template is="dom-if" name="home">
    <io-home-page date="2016-05-18T17:00:00Z" app="[[app]]"></io-home-page>
    </template>

    <template is="dom-if" name="schedule">
    <io-schedule-page date="2016-05-18T17:00:00Z" app="{ % templatetag openvariable % }app}}"></io-schedule-page>
    </template>
    ...
</lazy-pages>

<google-signin client-id="..." scopes="profile email"
                            user="{ % templatetag openvariable % }app.currentUser}}"></google-signin>

<iron-media-query query="(min-width:320px) and (max-width:768px)"
                                query-matches="{ % templatetag openvariable % }app.isPhoneSize}}"></iron-media-query>

The <google-signin> element updates its user property when users log in to our app. Since that property is bound to app.currentUser, any page that wants to access the current user simply needs to bind to app and read the currentUser sub-property. By itself, this technique is useful for sharing state across the app. However, another benefit was that we ended up creating a single sign-in element and reusing its results across the site. Same for the media queries. It would have been wasteful for every page to duplicate sign-in or create its own set of media queries. Instead, components which are responsible for app-wide functionality/data exist at the app-level.

Note: I'm purposefully showing two-way bindings ({ % templatetag openvariable % }) and one-way bindings ([[]]) in this example. Use ({ % templatetag openvariable % }) bindings when changes need to flow into an element as well as propagate back out (when changes are made internally). Use ([[]]) bindings when an element just needs to read data.

Page transitions #

As you navigate the Google I/O web app, you'll notice its slick page transitions (à la material design).

IOWA's page transitions in action.

When users navigate to a new page, a sequence of things happen:

The top nav slides a selection bar to the new link.
The page's heading fades away.
The page's content slides down then fades away.
By reversing those animations, the new page's heading and content appear.
(Optional) The new page does additional initialization work.

One of our challenges was figuring out how to craft this slick transition without sacrificing performance. There's a lot of dynamic work that takes place, and jank wasn't welcome at our party. Our solution was a combination of the Web Animations API and Promises. Using the two together gave us versatility, a plug and play animation system, and granular control to minimize das jank.

How it works

When users click to a new page (or hit back/forward), our router's runPageTransition() does its magic by running through a series of Promises. Using Promises allowed us to carefully orchestrate the animations and helped rationalize the "async-ness" of CSS Animations and dynamically loading content.

class Router {

    init() {
    window.addEventListener('popstate', e => this.runPageTransition());
    }

    runPageTransition() {
    let endPage = this.state.end.page;

    this.fire('page-transition-start');              // 1. Let current page know it's starting.

    IOWA.PageAnimation.runExitAnimation()            // 2. Play exist animation sequence.
        .then(() => {
        IOWA.Elements.LazyPages.selected = endPage;  // 3. Activate new page in <lazy-pages>.
        this.state.current = this.parseUrl(this.state.end.href);
        })
        .then(() => IOWA.PageAnimation.runEnterAnimation())  // 4. Play entry animation sequence.
        .then(() => this.fire('page-transition-done')) // 5. Tell new page transitions are done.
        .catch(e => IOWA.Util.reportError(e));
    }

}

Recall from the section "Keeping things DRY: common functionality across pages", pages listen for the page-transition-start and page-transition-done DOM events. Now you're seeing where those events get fired.

We used the Web Animations API instead of the runEnterAnimation/runExitAnimation helpers. In the case of runExitAnimation, we grab a couple of DOM nodes (the masthead and main content area), declare the start/end of each animation, and create a GroupEffect to run the two in parallel:

function runExitAnimation(section) {
    let main = section.querySelector('.slide-up');
    let masthead = section.querySelector('.masthead');

    let start = {transform: 'translate(0,0)', opacity: 1};
    let end = {transform: 'translate(0,-100px)', opacity: 0};
    let opts = {duration: 400, easing: 'cubic-bezier(.4, 0, .2, 1)'};
    let opts_delay = {duration: 400, delay: 200};

    return new GroupEffect([
    new KeyframeEffect(masthead, [start, end], opts),
    new KeyframeEffect(main, [{opacity: 1}, {opacity: 0}], opts_delay)
    ]);
}

Just modify the array to make view transitions more (or less) elaborate!

Scroll effects #

IOWA has a few interesting effects when you scroll the page. The first is our floating action button (FAB) that takes users back to the top of the page:

    <a href="#" tabindex="-1" aria-hidden="true" aria-label="back to top" onclick="backToTop">
      <paper-fab icon="io:expand-less" noink tabindex="-1"></paper-fab>
    </a>

The smooth scroll is implemented using Polymer's app-layout elements. They provide out-of-the-box scrolling effects like sticky/returning top navs, drop shadows, color and background transitions, parallax effects, and smooth scrolling.

    // Smooth scrolling the back to top FAB.
    function backToTop(e) {
      e.preventDefault();

      Polymer.AppLayout.scroll({top: 0, behavior: 'smooth',
                                target: document.documentElement});

      e.target.blur();  // Kick focus back to the page so user starts from the top of the doc.
    }

Another place we used the <app-layout> elements was for the sticky nav. As you can see in the video, it goes away when users scroll down the page and returns when scrolling back up.

Sticky scroll navs using .

We used the <app-header> element pretty much as-is. It was easy to drop in and gain fancy scrolling effects in the app. Sure, we could have implemented them ourselves, but having the details already codified in a reusable component was a huge time saver.

Declare the element. Customize it with attributes. You're done!

    <app-header reveals condenses effects="fade-background waterfall"></app-header>

Conclusion #

For the I/O progressive web app, we were able to build an entire frontend in several weeks thanks to web components and Polymer's pre-made material design widgets. The features of the native APIs (Custom Elements, Shadow DOM, <template>) lend themselves naturally to the dynamism of an SPA. Reusability saves a ton of time.

If you're interested in creating a progressive web app of your own, check out the App Toolbox. Polymer's App Toolbox is a collection of components, tools and templates for building PWAs with Polymer. It's an easy way to get up and running.

HTML Imports

2013-11-11T00:00:00Z

Why imports? #

Think about how you load different types of resources on the web. For JS, we have <script src>. For CSS, your go-to is probably <link rel="stylesheet">. For images it's <img>. Video has <video>. Audio, <audio>… Get to the point! The majority of the web's content has a simple and declarative way to load itself. Not so for HTML. Here's your options:

<iframe> - tried and true but heavy weight. An iframe's content lives entirely in a separate context than your page. While that's mostly a great feature, it creates additional challenges (shrink wrapping the size of the frame to its content is tough, extremely frustrating to script into/out of, nearly impossible to style).

AJAX - I love xhr.responseType="document", but you're saying I need JS to load HTML? That doesn't seem right.
CrazyHacks™ - embedded in strings, hidden as comments (e.g. <script type="text/html">). Yuck!

See the irony? The web's most basic content, HTML, requires the greatest amount of effort to work with. Fortunately, Web Components are here to get us back on track.

Getting started #

HTML Imports, part of the Web Components cast, is a way to include HTML documents in other HTML documents. You're not limited to markup either. An import can also include CSS, JavaScript, or anything else an .html file can contain. In other words, this makes imports a fantastic tool for loading related HTML/CSS/JS.

The basics #

Include an import on your page by declaring a <link rel="import">:

<head>
    <link rel="import" href="/path/to/imports/stuff.html">
</head>

The URL of an import is called an import location. To load content from another domain, the import location needs to be CORS-enabled:

<!-- Resources on other origins must be CORS-enabled. -->
<link rel="import" href="http://example.com/elements.html">

Feature detection and support #

To detect support, check if .import exists on the <link> element:

function supportsImports() {
    return 'import' in document.createElement('link');
}

if (supportsImports()) {
    // Good to go!
} else {
    // Use other libraries/require systems to load files.
}

Browser support is still in the early days. Chrome 31 was the first browser to see an implementation but other browser vendors are waiting to see how ES Modules play out. However, for other browsers the webcomponents.js polyfill works great until things are widely supported.

Bundling resources #

Imports provide convention for bundling HTML/CSS/JS (even other HTML Imports) into a single deliverable. It's an intrinsic feature, but a powerful one. If you're creating a theme, library, or just want to segment your app into logical chunks, giving users a single URL is compelling. Heck, you could even deliver an entire app via an import. Think about that for a second.

A real-world example is Bootstrap. Bootstrap is comprised of individual files (bootstrap.css, bootstrap.js, fonts), requires JQuery for its plugins, and provides markup examples. Developers like à la carte flexibility. It allows them buy in to the parts of the framework they want to use. That said, I'd wager your typical JoeDeveloper™ goes the easy route and downloads all of Bootstrap.

Imports make a ton of sense for something like Bootstrap. I present to you, the future of loading Bootstrap:

<head>
    <link rel="import" href="bootstrap.html">
</head>

Users simply load an HTML Import link. They don't need to fuss with the scatter-shot of files. Instead, the entirety of Bootstrap is managed and wrapped up in an import, bootstrap.html:

<link rel="stylesheet" href="bootstrap.css">
<link rel="stylesheet" href="fonts.css">
<script src="jquery.js"></script>
<script src="bootstrap.js"></script>
<script src="bootstrap-tooltip.js"></script>
<script src="bootstrap-dropdown.js"></script>
...

<!-- scaffolding markup -->
<template>
    ...
</template>

Let this sit. It's exciting stuff.

Load/error events #

The <link> element fires a load event when an import is loaded successfully and onerror when the attempt fails (e.g. if the resource 404s).

Imports try to load immediately. An easy way avoid headaches is to use the onload/onerror attributes:

<script>
    function handleLoad(e) {
    console.log('Loaded import: ' + e.target.href);
    }
    function handleError(e) {
    console.log('Error loading import: ' + e.target.href);
    }
</script>

<link rel="import" href="file.html"
        onload="handleLoad(event)" onerror="handleError(event)">

Or, if you're creating the import dynamically:

var link = document.createElement('link');
link.rel = 'import';
// link.setAttribute('async', ''); // make it async!
link.href = 'file.html';
link.onload = function(e) {...};
link.onerror = function(e) {...};
document.head.appendChild(link);

Using the content #

Including an import on a page doesn't mean "plop the content of that file here". It means "parser, go off an fetch this document so I can use it". To actually use the content, you have to take action and write script.

A critical aha! moment is realizing that an import is just a document. In fact, the content of an import is called an import document. You're able to manipulate the guts of an import using standard DOM APIs!

link.import #

To access the content of an import, use the link element's .import property:

var content = document.querySelector('link[rel="import"]').import;

link.import is null under the following conditions:

The browser doesn't support HTML Imports.
The <link> doesn't have rel="import".
The <link> has not been added to the DOM.
The <link> has been removed from the DOM.
The resource is not CORS-enabled.

Full example

Let's say warnings.html contains:

<div class="warning">
    <style>
    h3 {
        color: red !important;
    }
    </style>
    <h3>Warning!
    <p>This page is under construction
</div>

<div class="outdated">
    <h3>Heads up!
    <p>This content may be out of date
</div>

Importers can grab a specific portion of this document and clone it into their page:

<head>
    <link rel="import" href="warnings.html">
</head>
<body>
    ...
    <script>
    var link = document.querySelector('link[rel="import"]');
    var content = link.import;

    // Grab DOM from warning.html's document.
    var el = content.querySelector('.warning');

    document.body.appendChild(el.cloneNode(true));
    </script>
</body>

Scripting in imports #

Imports are not in the main document. They're satellite to it. However, your import can still act on the main page even though the main document reigns supreme. An import can access its own DOM and/or the DOM of the page that's importing it:

Example - import.html that adds one of its stylesheets to the main page

<link rel="stylesheet" href="http://www.example.com/styles.css">
<link rel="stylesheet" href="http://www.example.com/styles2.css">

<style>
/* Note: <style> in an import apply to the main
    document by default. That is, style tags don't need to be
    explicitly added to the main document. */
#somecontainer {
color: blue;
}
</style>
...

<script>
// importDoc references this import's document
var importDoc = document.currentScript.ownerDocument;

// mainDoc references the main document (the page that's importing us)
var mainDoc = document;

// Grab the first stylesheet from this import, clone it,
// and append it to the importing document.
    var styles = importDoc.querySelector('link[rel="stylesheet"]');
    mainDoc.head.appendChild(styles.cloneNode(true));
</script>

Notice what's going on here. The script inside the import references the imported document (document.currentScript.ownerDocument), and appends part of that document to the importing page (mainDoc.head.appendChild(...)). Pretty gnarly if you ask me.

Rules of JavaScript in an import:

Script in the import is executed in the context of the window that contains the importing document. So window.document refers to the main page document. This has two useful corollaries:
- functions defined in an import end up on window.
- you don't have to do anything difficult like append the import's <script> blocks to the main page. Again, script gets executed.
Imports do not block parsing of the main page. However, scripts inside them are processed in order. This means you get defer-like behavior while maintaining proper script order. More on this below.

Delivering Web Components #

The design of HTML Imports lends itself nicely to loading reusable content on the web. In particular, it's an ideal way to distribute Web Components. Everything from basic HTML <template>s to full blown Custom Elements with Shadow DOM [1, 2, 3]. When these technologies are used in tandem, imports become a #include for Web Components.

Including templates #

The HTML Template element is a natural fit for HTML Imports. <template> is great for scaffolding out sections of markup for the importing app to use as it desires. Wrapping content in a <template> also gives you the added benefit of making the content inert until used. That is, scripts don't run until the template is added to the DOM). Boom!

import.html

<template>
    <h1>Hello World!</h1>
    <!-- Img is not requested until the <template> goes live. -->
    <img src="world.png">
    <script>alert("Executed when the template is activated.");</script>
</template>

index.html

<head>
    <link rel="import" href="import.html">
</head>
<body>
    <div id="container"></div>
    <script>
    var link = document.querySelector('link[rel="import"]');

    // Clone the <template> in the import.
    var template = link.import.querySelector('template');
    var clone = document.importNode(template.content, true);

    document.querySelector('#container').appendChild(clone);
    </script>
</body>

Registering custom elements #

Custom Elements is another Web Component technology that plays absurdly well with HTML Imports. Imports can execute script, so why not define + register your custom elements so users don't have to? Call it…"auto-registration".

elements.html

<script>
    // Define and register <say-hi>.
    var proto = Object.create(HTMLElement.prototype);

    proto.createdCallback = function() {
    this.innerHTML = 'Hello, <b>' +
                        (this.getAttribute('name') || '?') + '</b>';
    };

    document.registerElement('say-hi', {prototype: proto});
</script>

<template id="t">
    <style>
    ::content > * {
        color: red;
    }
    </style>
    <span>I'm a shadow-element using Shadow DOM!</span>
    <content></content>
</template>

<script>
    (function() {
    var importDoc = document.currentScript.ownerDocument; // importee

    // Define and register <shadow-element>
    // that uses Shadow DOM and a template.
    var proto2 = Object.create(HTMLElement.prototype);

    proto2.createdCallback = function() {
        // get template in import
        var template = importDoc.querySelector('#t');

        // import template into
        var clone = document.importNode(template.content, true);

        var root = this.createShadowRoot();
        root.appendChild(clone);
    };

    document.registerElement('shadow-element', {prototype: proto2});
    })();
</script>

This import defines (and registers) two elements, <say-hi> and <shadow-element>. The first shows a basic custom element that registers itself inside the import. The second example shows how to implement a custom element that creates Shadow DOM from a <template>, then registers itself.

The best part about registering custom elements inside an HTML import is that the importer simply declares your element on their page. No wiring needed.

index.html

<head>
    <link rel="import" href="elements.html">
</head>
<body>
    <say-hi name="Eric"></say-hi>
    <shadow-element>
    <div>( I'm in the light dom )</div>
    </shadow-element>
</body>

In my opinion, this workflow alone makes HTML Imports an ideal way to share Web Components.

Managing dependencies and sub-imports #

Sub-imports #

It can be useful for one import to include another. For example, if you want to reuse or extend another component, use an import to load the other element(s).

Below is a real example from Polymer. It's a new tab component (<paper-tabs>) that reuses a layout and selector component. The dependencies are managed using HTML Imports.

paper-tabs.html (simplified):

<link rel="import" href="iron-selector.html">
<link rel="import" href="classes/iron-flex-layout.html">

<dom-module id="paper-tabs">
    <template>
    <style>...</style>
    <iron-selector class="layout horizonta center">
        <content select="*"></content>
    </iron-selector>
    </template>
    <script>...</script>
</dom-module>

App developers can import this new element using:

<link rel="import" href="paper-tabs.html">
<paper-tabs></paper-tabs>

When a new, more awesome <iron-selector2> comes along in the future, you can swap out <iron-selector> and start using it straight away. You won't break your users thanks to imports and web components.

Dependency management #

We all know that loading JQuery more than once per page causes errors. Isn't this going to be a huge problem for Web Components when multiple components use the same library? Not if we use HTML Imports! They can be used to manage dependencies.

By wrapping libraries in an HTML Import, you automatically de-dupe resources. The document is only parsed once. Scripts are only executed once. As an example, say you define an import, jquery.html, that loads a copy of JQuery.

jquery.html

<script src="http://cdn.com/jquery.js"></script>

This import can be reused in subsequent imports like so:

import2.html

<link rel="import" href="jquery.html">
<div>Hello, I'm import 2</div>

ajax-element.html

<link rel="import" href="jquery.html">
<link rel="import" href="import2.html">

<script>
    var proto = Object.create(HTMLElement.prototype);

    proto.makeRequest = function(url, done) {
    return $.ajax(url).done(function() {
        done();
    });
    };

    document.registerElement('ajax-element', {prototype: proto});
</script>

Even the main page itself can include jquery.html if it needs the library:

<head>
    <link rel="import" href="jquery.html">
    <link rel="import" href="ajax-element.html">
</head>
<body>

...

<script>
    $(document).ready(function() {
    var el = document.createElement('ajax-element');
    el.makeRequest('http://example.com');
    });
</script>
</body>

Despite jquery.html being included in many different import trees, it's document is only fetched and processed once by the browser. Examining the network panel proves this:

jquery.html is requested once

Performance considerations #

HTML Imports are totally awesome but as with any new web technology, you should use them wisely. Web development best practices still hold true. Below are some things to keep in mind.

Concatenate imports #

Reducing network requests is always important. If you have many top-level import links, consider combining them into a single resource and importing that file!

Vulcanize is an npm build tool from the Polymer team that recursively flattens a set of HTML Imports into a single file. Think of it as a concatenation build step for Web Components.

Imports leverage browser caching #

Many people forget that the browser's networking stack has been finely tuned over the years. Imports (and sub-imports) take advantage of this logic too. The http://cdn.com/bootstrap.html import might have sub-resources, but they'll be cached.

Content is useful only when you add it #

Think of content as inert until you call upon its services. Take a normal, dynamically created stylesheet:

var link = document.createElement('link');
link.rel = 'stylesheet';
link.href = 'styles.css';

The browser won't request styles.css until link is added to the DOM:

document.head.appendChild(link); // browser requests styles.css

Another example is dynamically created markup:

var h2 = document.createElement('h2');
h2.textContent = 'Booyah!';

The h2 is relatively meaningless until you add it to the DOM.

The same concept holds true for the import document. Unless you append it's content to the DOM, it's a no-op. In fact, the only thing that "executes" in the import document directly is <script>. See scripting in imports.

Optimizing for async loading #

Imports block rendering #

Imports block rendering of the main page. This is similar to what <link rel="stylesheet"> do. The reason the browser blocks rendering on stylesheets in the first place is to minimize FOUC. Imports behave similarly because they can contain stylsheets.

To be completely asynchronous and not block the parser or rendering, use the async attribute:

<link rel="import" href="/path/to/import_that_takes_5secs.html" async>

The reason async isn't the default for HTML Imports is because it requires developers to do more work. Synchronous by default means that HTML Imports that have custom element definitions inside of them are guaranteed to load and upgrade, in order. In a completely async world, developers would have to manage that dance and upgrade timings themselves.

You can also create an async import, dynamically:

var l = document.createElement('link');
l.rel = 'import';
l.href = 'elements.html';
l.setAttribute('async', '');
l.onload = function(e) { ... };

Imports do not block parsing #

Imports don't block parsing of the main page. Scripts inside imports are processed in order but don't block the importing page. This means you get defer-like behavior while maintaining proper script order. One benefit of putting your imports in the <head> is that it lets the parser start working on the content as soon as possible. With that said, it's critical to remember <script> in the main document still continues to block the page. The first <script> after an import will block page rendering. That's because an import can have script inside that needs to be executed before the script in the main page.

<head>
    <link rel="import" href="/path/to/import_that_takes_5secs.html">
    <script>console.log('I block page rendering');</script>
</head>

Depending on your app structure and use case, there are several ways to optimize async behavior. The techniques below mitigate blocking the main page rendering.

Scenario #1 (preferred): you don't have script in <head> or inlined in <body>

My recommendation for placing <script> is to avoid immediately following your imports. Move scripts as late in the game as possible…but you're already doing that best practice, AREN'T YOU!? ;)

Here's an example:

<head>
    <link rel="import" href="/path/to/import.html">
    <link rel="import" href="/path/to/import2.html">
    <!-- avoid including script -->
</head>
<body>
    <!-- avoid including script -->

    <div id="container"></div>

    <!-- avoid including script -->
    ...

    <script>
    // Other scripts n' stuff.

    // Bring in the import content.
    var link = document.querySelector('link[rel="import"]');
    var post = link.import.querySelector('#blog-post');

    var container = document.querySelector('#container');
    container.appendChild(post.cloneNode(true));
    </script>
</body>

Everything is at the bottom.

Scenario 1.5: the import adds itself

Another option is to have the import add its own content. If the import author establishes a contract for the app developer to follow, the import can add itself to an area of the main page:

import.html:

<div id="blog-post">...</div>
<script>
    var me = document.currentScript.ownerDocument;
    var post = me.querySelector('#blog-post');

    var container = document.querySelector('#container');
    container.appendChild(post.cloneNode(true));
</script>

index.html

<head>
    <link rel="import" href="/path/to/import.html">
</head>
<body>
    <!-- no need for script. the import takes care of things -->
</body>

Scenario #2: you have script in <head> or inlined in <body>

If you have an import that takes a long time to load, the first <script> that follows it on the page will block the page from rendering. Google Analytics for example, recommends putting the tracking code in the <head>, If you can't avoid putting <script> in the <head>, dynamically adding the import will prevent blocking the page:

<head>
    <script>
    function addImportLink(url) {
        var link = document.createElement('link');
        link.rel = 'import';
        link.href = url;
        link.onload = function(e) {
        var post = this.import.querySelector('#blog-post');

        var container = document.querySelector('#container');
        container.appendChild(post.cloneNode(true));
        };
        document.head.appendChild(link);
    }

    addImportLink('/path/to/import.html'); // Import is added early :)
    </script>
    <script>
    // other scripts
    </script>
</head>
<body>
    <div id="container"></div>
    ...
</body>

Alternatively, add the import near the end of the <body>:

<head>
    <script>
    // other scripts
    </script>
</head>
<body>
    <div id="container"></div>
    ...

    <script>
    function addImportLink(url) { ... }

    addImportLink('/path/to/import.html'); // Import is added very late :(
    </script>
</body>

Things to remember #

An import's mimetype is text/html.
Resources from other origins need to be CORS-enabled.
Imports from the same URL are retrieved and parsed once. That means script in an import is only executed the first time the import is seen.
Scripts in an import are processed in order, but do not block the main document parsing.
An import link doesn't mean "#include the content here". It means "parser, go off an fetch this document so I can use it later". While scripts execute at import time, stylesheets, markup, and other resources need to be added to the main page explicitly. Note, <style> don't need to be added explicitly. This is a major difference between HTML Imports and <iframe>, which says "load and render this content here".

Conclusion #

HTML Imports allow bundling HTML/CSS/JS as a single resource. While useful by themselves, this idea becomes extremely powerful in the world of Web Components. Developers can create reusable components for others to consume and bring in to their own app, all delivered through <link rel="import">.

HTML Imports are a simple concept, but enable a number of interesting use cases for the platform.

Use cases #

Distribute related HTML/CSS/JS as a single bundle. Theoretically, you could import an entire web app into another.
Code organization - segment concepts logically into different files, encouraging modularity and reusability**.
Deliver one or more Custom Element definitions. An import can be used to register and include them in an app. This practices good software patterns, keeping the element's interface/definition separate from how its used.
Manage dependencies - resources are automatically de-duped.
Chunk scripts - before imports, a large-sized JS library would have its file wholly parsed in order to start running, which was slow. With imports, the library can start working as soon as chunk A is parsed. Less latency!
```
`<link rel="import" href="chunks.html">`:

<script>/* script chunk A goes here */</script>
<script>/* script chunk B goes here */</script>
<script>/* script chunk C goes here */</script>
...
```
Parallelizes HTML parsing - first time the browser has been able to run two (or more) HTML parsers in parallel.
Enables switching between debug and non-debug modes in an app, just by changing the import target itself. Your app doesn't need to know if the import target is a bundled/compiled resource or an import tree.

What's the CSS :scope pseudo-class for?

2013-03-21T00:00:00Z

:scope is defined in CSS Selectors 4 as:

A pseudo-class which represents any element that is in the contextual reference element set. This is is a (potentially empty) explicitly-specified set of elements, such as that specified by the querySelector(), or the parent element of a <style scoped> element, which is used to "scope" a selector so that it only matches within a subtree.

An example of using this is within a <style scoped> (more info):

<style>
    li {
    color: blue;
    }
</style>

<ul>
    <style scoped>
    li {
        color: red;
    }
    :scope {
        border: 1px solid red;
    }
    </style>
    <li>abc</li>
    <li>def</li>
    <li>efg</li>
</ul>

<ul>
    <li>hij</li>
    <li>klm</li>
    <li>nop</li>
</ul>

This colors the li elements in the first ul red and, because of the :scope rule, puts a border around the ul. That's because in the context of this <style scoped>, the ul matches :scope. It's the local context. If we were to add a :scope rule in the outer <style> it would match the entire document. Essentially, equivalent to :root.

Contextual elements #

You're probably aware of the Element version of querySelector() and querySelectorAll(). Instead of querying the entire document, you can restrict the result set to a contextual element:

<ul>
    <li id="scope"><a>abc</a></li>
    <li>def</li>
    <li><a>efg</a></li>
</ul>
<script>
    document.querySelectorAll('ul a').length; // 2

    var scope = document.querySelector('#scope');
    scope.querySelectorAll('a').length; // 1
</script>

When these are called, the browser returns a NodeList that's filtered to only include the set of nodes that a.) match the selector and b.) which are also descendants of the context element. So in the second example, the browser finds all a elements, then filters out the ones not in the scope element. This works, but it can lead to some bizarre behavior if you're not careful. Read on.

When querySelector goes wrong #

There's a really important point in the Selectors spec that people often overlook. Even when querySelector[All]() is invoked on an element, selectors still evaluate in the context of the entire document. This means unanticipated things can happen:

    scope.querySelectorAll('ul a').length); // 1
    scope.querySelectorAll('body ul a').length); // 1

WTF! In the first example, ul is my element, yet I'm still able to use it and matches nodes. In the second, body isn't even a descendant of my element, but "body ul a" still matches. Both of these are confusing and not what you'd expect.

It's worth making the comparison to jQuery here, which takes the right approach and does what you'd expect:

    $(scope).find('ul a').length // 0
    $(scope).find('body ul a').length // 0

…enter :scope to solve these semantic shenanigans.

Fixing querySelector with :scope #

WebKit recently landed support for using the :scope pseudo-class in querySelector[All](). You can test it in Chrome Canary 27.

You can use it restrict selectors to a context element. Let's see an example. In the following, :scope is used to "scope" the selector to the scope element's subtree. That's right, I said scope three times!

    scope.querySelectorAll(':scope ul a').length); // 0
    scope.querySelectorAll(':scope body ul a').length); // 0
    scope.querySelectorAll(':scope a').length); // 1

Using :scope makes the semantics of the querySelector() methods a little more predictable and inline with what others like jQuery are already doing.

Performance win? #

Not yet :(

I was curious if using :scope in qS/qSA gives a performance boost. So… like a good engineer I threw together a test. My rationale: less surface area for the browser to do selector matching means speedier lookups.

In my experiment, WebKit currently takes ~1.5-2x longer than not using :scope. Drats! When crbug.com/222028 gets fixed, using it should theoretically give you a slight performance boost over not using it.

Shadow DOM 301

2013-03-21T00:00:00Z

This article discusses more of the amazing things you can do with Shadow DOM! It builds on the concepts discussed in Shadow DOM 101 and Shadow DOM 201.

Using multiple shadow roots #

If you're hosting a party, it gets stuffy if everyone is crammed into the same room. You want the option of distributing groups of people across multiple rooms. Elements hosting Shadow DOM can do this too, that is to say, they can host more than one shadow root at a time.

Let's see what happens if we try to attach multiple shadow roots to a host:

<div id="example1">Light DOM</div>
<script>
  var container = document.querySelector('#example1');
  var root1 = container.createShadowRoot();
  var root2 = container.createShadowRoot();
  root1.innerHTML = '<div>Root 1 FTW</div>';
  root2.innerHTML = '<div>Root 2 FTW</div>';
</script>

What renders is "Root 2 FTW", despite the fact that we had already attached a shadow tree. This is because the last shadow tree added to a host, wins. It's a LIFO stack as far as rendering is concerned. Examining the DevTools verifies this behavior.

So what's the point of using multiple shadows if only the last is invited to the rendering party? Enter shadow insertion points.

Shadow Insertion Points #

"Shadow insertion points" (<shadow>) are similar to normal insertion points (<content>) in that they're placeholders. However, instead of being placeholders for a host's content, they're hosts for other shadow trees. It's Shadow DOM Inception!

As you can probably imagine, things get more complicated the further you drill down the rabbit hole. For this reason, the spec is very clear about what happens when multiple <shadow> elements are in play:

Looking back to our original example, the first shadow root1 got left off the invite list. Adding a <shadow> insertion point brings it back:

<div id="example2">Light DOM</div>
<script>
var container = document.querySelector('#example2');
var root1 = container.createShadowRoot();
var root2 = container.createShadowRoot();
root1.innerHTML = '<div>Root 1 FTW</div><content></content>';
**root2.innerHTML = '<div>Root 2 FTW</div><shadow></shadow>';**
</script>

There are a couple of interesting things about this example:

"Root 2 FTW" still renders above "Root 1 FTW". This is because of where we've placed the <shadow> insertion point. If you want the reverse, move the insertion point: root2.innerHTML = '<shadow></shadow><div>Root 2 FTW</div>';.

Notice there's now a <content> insertion point in root1. This makes the text node "Light DOM" come along for the rendering ride.

What's rendered at <shadow>?

Sometimes it's useful to know the older shadow tree being rendered at a <shadow>. You can get a reference to that tree through .olderShadowRoot:

**root2.olderShadowRoot** === root1 //true

Obtaining a host's shadow root #

If an element is hosting Shadow DOM you can access its youngest shadow root using .shadowRoot:

var root = host.createShadowRoot();
console.log(host.shadowRoot === root); // true
console.log(document.body.shadowRoot); // null

If you're worried about people crossing into your shadows, redefine .shadowRoot to be null:

Object.defineProperty(host, 'shadowRoot', {
  get: function() { return null; },
  set: function(value) { }
});

A bit of a hack, but it works. In the end, it's important to remember that while amazingly fantastic, Shadow DOM has not been designed to be a security feature. Don't rely on it for complete content isolation.

Building Shadow DOM in JS #

If you prefer building DOM in JS, HTMLContentElement and HTMLShadowElement have interfaces for that.

<div id="example3">
  <span>Light DOM</span>
</div>
<script>
var container = document.querySelector('#example3');
var root1 = container.createShadowRoot();
var root2 = container.createShadowRoot();

var div = document.createElement('div');
div.textContent = 'Root 1 FTW';
root1.appendChild(div);

 // HTMLContentElement
var content = document.createElement('content');
content.select = 'span'; // selects any spans the host node contains
root1.appendChild(content);

var div = document.createElement('div');
div.textContent = 'Root 2 FTW';
root2.appendChild(div);

// HTMLShadowElement
var shadow = document.createElement('shadow');
root2.appendChild(shadow);
</script>

This example is nearly identical to the one in the previous section. The only difference is that now I'm using select to pull out the newly added <span>.

Working with insertion points #

Nodes that are selected out of the host element and "distribute" into the shadow tree are called…drumroll…distributed nodes! They're allowed to cross the shadow boundary when insertion points invite them in.

What's conceptually bizarre about insertion points is that they don't physically move DOM. The host's nodes stay intact. Insertion points merely re-project nodes from the host into the shadow tree. It's a presentation/rendering thing: ~~"Move these nodes over here"~~ "Render these nodes at this location."

For example:

<div><h2>Light DOM</h2></div>
<script>
var root = document.querySelector('div').createShadowRoot();
root.innerHTML = '<content select="h2"></content>';

var h2 = document.querySelector('h2');
console.log(root.querySelector('content[select="h2"] h2')); // null;
console.log(root.querySelector('content').contains(h2)); // false
</script>

Voilà! The h2 isn't a child of the shadow DOM. This leads to another tid bit:

Element.getDistributedNodes() #

We can't traverse into a <content>, but the .getDistributedNodes() API allows us to query the distributed nodes at an insertion point:

<div id="example4">
  <h2>Eric</h2>
  <h2>Bidelman</h2>
  <div>Digital Jedi</div>
  <h4>footer text</h4>
</div>

<template id="sdom">
  <header>
    <content select="h2"></content>
  </header>
  <section>
    <content select="div"></content>
  </section>
  <footer>
    <content select="h4:first-of-type"></content>
  </footer>
</template>

<script>
var container = document.querySelector('#example4');

var root = container.createShadowRoot();

var t = document.querySelector('#sdom');
var clone = document.importNode(t.content, true);
root.appendChild(clone);

var html = [];
[].forEach.call(root.querySelectorAll('content'), function(el) {
  html.push(el.outerHTML + ': ');
  var nodes = el.getDistributedNodes();
  [].forEach.call(nodes, function(node) {
    html.push(node.outerHTML);
  });
  html.push('\n');
});
</script>

Element.getDestinationInsertionPoints() #

Similar to .getDistributedNodes(), you can check what insertion points a node is distributed into by calling its .getDestinationInsertionPoints():

<div id="host">
  <h2>Light DOM
</div>

<script>
  var container = document.querySelector('div');

  var root1 = container.createShadowRoot();
  var root2 = container.createShadowRoot();
  root1.innerHTML = '<content select="h2"></content>';
  root2.innerHTML = '<shadow></shadow>';

  var h2 = document.querySelector('#host h2');
  var insertionPoints = h2.getDestinationInsertionPoints();
  [].forEach.call(insertionPoints, function(contentEl) {
    console.log(contentEl);
  });
</script>

Tool: Shadow DOM Visualizer #

Understanding the black magic that is Shadow DOM is difficult. I remember trying to wrap my head around it for the first time.

To help visualize how Shadow DOM rendering works, I've built a tool using d3.js. Both markup boxes on the left-hand side are editable. Feel free to paste in your own markup and play around to see how things work and insertion points swizzle host nodes into the shadow tree.

Launch Shadow DOM Visualizer

Give it a try and let me know what you think!

Event Model #

Some events cross the shadow boundary and some do not. In the cases where events cross the boundary, the event target is adjusted in order to maintain the encapsulation that the shadow root's upper boundary provides. That is, events are retargeted to look like they've come from the host element rather than internal elements to the Shadow DOM.

Play Action 1

This one is interesting. You should see a mouseout from the host element (<div data-host>) to the blue node. Even though it's a distributed node, it's still in the host, not the ShadowDOM. Mousing further down into yellow again causes a mouseout on the blue node.

Play Action 2

There is one mouseout that appears on host (at the very end). Normally you'd see mouseout events trigger for all of the yellow blocks. However, in this case these elements are internal to the Shadow DOM and the event doesn't bubble through its upper boundary.

Play Action 3

Notice that when you click the input, the focusin doesn't appear on the input but on the host node itself. It's been retargeted!

Events that are always stopped #

The following events never cross the shadow boundary:

abort
error
select
change
load
reset
resize
scroll
selectstart

Conclusion #

I hope you'll agree that Shadow DOM is incredibly powerful. For the first time ever, we have proper encapsulation without the extra baggage of <iframe>s or other older techniques.

Shadow DOM is certainly complex beast, but it's a beast worth adding to the web platform. Spend some time with it. Learn it. Ask questions.

If you want to learn more, see Dominic's intro article Shadow DOM 101 and my Shadow DOM 201: CSS & Styling article.

Shadow DOM 201

2013-03-15T00:00:00Z

This article discusses more of the amazing things you can do with Shadow DOM. It builds on the concepts discussed in Shadow DOM 101. If you're looking for an introduction, see that article.

Introduction #

Let's face it. There's nothing sexy about unstyled markup. Lucky for us, the brilliant folks behind Web Components foresaw this and didn't leave us hanging. The CSS Scoping Module defines many options for styling content in a shadow tree.

Style encapsulation #

One of the core features of Shadow DOM is the shadow boundary. It has a lot of nice properties, but one of the best is that it provides style encapsulation for free. Stated another way:

<div><h3>Light DOM</h3></div>
<script>
var root = document.querySelector('div').createShadowRoot();
root.innerHTML = `
  <style>
    h3 {
      color: red;
    }
  </style>
  <h3>Shadow DOM</h3>
`;
</script>

There are two interesting observations about this demo:

There are other h3s on this page, but the only one that matches the h3 selector, and therefore styled red, is the one in the ShadowRoot. Again, scoped styles by default.
Other styles rules defined on this page that target h3s don't bleed into my content. That's because selectors don't cross the shadow boundary.

Moral of the story? We have style encapsulation from the outside world. Thanks Shadow DOM!

Styling the host element #

The :host allows you to select and style the element hosting a shadow tree:

<button class="red">My Button</button>
<script>
var button = document.querySelector('button');
var root = button.createShadowRoot();
root.innerHTML = `
  <style>
    :host {
      text-transform: uppercase;
    }
  </style>
  <content></content>
`;
</script>

One gotcha is that rules in the parent page have higher specificity than :host rules defined in the element, but lower specificity than a style attribute defined on the host element. This allows users to override your styling from the outside. :host also only works in the context of a ShadowRoot so you can't use it outside of Shadow DOM.

The functional form of :host(<selector>) allows you to target the host element if it matches a <selector>.

Example - match only if the element itself has the class .different (e.g. <x-foo class="different"></x-foo>):

:host(.different) {
    ...
}

Reacting to user states #

A common use case for :host is when you're creating a Custom Element and want to react to different user states (:hover, :focus, :active, etc.).

<style>
  :host {
    opacity: 0.4;
    transition: opacity 420ms ease-in-out;
  }
  :host(:hover) {
    opacity: 1;
  }
  :host(:active) {
    position: relative;
    top: 3px;
    left: 3px;
  }
</style>

Theming an element #

The :host-context(<selector>) pseudo class matches the host element if it or any of its ancestors matches <selector>.

A common use of :host-context() is for theming an element based on its surrounds. For example, many people do theming by applying a class to <html> or <body>:

<body class="different">
  <x-foo></x-foo>
</body>

You can :host-context(.different) to style <x-foo> when it's a descendant of an element with the class .different:

:host-context(.different) {
  color: red;
}

This gives you the ability encapsulate style rules in an element's Shadow DOM that uniquely style it, based on its context.

Support multiple host types from within one shadow root #

Another use for :host is if you're creating a theming library and want to support styling many types of host elements from within the same Shadow DOM.

:host(x-foo) {
    /* Applies if the host is a <x-foo> element.*/
}

:host(x-foo:host) {
    /* Same as above. Applies if the host is a <x-foo> element. */
}

:host(div) {
    /* Applies if the host element is a <div>. */
}

Styling Shadow DOM internals from the outside #

The ::shadow pseudo-element and /deep/ combinator are like having a Vorpal sword of CSS authority. They allow piercing through Shadow DOM's boundary to style elements within shadow trees.

The ::shadow pseudo-element #

If an element has at least one shadow tree, the ::shadow pseudo-element matches the shadow root itself. It allows you to write selectors that style nodes internal to an element's shadow dom.

For example, if an element is hosting a shadow root, you can write #host::shadow span {} to style all of the spans within its shadow tree.

<style>
  #host::shadow span {
    color: red;
  }
</style>

<div id="host">
  <span>Light DOM</span>
</div>

<script>
  var host = document.querySelector('div');
  var root = host.createShadowRoot();
  root.innerHTML = `
    <span>Shadow DOM</span>
    <content></content>
  `;
</script>

Example (custom elements) - <x-tabs> has <x-panel> children in its Shadow DOM. Each panel hosts its own shadow tree containing h2 headings. To style those headings from the main page, one could write:

x-tabs::shadow x-panel::shadow h2 {
    ...
}

The /deep/ combinator #

The /deep/ combinator is similar to ::shadow, but more powerful. It completely ignores all shadow boundaries and crosses into any number of shadow trees. Put simply, /deep/ allows you to drill into an element's guts and target any node.

The /deep/ combinator is particularly useful in the world of Custom Elements where it's common to have multiple levels of Shadow DOM. Prime examples are nesting a bunch of custom elements (each hosting their own shadow tree) or creating an element that inherits from another using <shadow>.

Example (custom elements) - select all <x-panel> elements that are descendants of <x-tabs>, anywhere in the tree:

x-tabs /deep/ x-panel {
    ...
}

Example - style all elements with the class .library-theme, anywhere in a shadow tree:

body /deep/ .library-theme {
    ...
}

Working with querySelector() #

Just like .shadowRoot opens shadow trees up for DOM traversal, the combinators open shadow trees for selector traversal. Instead of writing a nested chain of madness, you can write a single statement:

// No fun.
document.querySelector('x-tabs').shadowRoot
        .querySelector('x-panel').shadowRoot
        .querySelector('#foo');

// Fun.
document.querySelector('x-tabs::shadow x-panel::shadow #foo');

Styling native elements #

Native HTML controls are a challenge to style. Many people simply give up and roll their own. However, with ::shadow and /deep/, any element in the web platform that uses Shadow DOM can be styled. Great examples are the <input> types and <video>:

video /deep/ input[type="range"] {
  background: hotpink;
}

Creating style hooks #

Customization is good. In certain cases, you may want to poke holes in your Shadow's styling shield and create hooks for others to style.

Using ::shadow and /deep/ #

There's a lot of power behind /deep/. It gives component authors a way to designate individual elements as styleable or a slew of elements as themeable.

Example - style all elements that have the class .library-theme, ignoring all shadow trees:

body /deep/ .library-theme {
    ...
}

Using custom pseudo elements #

Both WebKit and Firefox define pseudo elements for styling internal pieces of native browser elements. A good example is the input[type=range]. You can style the slider thumb <span style="color:blue">blue</span> by targeting ::-webkit-slider-thumb:

input[type=range].custom::-webkit-slider-thumb {
  -webkit-appearance: none;
  background-color: blue;
  width: 10px;
  height: 40px;
}

Similar to how browsers provide styling hooks into some internals, authors of Shadow DOM content can designate certain elements as styleable by outsiders. This is done through custom pseudo elements.

You can designate an element as a custom pseudo element by using the pseudo attribute. Its value, or name, needs to be prefixed with "x-". Doing so creates an association with that element in the shadow tree and gives outsiders a designated lane to cross the shadow boundary.

Here's an example of creating a custom slider widget and allowing someone to style its slider thumb blue:

<style>
  #host::x-slider-thumb {
    background-color: blue;
  }
</style>
<div id="host"></div>
<script>
  var root = document.querySelector('#host').createShadowRoot();
  root.innerHTML = `
    <div>
      <div pseudo="x-slider-thumb"></div>' +
    </div>
  `;
</script>

Using CSS Variables #

A powerful way to create theming hooks will be through CSS Variables. Essentially, creating "style placeholders" for other users to fill in.

Imagine a custom element author who marks out variable placeholders in their Shadow DOM. One for styling an internal button's font and another for its color:

button {
  color: var(--button-text-color, pink); /* default color will be pink */
  font-family: var(--button-font);
}

Then, the embedder of the element defines those values to their liking. Perhaps to match the super cool Comic Sans theme of their own page:

#host {
  --button-text-color: green;
  --button-font: "Comic Sans MS", "Comic Sans", cursive;
}

Due to the way CSS Variables inherit, everything is peachy and this works beautifully! The whole picture looks like this:

<style>
  #host {
    --button-text-color: green;
    --button-font: "Comic Sans MS", "Comic Sans", cursive;
  }
</style>
<div id="host">Host node</div>
<script>
  var root = document.querySelector('#host').createShadowRoot();
  root.innerHTML = `
    <style>
      button {
        color: var(--button-text-color, pink);
        font-family: var(--button-font);
      }
    </style>
    <content></content>
  `;
</script>

Resetting styles #

Inheritable styles like fonts, colors, and line-heights continue to affect elements in the Shadow DOM. However for maximum flexibility, Shadow DOM gives us the resetStyleInheritance property to control what happens at the shadow boundary. Think of it as a way to start fresh when creating a new component.

resetStyleInheritance

false - Default. inheritable CSS properties continue to inherit.
true - resets inheritable properties to initial at the boundary.

Below is a demo that shows how the shadow tree is affected by changing resetStyleInheritance:

<div>
  <h3>Light DOM</h3>
</div>

<script>
  var root = document.querySelector('div').createShadowRoot();
  root.resetStyleInheritance = <span id="code-resetStyleInheritance">false</span>;
  root.innerHTML = `
    <style>
      h3 {
        color: red;
      }
    </style>
    <h3>Shadow DOM</h3>
    <content select="h3"></content>
  `;
</script>

<div class="demoarea" style="width:225px;">
  <div id="style-ex-inheritance"><h3 class="border">Light DOM</div>
</div>
<div id="inherit-buttons">
  <button id="demo-resetStyleInheritance">resetStyleInheritance=false</button>
</div>

<script>
  var container = document.querySelector('#style-ex-inheritance');
  var root = container.createShadowRoot();
  //root.resetStyleInheritance = false;
  root.innerHTML = '<style>h3{ color: red; }</style><h3>Shadow DOM<content select="h3"></content>';

  document.querySelector('#demo-resetStyleInheritance').addEventListener('click', function(e) {
    root.resetStyleInheritance = !root.resetStyleInheritance;
    e.target.textContent = 'resetStyleInheritance=' + root.resetStyleInheritance;
    document.querySelector('#code-resetStyleInheritance').textContent = root.resetStyleInheritance;
  });
</script>

Understanding .resetStyleInheritance is a bit trickier, primarily because it only has an affect on CSS properties which are inheritable. It says: when you're looking for a property to inherit, at the boundary between the page and the ShadowRoot, don't inherit values from the host but use the initial value instead (per the CSS spec).

If you're unsure about which properties inherit in CSS, check out this handy list or toggle the "Show inherited" checkbox in the Element panel.

Styling distributed nodes #

Distributed nodes are elements that render at an insertion point (a <content> element). The <content> element allows you to select nodes from the Light DOM and render them at predefined locations in your Shadow DOM. They're not logically in the Shadow DOM; they're still children of the host element. Insertion points are just a rendering thing.

Distributed nodes retain styles from the main document. That is, style rules from the main page continue to apply to the elements, even when they render at an insertion point. Again, distributed nodes are still logically in the light dom and don't move. They just render elsewhere. However, when the nodes get distributed into the Shadow DOM, they can take on additional styles defined inside the shadow tree.

::content pseudo element #

Distributed nodes are children of the host element, so how can we target them from within the Shadow DOM? The answer is the CSS ::content pseudo element. It's a way to target Light DOM nodes that pass through an insertion point. For example:

::content > h3 styles any h3 tags that pass through an insertion point.

Let's see an example:

<div>
  <h3>Light DOM</h3>
  <section>
    <div>I'm not underlined</div>
    <p>I'm underlined in Shadow DOM!</p>
  </section>
</div>

<script>
var div = document.querySelector('div');
var root = div.createShadowRoot();
root.innerHTML = `
  <style>
    h3 { color: red; }
      content[select="h3"]::content > h3 {
      color: green;
    }
    ::content section p {
      text-decoration: underline;
    }
  </style>
  <h3>Shadow DOM</h3>
  <content select="h3"></content>
  <content select="section"></content>
`;
</script>

Resetting styles at insertion points #

When creating a ShadowRoot, you have the option of resetting the inherited styles. <content> and <shadow> insertion points also have this option. When using these elements, either set the .resetStyleInheritance in JS or use the boolean reset-style-inheritance attribute on the element itself.

For a ShadowRoot or <shadow> insertion points: reset-style-inheritance means inheritable CSS properties are set to initial at the host, before they hit your shadow content. This location is known as the upper boundary.
For <content> insertion points: reset-style-inheritance means inheritable CSS properties are set to initial before the host's children are distributed at the insertion point. This location is known as the lower boundary.

Conclusion #

As authors of custom elements, we have a ton of options for controlling the look and feel of our content. Shadow DOM forms the basis for this brave new world.

Shadow DOM gives us scoped style encapsulation and a means to let in as much (or as little) of the outside world as we choose. By defining custom pseudo elements or including CSS Variable placeholders, authors can provide third-parties convenient styling hooks to further customize their content. All in all, web authors are in full control of how their content is represented.

HTML's New Template Tag

2013-02-26T00:00:00Z

Introduction #

The concept of templating is not new to web development. In fact, server-side templating languages/engines like Django (Python), ERB/Haml (Ruby), and Smarty (PHP) have been around for a long time. In the last couple of years however, we've seen an explosion of MVC frameworks spring up. All of them are slightly different, yet most share a common mechanic for rendering their presentational layer (aka da view): templates.

Let's face it. Templates are fantastic. Go ahead, ask around. Even its definition makes you feel warm and cozy:

"…does not have to be recreated each time…" Don't know about you, but I love avoiding extra work. Why then does the web platform lack native support for something developers clearly care about?

The WhatWG HTML Templates specification is the answer. It defines a new <template> element which describes a standard DOM-based approach for client-side templating. Templates allow you to declare fragments of markup which are parsed as HTML, go unused at page load, but can be instantiated later on at runtime. To quote Rafael Weinstein:

They're a place to put a big wad of HTML that you don't want the browser to mess with at all…for any reason.

Rafael Weinstein (spec author)

Feature Detection #

To feature detect <template>, create the DOM element and check that the .content property exists:

function supportsTemplate() {
    return 'content' in document.createElement('template');
}

if (supportsTemplate()) {
    // Good to go!
} else {
    // Use old templating techniques or libraries.
}

Declaring template content #

The HTML <template> element represents a template in your markup. It contains "template contents"; essentially inert chunks of cloneable DOM. Think of templates as pieces of scaffolding that you can use (and reuse) throughout the lifetime of your app.

To create a templated content, declare some markup and wrap it in the <template> element:

<template id="mytemplate">
    <img src="" alt="great image">
    <div class="comment"></div>
</template>

The pillars #

Wrapping content in a <template> gives us few important properties.

Its content is effectively inert until activated. Essentially, your markup is hidden DOM and does not render.
Any content within a template won't have side effects. Script doesn't run, images don't load, audio doesn't play,…until the template is used.
Content is considered not to be in the document. Using document.getElementById() or querySelector() in the main page won't return child nodes of a template.
Templates can be placed anywhere inside of <head>, <body>, or <frameset> and can contain any type of content which is allowed in those elements. Note that "anywhere" means that <template> can safely be used in places that the HTML parser disallows…all but content model children. It can also be placed as a child of <table> or <select>:

<table>
  <tr>
    <template id="cells-to-repeat">
      <td>some content</td>
    </template>
  </tr>
</table>

Activating a template #

To use a template, you need to activate it. Otherwise its content will never render. The simplest way to do this is by creating a deep copy of its .content using document.importNode(). The .content property is a read-only DocumentFragment containing the guts of the template.

var t = document.querySelector('#mytemplate');
// Populate the src at runtime.
t.content.querySelector('img').src = 'logo.png';

var clone = document.importNode(t.content, true);
document.body.appendChild(clone);

After stamping out a template, its content "goes live". In this particular example, the content is cloned, the image request is made, and the final markup is rendered.

Demos #

Example: Inert script #

This example demonstrates the inertness of template content. The <script> only runs when the button is pressed, stamping out the template.

<button onclick="useIt()">Use me</button>
<div id="container"></div>
<script>
  function useIt() {
    var content = document.querySelector('template').content;
    // Update something in the template DOM.
    var span = content.querySelector('span');
    span.textContent = parseInt(span.textContent) + 1;
    document.querySelector('#container').appendChild(
      document.importNode(content, true)
    );
  }
</script>

<template>
  <div>Template used: <span>0</span></div>
  <script>alert('Thanks!')</script>
</template>

Example: Creating Shadow DOM from a template #

Most people attach Shadow DOM to a host by setting a string of markup to .innerHTML:

<div id="host"></div>
<script>
  var shadow = document.querySelector('#host').createShadowRoot();
  shadow.innerHTML = '<span>Host node</span>';
</script>

The problem with this approach is that the more complex your Shadow DOM gets, the more string concatenation you're doing. It doesn't scale, things get messy fast, and babies start to cry. This approach is also how XSS was born in the first place! <template> to the rescue.

Something more sane would be to work with DOM directly by appending template content to a shadow root:

<template>
<style>
  :host {
    background: #f8f8f8;
    padding: 10px;
    transition: all 400ms ease-in-out;
    box-sizing: border-box;
    border-radius: 5px;
    width: 450px;
    max-width: 100%;
  }
  :host(:hover) {
    background: #ccc;
  }
  div {
    position: relative;
  }
  header {
    padding: 5px;
    border-bottom: 1px solid #aaa;
  }
  h3 {
    margin: 0 !important;
  }
  textarea {
    font-family: inherit;
    width: 100%;
    height: 100px;
    box-sizing: border-box;
    border: 1px solid #aaa;
  }
  footer {
    position: absolute;
    bottom: 10px;
    right: 5px;
  }
</style>
<div>
  <header>
    <h3>Add a Comment
  </header>
  <content select="p"></content>
  <textarea></textarea>
  <footer>
    <button>Post</button>
  </footer>
</div>
</template>

<div id="host">
  <p>Instructions go here</p>
</div>

<script>
  var shadow = document.querySelector('#host').createShadowRoot();
  shadow.appendChild(document.querySelector('template').content);
</script>

Gotchas #

Here are a few gotchas I've come across when using <template> in the wild:

If you're using modpagespeed, be careful of this bug. Templates that define inline <style scoped>, many be moved to the head with PageSpeed's CSS rewriting rules.
There's no way to "prerender" a template, meaning you cannot preload assets, process JS, download initial CSS, etc. That goes for both server and client. The only time a template renders is when it goes live.
Be careful with nested templates. They don't behave as you might expect. For example:
```
<template>
  <ul>
    <template>
      <li>Stuff</li>
    </template>
  </ul>
</template>
```
Activating the outer template will not active inner templates. That is to say, nested templates require that their children also be manually activated.

The road to a standard #

Let's not forget where we came from. The road to standards-based HTML templates has been a long one. Over the years, we've come up with some pretty clever tricks for creating reusable templates. Below are two common ones that I've come across. I'm including them in this article for comparison.

Method 1: Offscreen DOM #

One approach people have been using for a long time is to create "offscreen" DOM and hide it from view using the hidden attribute or display:none.

<div id="mytemplate" hidden>
  <img src="logo.png">
  <div class="comment"></div>
</div>

While this technique works, there are a number of downsides. The rundown of this technique:

Using DOM - the browser knows DOM. It's good at it. We can easily clone it.
Nothing is rendered - adding hidden prevents the block from showing.
Not inert - even though our content is hidden, a network request is still made for the image.
Painful styling and theming - an embedding page must prefix all of its CSS rules with #mytemplate in order to scope styles down to the template. This is brittle and there are no guarantees we won't encounter future naming collisions. For example, we're hosed if the embedding page already has an element with that id.

Method 2: Overloading script #

Another technique is overloading <script> and manipulating its content as a string. John Resig was probably the first to show this back in 2008 with his Micro Templating utility. Now there are many others, including some new kids on the block like handlebars.js.

For example:

<script id="mytemplate" type="text/x-handlebars-template">
  <img src="logo.png">
  <div class="comment"></div>
</script>

The rundown of this technique:

Nothing is rendered - the browser doesn't render this block because <script> is display:none by default.
Inert - the browser doesn't parse the script content as JS because its type is set to something other than "text/javascript".
Security issues - encourages the use of .innerHTML. Run-time string parsing of user-supplied data can easily lead to XSS vulnerabilities.

Conclusion #

Remember when jQuery made working with DOM dead simple? The result was querySelector()/querySelectorAll() being added to the platform. Obvious win, right? A library popularized fetching DOM with CSS selectors and standards later adopted it. It doesn't always work that way, but I love when it does.

I think <template> is a similar case. It standardizes the way we do client-side templating, but more importantly, it removes the need for our 2008 hacks. Making the entire web authoring process more sane, more maintainable, and more full-featured is always a good thing in my book.

Additional resources #

WhatWG Specification
Introduction to Web Components
<web>components</web> (video) - a fantastically comprehensive presentation by yours truly.

Screensharing a browser tab in HTML5?

2012-09-21T00:00:00Z

In the past couple of years, I've helped a few different companies achieve screensharing-like functionality using only browser technologies. From my experience, implementing VNC solely in web platform technologies (i.e. no plugins) is a hard problem. There are a lot of things to consider and a lot of challenges to overcome. Relaying mouse pointer position, forwarding keystrokes, and achieving full 24-bit color repaints at 60fps are just a few of the issues.

Capturing tab content #

If we remove the complexities of traditional screen sharing and focus on sharing the contents of a browser tab, the problem greatly simplifies to a.) capturing the visible tab in its current state, and b.) sending that "frame" across the wire. Essentially, we need a way to snapshot the DOM and share it out.

The sharing part is easy. Websockets are very capable of sending data in different formats (string, JSON, binary). The snapshotting part is a much harder problem. Projects like html2canvas have tackled screencapturing HTML by re-implementing the browser's rendering engine…in JavaScript! Another example is Google Feedback, though it's not open-source. These types of projects are very cool, but they're also dreadfully slow. You'd be lucky to get 1fps throughput, much less that covetted 60fps.

This article discusses a few of my favorite proof-of-concept solutions for "screensharing" a tab.

Method 1: Mutation Observers + WebSocket #

One approach for mirroring a tab was demonstrated by +Rafael Weinstein earlier this year. His technique uses Mutation Observers and a WebSocket.

Essentially, the tab that the presenter is sharing watches for changes to the page and sends diffs to the viewer using a websocket. As the user scrolls or interacts with the page, the observers pick up these changes and report them back to the viewer using Rafael's mutation summary library. This keeps things performant. The entire page isn't sent for every frame.

As Rafael points out in the video, this is merely a proof of concept. Still, I think it's a neat way to combine a newer platform feature like Mutation Observers with an older one like Websockets.

Method 2: Blob from an HTMLDocument + Binary WebSocket #

This next method is one that recently dawned on me. It's similar to the Mutation Observers approach, but instead of sending summary diffs, it creates a Blob clone of the entire HTMLDocument and sends it across a binary websocket. Here's the setup by setup:

Rewrite all URLs on the page to be absolute. This prevents static image and CSS assets from containing broken links.
Clone the page's document element: document.documentElement.cloneNode(true);
Make the clone readonly, non-selectable, and prevent scrolling using CSS pointer-events: 'none';user-select:'none';overflow:hidden;
Capture the current scroll position of the page and add them as data-* attributes on the duplicate.
Create a new Blob() from the .outerHTML of the duplicate.

The code looks something like this (I've made simplifications from the full source):

function screenshotPage() {
    // 1. Rewrite current doc's imgs, css, and script URLs to be absolute before
    // we duplicate. This ensures no broken links when viewing the duplicate.
    urlsToAbsolute(document.images);
    urlsToAbsolute(document.querySelectorAll("link[rel='stylesheet']"));
    urlsToAbsolute(document.scripts);

    // 2. Duplicate entire document tree.
    var screenshot = document.documentElement.cloneNode(true);

    // 3. Screenshot should be readyonly, no scrolling, and no selections.
    screenshot.style.pointerEvents = 'none';
    screenshot.style.overflow = 'hidden';
    screenshot.style.userSelect = 'none'; // Note: need vendor prefixes

    // 4. … read on …

    // 5. Create a new .html file from the cloned content.
    var blob = new Blob([screenshot.outerHTML], {type: 'text/html'});

    // Open a popup to new file by creating a blob URL.
    window.open(window.URL.createObjectURL(blob));
}

urlsToAbsolute() contains simple regexs to rewrite relative/schemeless URLs to absolute ones. That's necessary so images, css, fonts, and scripts don't break when viewed in the context of a blob URL (e.g. from a different origin).

One last tweak I made was to add scroll support. When presenter scrolls the page, the viewer should follow along. To do that, I stash the current scrollX and scrollY positions as data-* attributes on the duplicate HTMLDocument. Before the final Blob is created, a bit of JS is injected that fires on page load:

// 4. Preserve current x,y scroll position of this page. See addOnPageLoad().
screenshot.dataset.scrollX = window.scrollX;
screenshot.dataset.scrollY = window.scrollY;

// 4.5. When screenshot loads (e.g. in blob URL), scroll it to the same location
// of this page. Do this by appending a window.onDOMContentLoaded listener
// which pulls out the screenshot (dupe's) saved scrollX/Y state on the DOM.
var script = document.createElement('script');
script.textContent = '(' + addOnPageLoad_.toString() + ')();'; // self calling.
screenshot.querySelector('body').appendChild(script);

// NOTE: Not to be invoked directly. When the screenshot loads, scroll it
// to the same x,y location of original page.
function addOnPageLoad() {
    window.addEventListener('DOMContentLoaded', function(e) {
    var scrollX = document.documentElement.dataset.scrollX || 0;
    var scrollY = document.documentElement.dataset.scrollY || 0;
    window.scrollTo(scrollX, scrollY);
    });

Faking the scrolling gives the impression that we've screenshoted a portion of the original page, when in fact, we've duplicated the entire thing and merely repositioned it. #clever

Demo #

But for tab sharing, we need to continuously capture the tab and send it to viewers. For that I've written a small Node websocket server, app, and bookmarklet that demonstrates the flow. If you're not interested in the code, here's a short video of things in action:

Future Improvements #

One optimization is not to duplicate the entire document on every frame. That's wasteful and something the Mutation Observer example does well at. Another improvement is to handle relative CSS background images in urlsToAbsolute(). That's something the current script doesn't consider.

Method 3: Chrome Extension API + Binary WebSocket #

At Google I/O 2012, I demonstrated another approach for screensharing the contents of a browser tab. However, this one is a cheat. It requires a Chrome Extension API: not pure HTML5 magic.

The source for this one is also up on Github, but the gist is:

Capture the current tab as a .png dataURL. Chrome Extensions have an API for that chrome.tabs.captureVisibleTab().
Convert the dataURL to a Blob. See convertDataURIToBlob() helper.
Send each Blob (frame) to the viewer using a binary websocket by setting socket.responseType='blob'.

Example #

Here's code to screenshot the current tab as a png and send the frame through a websocket:

var IMG_MIMETYPE = 'images/jpeg'; // Update to image/webp when crbug.com/112957 is fixed.
var IMG_QUALITY = 80; // [0-100]
var SEND_INTERVAL = 250; // ms

var ws = new WebSocket('ws://…', 'dumby-protocol');
ws.binaryType = 'blob';

function captureAndSendTab() {
    var opts = {format: IMG_MIMETYPE, quality: IMG_QUALITY};
    chrome.tabs.captureVisibleTab(null, opts, function(dataUrl) {
    // captureVisibleTab returns a dataURL. Decode it -> convert to blob -> send.
    ws.send(convertDataURIToBlob(dataUrl, IMG_MIMETYPE));
    });
}

var intervalId = setInterval(function() {
    if (ws.bufferedAmount == 0) {
    captureAndSendTab();
    }
}, SEND_INTERVAL);

Future Improvements #

The framerate is surprisingly good for this one, but it could be even better. One improvement would be to remove the overhead of converting the dataURL to a Blob. Unfortunately, chrome.tabs.captureVisibleTab() only gives us a dataURL. If it returned a Blob or Typed Array, we could send that directly through the websocket rather than doing the conversion to a Blob ourselves. Please star crbug.com/32498 to make that happen!

Method 4: WebRTC - the true future #

Last but not least!

The future of screensharing in the browser will be realized by WebRTC. In August 14, 2012, the team proposed a WebRTC Tab Content Capture API for sharing tab contents:

Until this guy is ready we're left with methods 1-3.

Conclusion #

So browser tab sharing is possible with today's web technology!

But…that statement should be taken with a grain of salt. While neat, the techniques in this article fall short of a great sharing UX in one way or another. That will all change with the WebRTC Tab Content Capture effort, but until it's a reality, we're left with browser plugins or limited solutions like the ones covered here.

Have more techniques? Post a comment!

Capture audio and video in HTML5

2012-02-22T00:00:00Z

Introduction #

Audio/Video capture has been the "Holy Grail" of web development for a long time. For many years we've had to rely on browser plugins (Flash or Silverlight) to get the job done. Come on!

HTML5 to the rescue. It might not be apparent, but the rise of HTML5 has brought a surge of access to device hardware. Geolocation (GPS), the Orientation API (accelerometer), WebGL (GPU), and the Web Audio API (audio hardware) are perfect examples. These features are ridiculously powerful, exposing high level JavaScript APIs that sit on top of the system's underlying hardware capabilities.

This tutorial introduces a new API, [navigator.getUserMedia()][getusermedia-spec], which allows web apps to access a user's camera and microphone.

The road to getUserMedia() #

If you're not aware of its history, the way we arrived at the getUserMedia() API is an interesting tale.

Several variants of "Media Capture APIs" have evolved over the past few years. Many folks recognized the need to be able to access native devices on the web, but that led everyone and their mom to put together a new spec. Things got so messy that the W3C finally decided to form a working group. Their sole purpose? Make sense of the madness! The Device APIs Policy (DAP) Working Group has been tasked to consolidate + standardize the plethora of proposals.

I'll try to summarize what happened in 2011…

Round 1: HTML Media Capture #

HTML Media Capture was the DAP's first go at standardizing media capture on the web. It works by overloading the <input type="file"> and adding new values for the accept parameter.

If you wanted to let users take a snapshot of themselves with the webcam, that's possible with capture=camera:

<input type="file" accept="image/*;capture=camera">

Recording a video or audio is similar:

<input type="file" accept="video/*;capture=camcorder">
<input type="file" accept="audio/*;capture=microphone">

Kinda nice right? I particularly like that it reuses a file input. Semantically, it makes a lot of sense. Where this particular "API" falls short is the ability to do realtime effects (e.g. render live webcam data to a <canvas> and apply WebGL filters). HTML Media Capture only allows you to record a media file or take a snapshot in time.

Support:

Android 3.0 browser - one of the first implementations. Check out this video to see it in action.
Chrome for Android (0.16)
Firefox Mobile 10.0
iOS6 Safari and Chrome (partial support)

Round 2: device element #

Many thought HTML Media Capture was too limiting, so a new spec emerged that supported any type of (future) device. Not surprisingly, the design called for a new element, the <device> element, which became the predecessor to getUserMedia().

Opera was among the first browsers to create initial implementations of video capture based on the <device> element. Soon after (the same day to be precise), the WhatWG decided to scrap the <device> tag in favor of another up and comer, this time a JavaScript API called navigator.getUserMedia(). A week later, Opera put out new builds that included support for the updated getUserMedia() spec. Later that year, Microsoft joined the party by releasing a Lab for IE9 supporting the new spec.

Here's what <device> would have looked like:

<device type="media" onchange="update(this.data)"></device>
<video autoplay></video>
<script>
    function update(stream) {
    document.querySelector('video').src = stream.url;
    }
</script>

Support:

Unfortunately, no released browser ever included <device>. One less API to worry about I guess :) <device> did have two great things going for it though: 1.) it was semantic, and 2.) it was easily extendible to support more than just audio/video devices.

Take a breath. This stuff moves fast!

Round 3: WebRTC #

The <device> element eventually went the way of the Dodo.

The pace to find a suitable capture API accelerated thanks to the larger [WebRTC][webrtc-spec] (Web Real Time Communications) effort. That spec is overseen by the W3C WebRTC Working Group. Google, Opera, Mozilla, and a few others have implementations.

getUserMedia() is related to WebRTC because it's the gateway into that set of APIs. It provides the means to access the user's local camera/microphone stream.

Support:

getUserMedia() has been supported since Chrome 21, Opera 18, and Firefox 17.

Getting started #

With navigator.getUserMedia(), we can finally tap into webcam and microphone input without a plugin. Camera access is now a call away, not an install away. It's baked directly into the browser. Excited yet?

Feature detection #

Feature detecting is a simple check for the existence of navigator.getUserMedia:

function hasGetUserMedia() {
    return !!(navigator.getUserMedia || navigator.webkitGetUserMedia ||
            navigator.mozGetUserMedia || navigator.msGetUserMedia);
}

if (hasGetUserMedia()) {
    // Good to go!
} else {
    alert('getUserMedia() is not supported in your browser');
}

You can also use Modernizr to detect getUserMedia to avoid the vendor prefix dance yourself:

if (Modernizr.getusermedia){
    var gUM = Modernizr.prefixed('getUserMedia', navigator);
    gUM({video: true}, function( //...
    //...
}

Gaining access to an input device #

To use the webcam or microphone, we need to request permission. The first parameter to getUserMedia() is an object specifying the details and requirements for each type of media you want to access. For example, if you want to access the webcam, the first parameter should be {video: true}. To use both the microphone and camera, pass {video: true, audio: true}:

<video autoplay></video>

<script>
    var errorCallback = function(e) {
    console.log('Reeeejected!', e);
    };

    // Not showing vendor prefixes.
    navigator.getUserMedia({video: true, audio: true}, function(localMediaStream) {
    var video = document.querySelector('video');
    video.src = window.URL.createObjectURL(localMediaStream);

    // Note: onloadedmetadata doesn't fire in Chrome when using it with getUserMedia.
    // See crbug.com/110938.
    video.onloadedmetadata = function(e) {
        // Ready to go. Do some stuff.
    };
    }, errorCallback);
</script>

OK. So what's going on here? Media capture is a perfect example of new HTML5 APIs working together. It works in conjunction with our other HTML5 buddies, <audio> and <video>. Notice that we're not setting a src attribute or including <source> elements on the <video> element. Instead of feeding the video a URL to a media file, we're feeding it a Blob URL obtained from a LocalMediaStream object representing the webcam.

I'm also telling the <video> to autoplay, otherwise it would be frozen on the first frame. Adding controls also works as you'd expected.

If you want something that works cross-browser, try this:

navigator.getUserMedia  = navigator.getUserMedia ||
                            navigator.webkitGetUserMedia ||
                            navigator.mozGetUserMedia ||
                            navigator.msGetUserMedia;

var video = document.querySelector('video');

if (navigator.getUserMedia) {
    navigator.getUserMedia({audio: true, video: true}, function(stream) {
    video.src = window.URL.createObjectURL(stream);
    }, errorCallback);
} else {
    video.src = 'somevideo.webm'; // fallback.
}

Setting media constraints (resolution, height, width) #

The first parameter to getUserMedia() can also be used to specify more requirements (or constraints) on the returned media stream. For example, instead of just indicating you want basic access to video (e.g. {vide: true}), you can additionally require the stream to be HD:

var hdConstraints = {
    video: {
    mandatory: {
        minWidth: 1280,
        minHeight: 720
    }
    }
};

navigator.getUserMedia(hdConstraints, successCallback, errorCallback);

...

var vgaConstraints = {
    video: {
    mandatory: {
        maxWidth: 640,
        maxHeight: 360
    }
    }
};

navigator.getUserMedia(vgaConstraints, successCallback, errorCallback);

For more configurations, see the constraints API

Selecting a media source #

In Chrome 30 or later, getUserMedia() also supports selecting the video/audio source using the MediaStreamTrack.getSources() API.

In this example, the last microphone and camera that's found is selected as the media stream source:

MediaStreamTrack.getSources(function(sourceInfos) {
    var audioSource = null;
    var videoSource = null;

    for (var i = 0; i != sourceInfos.length; ++i) {
    var sourceInfo = sourceInfos[i];
    if (sourceInfo.kind === 'audio') {
        console.log(sourceInfo.id, sourceInfo.label || 'microphone');

        audioSource = sourceInfo.id;
    } else if (sourceInfo.kind === 'video') {
        console.log(sourceInfo.id, sourceInfo.label || 'camera');

        videoSource = sourceInfo.id;
    } else {
        console.log('Some other kind of source: ', sourceInfo);
    }
    }

    sourceSelected(audioSource, videoSource);
});

function sourceSelected(audioSource, videoSource) {
    var constraints = {
    audio: {
        optional: [{sourceId: audioSource}]
    },
    video: {
        optional: [{sourceId: videoSource}]
    }
    };

    navigator.getUserMedia(constraints, successCallback, errorCallback);
}

Check out Sam Dutton's great demo of how to let users select the media source.

Security #

Some browsers throw up an infobar upon calling getUserMedia(), which gives users the option to grant or deny access to their camera/mic. The spec unfortunately is very quiet when it comes to security. For example, here is Chrome's permission dialog:

Permission dialog in Chrome

If your app is running from SSL (https://), this permission will be persistent. That is, users won't have to grant/deny access every time.

Providing fallback #

For users that don't have support for getUserMedia(), one option is to fallback to an existing video file if the API isn't supported and/or the call fails for some reason:

// Not showing vendor prefixes or code that works cross-browser:

function fallback(e) {
    video.src = 'fallbackvideo.webm';
}

function success(stream) {
    video.src = window.URL.createObjectURL(stream);
}

if (!navigator.getUserMedia) {
    fallback();
} else {
    navigator.getUserMedia({video: true}, success, fallback);
}

The synchronous FileSystem API for workers

2011-10-25T00:00:00Z

Introduction #

The HTML5 FileSystem API and Web Workers are massively powerful in their own regard. The FileSystem API finally brings hierarchical storage and file I/O to web applications and Workers bring true asynchronous 'multi-threading' to JavaScript. However, when you use these APIs together, you can build some truly interesting apps.

This tutorial provides a guide and code examples for leveraging the HTML5 FileSystem inside of a Web Worker. It assumes a working knowledge of both APIs. If you're not quite ready to dive in or are interested in learning more about those APIs, read two great tutorials that discuss the basics: Exploring the FileSystem APIs and Basics of Web Workers.

Synchronous vs. Asynchronous APIs #

Asynchronous JavaScript APIs can be tough to use. They're large. They're complex. But what's most frustrating is that they offer plenty of opportunities for things to go wrong. The last thing you want to deal with is layering on a complex asynchronous API (FileSystem) in an already asynchronous world (Workers)! The good news is that the FileSystem API defines a synchronous version to ease the pain in Web Workers.

For the most part, the synchronous API is exactly the same as its asynchronous cousin. The methods, properties, features, and functionality will be familiar. The major deviations are:

The synchronous API can only be used within a Web Worker context, whereas the asynchronous API can be used in and out of a Worker.
Callbacks are out. API methods now return values.
The global methods on the window object (requestFileSystem() and resolveLocalFileSystemURL()) become requestFileSystemSync() and resolveLocalFileSystemSyncURL().

Apart from these exceptions, the APIs are the same. OK, we're good to go!

Requesting a filesystem #

A web application obtains access to the synchronous filesystem by requesting a LocalFileSystemSync object from within a Web Worker. The requestFileSystemSync() is exposed to the Worker's global scope:

var fs = requestFileSystemSync(TEMPORARY, 1024*1024 /*1MB*/);

Notice the new return value now that we're using the synchronous API as well as the absence of success and error callbacks.

As with the normal FileSystem API, methods are prefixed at the moment:

self.requestFileSystemSync = self.webkitRequestFileSystemSync ||
                                 self.requestFileSystemSync;

Dealing with quota #

Currently, it's not possible to request PERSISTENT quota in a Worker context. I recommend taking care of quota issues outside of Workers. The process could look like something this:

worker.js: Wrap any FileSystem API code in a try/catch so any QUOTA_EXCEED_ERR errors are caught.
worker.js: If you catch a QUOTA_EXCEED_ERR, send a postMessage('get me more quota') back to the main app.
main app: Go through the window.webkitStorageInfo.requestQuota() dance when #2 is received.
main app: After the user grants more quota, send postMessage('resume writes') back to the worker to inform it of additional storage space.

That's a fairly involved workaround, but it should work. See requesting quota for more information on using PERSISTENT storage with the FileSystem API.

Working with files and directories #

The synchronous version of getFile() and getDirectory() return a FileEntrySync and DirectoryEntrySync, respectively.

For example, the following code creates an empty file called "log.txt" in the root directory.

var fileEntry = fs.root.getFile('log.txt', {create: true});

The following creates a new directory in the root folder.

var dirEntry = fs.root.getDirectory('mydir', {create: true});

Handling errors #

If you've never had to debug Web Worker code, I envy you! It can be a real pain to figure out what is going wrong.

The lack of error callbacks in the synchronous world makes dealing with problems trickier than they should be. If we add the general complexity of debugging Web Worker code, you'll be frustrated in no time. One thing that can make life easier is to wrap all of your relevant Worker code in a try/catch. Then, if any errors occur, forward the error to the main app using postMessage():

function onError(e) {
    postMessage('ERROR: ' + e.toString());
}

try {
    // Error thrown if "log.txt" already exists.
    var fileEntry = fs.root.getFile('log.txt', {create: true, exclusive: true});
} catch (e) {
    onError(e);
}

Passing around Files, Blobs, and ArrayBuffers #

When Web Workers first came on the scene, they only allowed string data to be sent in postMessage(). Later, browsers began accepting serializable data, which meant passing a JSON object was possible. Recently however, some browsers like Chrome accept more complex data types to be passed through postMessage() using the structured clone algorithm.

What does this really mean? It means that it's a heck of a lot easier to pass binary data between main app and the Worker thread. Browsers that support structured cloning for Workers allow you to pass Typed Arrays, ArrayBuffers, Files, or Blobs into Workers. Although the data is still a copy, being able to pass a File means a performance benefit over the former approach, which involved base64ing the file before passing it into postMessage().

The following example passes a user-selected list of files to an dedicated Worker. The Worker simply passes through the file list (simple to show the returned data is actually a FileList) and the main app reads each file as an ArrayBuffer.

The sample also uses an improved version of the inline Web Worker technique described in Basics of Web Workers.

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="chrome=1">
    <title>Passing a FileList to a Worker</title>
    <script type="javascript/worker" id="fileListWorker">
    self.onmessage = function(e) {
    // TODO: do something interesting with the files.
    postMessage(e.data); // Pass through.
    };
    </script>
</head>
<body>
</body>

<input type="file" multiple>

<script>
document.querySelector('input[type="file"]').addEventListener('change', function(e) {
    var files = this.files;
    loadInlineWorker('#fileListWorker', function(worker) {

    // Setup handler to process messages from the worker.
    worker.onmessage = function(e) {

        // Read each file aysnc. as an array buffer.
        for (var i = 0, file; file = files[i]; ++i) {
        var reader = new FileReader();
        reader.onload = function(e) {
            console.log(this.result); // this.result is the read file as an ArrayBuffer.
        };
        reader.onerror = function(e) {
            console.log(e);
        };
        reader.readAsArrayBuffer(file);
        }

    };

    worker.postMessage(files);
    });
}, false);


function loadInlineWorker(selector, callback) {
    window.URL = window.URL || window.webkitURL || null;

    var script = document.querySelector(selector);
    if (script.type === 'javascript/worker') {
    var blob = new Blob([script.textContent]);
    callback(new Worker(window.URL.createObjectURL(blob));
    }
}
</script>
</html>

Reading files in a Worker #

It's perfectly acceptable to use the asynchronous FileReader API to read files in a Worker. However, there's a better way. In Workers, there's a synchronous API (FileReaderSync) that streamlines reading files:

Main app:

<!DOCTYPE html>
<html>
<head>
    <title>Using FileReaderSync Example</title>
    <style>
    #error { color: red; }
    </style>
</head>
<body>
<input type="file" multiple />
<output id="error"></output>
<script>
    var worker = new Worker('worker.js');

    worker.onmessage = function(e) {
    console.log(e.data); // e.data should be an array of ArrayBuffers.
    };

    worker.onerror = function(e) {
    document.querySelector('#error').textContent = [
        'ERROR: Line ', e.lineno, ' in ', e.filename, ': ', e.message].join('');
    };

    document.querySelector('input[type="file"]').addEventListener('change', function(e) {
    worker.postMessage(this.files);
    }, false);
</script>
</body>
</html>

worker.js

self.addEventListener('message', function(e) {
    var files = e.data;
    var buffers = [];

    // Read each file synchronously as an ArrayBuffer and
    // stash it in a global array to return to the main app.
    [].forEach.call(files, function(file) {
    var reader = new FileReaderSync();
    buffers.push(reader.readAsArrayBuffer(file));
    });

    postMessage(buffers);
}, false);

As expected, callbacks are gone with the synchronous FileReader. This simplifies the amount of callback nesting when reading files. Instead, the readAs* methods returns the read file.

Example: Fetching all entries #

In some cases, the synchronous API is much cleaner for certain tasks. Fewer callbacks are nice and certainly make things more readable. The real downside of the synchronous API stems from the limitations of Workers.

For security reasons, data between the calling app and a Web Worker thread is never shared. Data is always copied to and from the Worker when postMessage() is called. As a result, not every data type can be passed.

Unfortunately, FileEntrySync and DirectoryEntrySync don't currently fall into the accepted types. So how can you get entries back to the calling app? One way to circumvent the limitation is to return a list of filesystem: URLs instead of a list of entries. filesystem: URLs are just strings, so they're super easy to pass around. Furthermore, they can be resolved to entries in the main app using resolveLocalFileSystemURL(). That gets you back to a FileEntrySync/DirectoryEntrySync object.

Main app:

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="chrome=1">
<title>Listing filesystem entries using the synchronous API</title>
</head>
<body>
<script>
    window.resolveLocalFileSystemURL = window.resolveLocalFileSystemURL ||
                                        window.webkitResolveLocalFileSystemURL;

    var worker = new Worker('worker.js');
    worker.onmessage = function(e) {
    var urls = e.data.entries;
    urls.forEach(function(url, i) {
        window.resolveLocalFileSystemURL(url, function(fileEntry) {
        console.log(fileEntry.name); // Print out file's name.
        });
    });
    };

    worker.postMessage({'cmd': 'list'});
</script>
</body>
</html>

worker.js

self.requestFileSystemSync = self.webkitRequestFileSystemSync ||
                                self.requestFileSystemSync;

var paths = []; // Global to hold the list of entry filesystem URLs.

function getAllEntries(dirReader) {
    var entries = dirReader.readEntries();

    for (var i = 0, entry; entry = entries[i]; ++i) {
    paths.push(entry.toURL()); // Stash this entry's filesystem: URL.

    // If this is a directory, we have more traversing to do.
    if (entry.isDirectory) {
        getAllEntries(entry.createReader());
    }
    }
}

function onError(e) {
    postMessage('ERROR: ' + e.toString()); // Forward the error to main app.
}

self.onmessage = function(e) {
    var data = e.data;

    // Ignore everything else except our 'list' command.
    if (!data.cmd || data.cmd != 'list') {
    return;
    }

    try {
    var fs = requestFileSystemSync(TEMPORARY, 1024*1024 /*1MB*/);

    getAllEntries(fs.root.createReader());

    self.postMessage({entries: paths});
    } catch (e) {
    onError(e);
    }
};

Example: Downloading files using XHR2 #

A common use case for Workers is to download a bunch of files using XHR2, and write those files to the HTML5 FileSystem. This is a perfect task for a Worker thread!

The following example only fetches and writes one file, but you can image expanding it to download a set of files.

Main app:

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="chrome=1">
<title>Download files using a XHR2, a Worker, and saving to filesystem</title>
</head>
<body>
<script>
    var worker = new Worker('downloader.js');
    worker.onmessage = function(e) {
    console.log(e.data);
    };
    worker.postMessage({fileName: 'GoogleLogo',
                        url: 'googlelogo.png', type: 'image/png'});
</script>
</body>
</html>

downloader.js:

self.requestFileSystemSync = self.webkitRequestFileSystemSync ||
                                self.requestFileSystemSync;

function makeRequest(url) {
    try {
    var xhr = new XMLHttpRequest();
    xhr.open('GET', url, false); // Note: synchronous
    xhr.responseType = 'arraybuffer';
    xhr.send();
    return xhr.response;
    } catch(e) {
    return "XHR Error " + e.toString();
    }
}

function onError(e) {
    postMessage('ERROR: ' + e.toString());
}

onmessage = function(e) {
    var data = e.data;

    // Make sure we have the right parameters.
    if (!data.fileName || !data.url || !data.type) {
    return;
    }
    
    try {
    var fs = requestFileSystemSync(TEMPORARY, 1024 * 1024 /*1MB*/);

    postMessage('Got file system.');

    var fileEntry = fs.root.getFile(data.fileName, {create: true});

    postMessage('Got file entry.');

    var arrayBuffer = makeRequest(data.url);
    var blob = new Blob([new Uint8Array(arrayBuffer)], {type: data.type});

    try {
        postMessage('Begin writing');
        fileEntry.createWriter().write(blob);
        postMessage('Writing complete');
        postMessage(fileEntry.toURL());
    } catch (e) {
        onError(e);
    }

    } catch (e) {
    onError(e);
    }
};

Conclusion #

Web Workers are an underutilized and under-appreciated feature of HTML5. Most developers I talk to don't need the extra computational benefits, but they can be used for more than just pure computation. If you're skeptical (as I was), I hope this article has helped change your mind. Offloading things like disc operations (Filesystem API calls) or HTTP requests to a Worker are a natural fit and also help compartmentalize your code. The HTML5 File APIs inside of Workers opens a whole new can of awesomeness for web apps that a lot of folks haven't explored.

Downloading resources in HTML5 - a[download]

2011-08-01T00:00:00Z

Chrome now supports the HTML spec's new download attribute to a elements. When used, this attribute signifies that the resource it points to should be downloaded by the browser rather than navigating to it.

From Downloading Resources:

For example, clicking the following link downloads the .png as "MyGoogleLogo.png" instead of navigating to its href value: download me. The markup for this is as follows:

<a href="http://web-central.appspot.com/.../web-fundamentals-icon192x192.png" download="WebfundamentalsLogo">download me</a>

The real benefit of a[download] will be when working with blob: URLs and filesystem: URLs URLs. It'll give users a way to download content created/modified within your app.

Full Demo

One thing to note is that in the above example, the image has same origin with respect to website. If you try to use a link of image from different origin the link may not work as a navigating link rather than a downloading link. This is because many versions of browser does not support the download policy on cross-origin files. For example Chrome versions prior to 65 did allow downloading cross origin files and it was deprecated in later versions. Read this for more detail. You can use Content-Disposition header to force a download from other origin.

Browser support: only the current Chrome dev channel release (14.0.835.15+) supports this attribute.

Registering a custom protocol handler

2011-06-29T00:00:00Z

Chrome 13 finally includes navigator.registerProtocolHandler. This API allows web apps to register themselves as possible handlers for particular protocols. For example, users could select your application to handle "mailto" links.

navigator.registerProtocolHandler(
    'web+mystuff', 'http://example.com/rph?q=%s', 'My App');

The first parameter is the protocol. The second is the URL pattern of the application that should handle this scheme. The pattern should include a '%s' as a placeholder for data and it must must be on the same origin as the app attempting to register the protocol. Once the user approves access, you can use this link through your app, other sites, etc.:

<a href="web+mystuff:some+data">Open in "My App"</a>

Clicking that link makes a GET request to http://example.com/rph?q=web%2Bmystuff%3A:some%20data. Thus, you have to parse q parameter and manually strip out data from the protocol.

It's worth noting that Firefox has had navigator.registerProtocolHandler implemented since FF3. One difference in Chrome's implementation is around custom protocols. Those need to be prefixed with "web+", as seen in the example above. The following protocols do not need a "web+" prefix: "mailto", "mms", "nntp", "rtsp", "webcal".

More information on this API can be found on the MDN article.

New tricks in XMLHttpRequest2

2011-05-27T00:00:00Z

Introduction #

One of the unsung heros in the HTML5 universe is XMLHttpRequest. Strictly speaking XHR2 isn't HTML5. However, it's part of the incremental improvements browser vendors are making to the core platform. I'm including XHR2 in our new bag of goodies because it plays such an integral part in today's complex web apps.

Turns out our old friend got a huge makeover but many folks are unaware of its new features. XMLHttpRequest Level 2 introduces a slew of new capabilities which put an end to complicated hacks in our web apps; things like cross-origin requests, uploading progress events, and support for uploading/downloading binary data. These allow AJAX to work in concert with many of the bleeding edge HTML5 APIs such as File System API, Web Audio API, and WebGL.

This tutorial highlights some of the new features in XMLHttpRequest, especially those that can be used for working with files.

Fetching data #

Fetching a file as a binary blob has been painful with XHR. Technically, it wasn't even possible. One trick that has been well documented involves overriding the mime type with a user-defined charset as seen below.

The old way to fetch an image:

var xhr = new XMLHttpRequest();
xhr.open('GET', '/path/to/image.png', true);

// Hack to pass bytes through unprocessed.
xhr.overrideMimeType('text/plain; charset=x-user-defined');

xhr.onreadystatechange = function(e) {
  if (this.readyState == 4 && this.status == 200) {
    var binStr = this.responseText;
    for (var i = 0, len = binStr.length; i < len; ++i) {
      var c = binStr.charCodeAt(i);
      //String.fromCharCode(c & 0xff);
      var byte = c & 0xff;  // byte at offset i
    }
  }
};

xhr.send();

While this works, what you actually get back in the responseText is not a binary blob. It is a binary string representing the image file. We're tricking the server into passing the data back, unprocessed. Even though this little gem works, I'm going to call it black magic and advise against it. Anytime you resort to character code hacks and string manipulation for coercing data into a desirable format, that's a problem.

Specifying a response format #

In the previous example, we downloaded the image as a binary "file" by overriding the server's mime type and processing the response text as a binary string. Instead, let's leverage XMLHttpRequest's new responseType and response properties to inform the browser what format we want the data returned as.

xhr.responseType: Before sending a request, set the xhr.responseType to "text", "arraybuffer", "blob", or "document", depending on your data needs. Note, setting xhr.responseType = '' (or omitting) will default the response to "text".
xhr.response: After a successful request, the xhr's response property will contain the requested data as a DOMString, ArrayBuffer, Blob, or Document (depending on what was set for responseType.)

With this new awesomeness, we can rework the previous example, but this time, fetch the image as an Blob instead of a string:

var xhr = new XMLHttpRequest();
xhr.open('GET', '/path/to/image.png', true);
xhr.responseType = 'blob';

xhr.onload = function(e) {
  if (this.status == 200) {
    // Note: .response instead of .responseText
    var blob = new Blob([this.response], {type: 'image/png'});
    ...
  }
};

xhr.send();

Much nicer!

ArrayBuffer responses #

An ArrayBuffer is a generic fixed-length container for binary data. They are super handy if you need a generalized buffer of raw data, but the real power behind these guys is that you can create "views" of the underlying data using JavaScript typed arrays. In fact, multiple views can be created from a single ArrayBuffer source. For example, you could create an 8-bit integer array that shares the same ArrayBuffer as an existing 32-bit integer array from the same data. The underlying data remains the same, we just create different representations of it.

As an example, the following fetches our same image as an ArrayBuffer, but this time, creates an unsigned 8-bit integer array from that data buffer:

var xhr = new XMLHttpRequest();
xhr.open('GET', '/path/to/image.png', true);
xhr.responseType = 'arraybuffer';

xhr.onload = function(e) {
  var uInt8Array = new Uint8Array(this.response); // this.response == uInt8Array.buffer
  // var byte3 = uInt8Array[4]; // byte at offset 4
  ...
};

xhr.send();

Blob responses #

If you want to work directly with a Blob and/or don't need to manipulate any of the file's bytes, use xhr.responseType='blob':

window.URL = window.URL || window.webkitURL;  // Take care of vendor prefixes.

var xhr = new XMLHttpRequest();
xhr.open('GET', '/path/to/image.png', true);
xhr.responseType = 'blob';

xhr.onload = function(e) {
  if (this.status == 200) {
    var blob = this.response;

    var img = document.createElement('img');
    img.onload = function(e) {
      window.URL.revokeObjectURL(img.src); // Clean up after yourself.
    };
    img.src = window.URL.createObjectURL(blob);
    document.body.appendChild(img);
    ...
  }
};

xhr.send();

A Blob can be used in a number of places, including saving it to indexedDB, writing it to the HTML5 File System, or creating an Blob URL, as seen in this example.

Sending data #

Being able to download data in different formats is great, but it doesn't get us anywhere if we can't send these rich formats back to home base (the server). XMLHttpRequest has limited us to sending DOMString or Document (XML) data for some time. Not anymore. A revamped send() method has been overridden to accept any of the following types: DOMString, Document, FormData, Blob, File, ArrayBuffer. The examples in the rest of this section demonstrate sending data using each type.

Sending string data: xhr.send(DOMString) #

function sendText(txt) {
  var xhr = new XMLHttpRequest();
  xhr.open('POST', '/server', true);
  xhr.onload = function(e) {
    if (this.status == 200) {
      console.log(this.responseText);
    }
  };

  xhr.send(txt);
}

sendText('test string');

function sendTextNew(txt) {
  var xhr = new XMLHttpRequest();
  xhr.open('POST', '/server', true);
  xhr.responseType = 'text';
  xhr.onload = function(e) {
    if (this.status == 200) {
      console.log(this.response);
    }
  };
  xhr.send(txt);
}

sendTextNew('test string');

There's nothing new here, though the right snippet is slightly different. It sets responseType='text' for comparison. Again, omitting that line yields the same results.

Submitting forms: xhr.send(FormData) #

Many people are probably accustomed to using jQuery plugins or other libraries to handle AJAX form submissions. Instead, we can use FormData, another new data type conceived for XHR2. FormData is convenient for creating an HTML <form> on-the-fly, in JavaScript. That form can then be submitted using AJAX:

function sendForm() {
  var formData = new FormData();
  formData.append('username', 'johndoe');
  formData.append('id', 123456);

  var xhr = new XMLHttpRequest();
  xhr.open('POST', '/server', true);
  xhr.onload = function(e) { ... };

  xhr.send(formData);
}

Essentially, we're just dynamically creating a <form> and tacking on <input> values to it by calling the append method.

Of course, you don't need to create a <form> from scratch. FormData objects can be initialized from and existing HTMLFormElement on the page. For example:

<form id="myform" name="myform" action="/server">
  <input type="text" name="username" value="johndoe">
  <input type="number" name="id" value="123456">
  <input type="submit" onclick="return sendForm(this.form);">
</form>

function sendForm(form) {
  var formData = new FormData(form);

  formData.append('secret_token', '1234567890'); // Append extra data before send.

  var xhr = new XMLHttpRequest();
  xhr.open('POST', form.action, true);
  xhr.onload = function(e) { ... };

  xhr.send(formData);

  return false; // Prevent page from submitting.
}

An HTML form can include file uploads (e.g. <input type="file">) and FormData can handle that too. Simply append the file(s) and the browser will construct a multipart/form-data request when send() is called:

function uploadFiles(url, files) {
  var formData = new FormData();

  for (var i = 0, file; file = files[i]; ++i) {
    formData.append(file.name, file);
  }

  var xhr = new XMLHttpRequest();
  xhr.open('POST', url, true);
  xhr.onload = function(e) { ... };

  xhr.send(formData);  // multipart/form-data
}

document.querySelector('input[type="file"]').addEventListener('change', function(e) {
  uploadFiles('/server', this.files);
}, false);

Uploading a file or blob: xhr.send(Blob) #

We can also send File or Blob data using XHR. Keep in mind all Files are Blobs, so either works here.

This example creates a new text file from scratch using the Blob() constructor and uploads that Blob to the server. The code also sets up a handler to inform the user of the upload's progress:

<progress min="0" max="100" value="0">0% complete</progress>

function upload(blobOrFile) {
  var xhr = new XMLHttpRequest();
  xhr.open('POST', '/server', true);
  xhr.onload = function(e) { ... };

  // Listen to the upload progress.
  var progressBar = document.querySelector('progress');
  xhr.upload.onprogress = function(e) {
    if (e.lengthComputable) {
      progressBar.value = (e.loaded / e.total) * 100;
      progressBar.textContent = progressBar.value; // Fallback for unsupported browsers.
    }
  };

  xhr.send(blobOrFile);
}

upload(new Blob(['hello world'], {type: 'text/plain'}));

Uploading a chunk of bytes: xhr.send(ArrayBuffer) #

Last but not least, we can send ArrayBuffers as the XHR's payload.

function sendArrayBuffer() {
  var xhr = new XMLHttpRequest();
  xhr.open('POST', '/server', true);
  xhr.onload = function(e) { ... };

  var uInt8Array = new Uint8Array([1, 2, 3]);

  xhr.send(uInt8Array.buffer);
}

CORS allows web applications on one domain to make cross domain AJAX requests to another domain. It's dead simple to enable, only requiring a single response header to be sent by the server.

Enabling CORS requests #

Let's say your application lives on example.com and you want to pull data from www.example2.com. Normally if you tried to make this type of AJAX call, the request would fail and the browser would throw an origin mismatch error. With CORS, www.example2.com can choose to allow requests from example.com by simply adding a header:

Access-Control-Allow-Origin: http://example.com

Access-Control-Allow-Origin can be added to a single resource under a site or across the entire domain. To allow any domain to make a request to you, set:

Access-Control-Allow-Origin: *

In fact, this site (html5rocks.com) has enabled CORS on all of its pages. Fire up the Developer Tools and you'll see the Access-Control-Allow-Origin in our response:

`Access-Control-Allow-Origin` header on html5rocks.com

Enabling cross-origin requests is easy, so please, please, please enable CORS if your data is public!

Making a cross-domain request #

If the server endpoint has enabled CORS, making the cross-origin request is no different than a normal XMLHttpRequest request. For example, here is a request example.com can now make to www.example2.com:

var xhr = new XMLHttpRequest();
xhr.open('GET', 'http://www.example2.com/hello.json');
xhr.onload = function(e) {
  var data = JSON.parse(this.response);
  ...
}
xhr.send();

Practical examples #

Download + save files to the HTML5 file system #

Let's say you have an image gallery and want to fetch a bunch of images then save them locally using the HTML5 File System. One way to accomplish this would be to request images as Blobs and write them out using FileWriter:

window.requestFileSystem  = window.requestFileSystem || window.webkitRequestFileSystem;

function onError(e) {
  console.log('Error', e);
}

var xhr = new XMLHttpRequest();
xhr.open('GET', '/path/to/image.png', true);
xhr.responseType = 'blob';

xhr.onload = function(e) {

  window.requestFileSystem(TEMPORARY, 1024 * 1024, function(fs) {
    fs.root.getFile('image.png', {create: true}, function(fileEntry) {
      fileEntry.createWriter(function(writer) {

        writer.onwrite = function(e) { ... };
        writer.onerror = function(e) { ... };

        var blob = new Blob([xhr.response], {type: 'image/png'});

        writer.write(blob);

      }, onError);
    }, onError);
  }, onError);
};

xhr.send();

Slicing a file and uploading each portion #

Using the File APIs, we can minimize the work to upload a large file. The technique is to slice the upload into multiple chunks, spawn an XHR for each portion, and put the file together on the server. This is similar to how GMail uploads large attachments so quickly. Such a technique could also be used to get around Google App Engine's 32MB http request limit.

function upload(blobOrFile) {
  var xhr = new XMLHttpRequest();
  xhr.open('POST', '/server', true);
  xhr.onload = function(e) { ... };
  xhr.send(blobOrFile);
}

document.querySelector('input[type="file"]').addEventListener('change', function(e) {
  var blob = this.files[0];

  const BYTES_PER_CHUNK = 1024 * 1024; // 1MB chunk sizes.
  const SIZE = blob.size;

  var start = 0;
  var end = BYTES_PER_CHUNK;

  while(start < SIZE) {
    upload(blob.slice(start, end));

    start = end;
    end = start + BYTES_PER_CHUNK;
  }
}, false);

})();

What is not shown here is the code to reconstruct the file on the server.

References #

XMLHttpRequest Level 2 specification
Cross Origin Resource Sharing (CORS) specification
File API specification
FileSystem API specification

Mobifying your HTML5 site

2011-03-03T00:00:00Z

Introduction #

Developing for the mobile web is a hot topic these days. This year, for the first time ever, smart phones out sold PCs. More and more users are using a mobile device to traverse the web, which means it's becoming critical for developers to optimize their sites for the mobile browsers.

The "mobile" battlefield is still uncharted waters for a large number of developers. Many folks have existing legacy sites that neglect mobile users altogether. Instead, the site was designed primarily for desktop browsing and degrades poorly in mobile browsers. This site (html5rocks.com) is no exception. At launch, we put little effort into a mobile version of the site.

Creating a mobile-friendly html5rocks.com #

As an exercise I thought it would be interesting to take html5rocks (an existing HTML5 site) and augment it with a mobile-friendly version. I was mainly concerned with the minimum amount of work required to target smart phones. The goal of my exercise was not to create an entirely new mobile site and maintain two codebases. That would have taken forever and have been a huge waste of time. We had already defined the site's structure (markup). We had a look and feel (CSS). The core functionality (JS) was there. Point is, many sites are in this same boat.

This article examines how we created a mobile version of html5rocks optimized for Android and iOS devices. Just load html5rocks.com on a device that supports one of those OSs to see the difference. There are no redirects to a m.html5rocks.com or other tomfoolery of that nature. You get html5rocks as is… with the added benefit of something that looks great and works well on a mobile device.

html5rocks.com on desktop (left) and mobile (right)

CSS Media Queries #

HTML4 and CSS2 have supported media-dependent style sheets for some time. For example:

<link rel="stylesheet" media="print" href="printer.css">

would target print devices and provide specific styling for the page content when it is printed. CSS3 takes the idea of media types one step further and enhances their functionality with media queries. Media queries extend the usefulness of media types by allowing more precise labeling of style sheets. This enables the content's presentation to be customized to a specific range of output devices without having to change the content itself. Sounds perfect for an existing layout that needs modifying!

You can use media queries in the media attribute of your external stylesheets to target screen width, device width, orientation, etc. For the full list, see the W3C media query specification.

Targeting screen sizes #

In the following example, phone.css would apply to devices that the browser considers "handheld" or devices with screens <= 320px wide.

 <link rel='stylesheet'
  media='handheld, only screen and (max-device-width: 320px)' href='phone.css'>

Prefixing a media queries with the "only" keyword will cause non CSS3-compliant browsers to ignore the rule.

The following would target screen sizes between 641px and 800px:

 <link rel='stylesheet'
  media='only screen and (min-width: 641px) and (max-width: 800px)' href='ipad.css'>

Media queries can also appear within inline <style> tags. The following targets all media types when in a portrait orientation:

 <style>
  @media only all and (orientation: portrait) { ... }
 </style>

`media="handheld"` #

We need to stop for a minute and discuss media="handheld". Fact is, Android and iOS ignore media="handheld". The claim is that users will miss the high end content provided by stylesheets targeting media="screen" and developers are less likely to maintain a lower quality media="handheld" version. So, as part of their "full web" motto, most modern smart phone browsers simply ignore handheld style sheets.

It would be ideal to use this feature to target mobile devices, but various browsers have implemented it in different ways:

Some read only the handheld style sheet.
Some read only the handheld style sheet if there is one, but default to the screen style sheet otherwise.
Some read both the handheld style sheet and the screen style sheet.
Some read only the screen style sheet.

Opera Mini does not ignore media="handheld". The trick to getting Windows Mobile to recognize media="handheld" is to capitalize the media attribute value for the screen stylesheet:

 <!-- media="handheld" trick for Windows Mobile -->
 <link rel="stylesheet" href="screen.css" media="Screen">
 <link rel="stylesheet" href="mobile.css" media="handheld">

How html5rocks uses media queries #

Media queries are used heavily throughout mobile html5rocks. They allowed us tweak the layout without having to make significant changes to our Django template markup… a real lifesaver! In addition, their support across the various browsers is pretty good.

In the <head> of each page you'll see the following stylesheets:

 <link rel='stylesheet'
  media='all' href='/static/css/base.min.css' />
 <link rel='stylesheet'
  media='only screen and (max-width: 800px)' href='/static/css/mobile.min.css' />

base.css has always defined the main look and feel of html5rocks.com, but now we're applying new styles (mobile.css) for screen widths under 800px. Its media query covers smart phones (~320px) and the iPad (~768px). The effect: we're incrementally overriding styles in base.css (only as necessary) to make things look better in mobile.

Some of the styling changes that mobile.css enforces:

Reduces extra whitespace/padding across the site. Small screens means space is at a premium!
Removes :hover states. They'll never been seen on touch devices.
Adjusts the layout to be single column. More on this later.
Removes the box-shadow around the site's main container div. Large box shadows reduce page performance.
Used CSS flex box model box-ordinal-group to change the ordering of each section on the homepage. You'll notice the "LEARN BY MAJOR HTML5 FEATURE GROUPS" comes before the "TUTORIALS" section on the homepage but after it on the mobile version. This ordering made more sense for mobile and didn't require markup changes. CSS flexbox FTW!
Removes opacity changes. Changing alpha values is a performance hit on mobile.

Mobile meta tags #

Mobile WebKit supports a few goodies that give users a better browsing experience on certain devices.

Viewport settings #

The first meta setting (and one that you'll use most often) is the viewport property. Setting a viewport tells the browser how content should fit on the device's screen and informs the browser that the site is optimized for mobile. For example:

 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">

tells the browser to set the viewport to the width of the device with an initial scale of 1. This example also allows zooming, something that may be desirable for a web site but not a web app. We could prevent zooming with user-scalable=no or cap the scaling to a certain level:

 <meta name=viewport
  content="width=device-width, initial-scale=1.0, minimum-scale=0.5 maximum-scale=1.0">

Note: width can also take a pixel value. Setting width=320 would achieve the same result as width=device-width on the iPhone and a few other smart phones. Keep in mind not all phones have this exact width, so it's best to stick with device-width and let the browser figure out the rest.

Android extends the viewport meta tag by allowing developers to to specify which screen resolution the site has been developed for:

 <meta name="viewport" content="target-densitydpi=device-dpi">

Possible values for target-densitydpi are device-dpi, high-dpi, medium-dpi, low-dpi.

If you want to modify your web page for different screen densities, use the -webkit-device-pixel-ratio CSS media query and/or the window.devicePixelRatio property in JavaScript, then set the target-densitydpi meta property to device-dpi. This stops Android from performing scaling in your web page and allows you to make the necessary adjustments for each density, via CSS and JavaScript.

See Android's WebView documentation for more information on targeting device resolutions.

Full-screen browsing #

There are two other meta values that are iOS-sfic. apple-mobile-web-app-capable and apple-mobile-web-app-status-bar-style will render page content in app-like full screen mode and make the status bar translucent:

 <meta name="apple-mobile-web-app-capable" content="yes">
 <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">

For more information on all of the available meta options, see the Safari reference documentation.

Home screen icons #

iOS and Android devices also accept a rel="apple-touch-icon" (iOS) and rel="apple-touch-icon-precomposed" (Android) for links. These create a flashy app-like icon on the user's home screen when they bookmark your site:

 <link rel="apple-touch-icon"
      href="/static/images/identity/HTML5_Badge_64.png" />
 <link rel="apple-touch-icon-precomposed"
      href="/static/images/identity/HTML5_Badge_64.png" />

How html5rocks uses mobile meta tags #

Putting everything together, here is a snippet from the <head> section of html5rocks:

 <head>
  ...
   <meta name="viewport"
        content="width=device-width, initial-scale=1.0, minimum-scale=1.0" />

   <link rel="apple-touch-icon"
        href="/static/images/identity/HTML5_Badge_64.png" />
   <link rel="apple-touch-icon-precomposed"
        href="/static/images/identity/HTML5_Badge_64.png" />
  ...
 </head>

Vertical layout #

On smaller screens, it is far more convenient to scroll vertically than horizontally. Keeping content in a single-column, vertical layout is preferred for mobile. For html5rocks, we used CSS3 media queries to create such a layout. Again, without changing markup.

Single-column vertical layout throughout the site.

Mobile optimizations #

Most of the optimizations that we made are things that should have been done in the first place. Things like reducing the number of network requests, JS/CSS compression, gzipping (comes from free on App Engine), and minimizing DOM manipulations. These techniques are common best practices but are occasionally overlooked when rushing a site out the door.

Auto-hide address bar #

Mobile browsers lacks the screen real estate of its desktop counterpart. To make things worst, on different platforms, you sometimes end up with a big ol' address bar at the top of the screen… even after the page is done loading.

One easy way to deal with it is to scroll the page using JavaScript. Even doing so by one pixel will take care of the pesky address bar. To force-hide the address bar on html5rocks, I attached an onload event handler to the window object and scrolled the page vertically by one pixel:

Ugly address bar takes up screen real estate.

  // Hides mobile browser's address bar when page is done loading.
  window.addEventListener('load', function(e) {
    setTimeout(function() { window.scrollTo(0, 1); }, 1);
  }, false);

We also wrapped this listener is our is_mobile template variable since it's not needed on the desktop.

Reduce network requests, save bandwidth #

It's a known fact that reducing the number of HTTP requests can greatly improve performance. Mobile devices further limit the number of concurrent connections the browser can make, so mobile sites will benefit even more from reducing these extraneous requests. Furthermore, shaving off every byte is critical because bandwidth is often limited on phones. You may be costing users money!

The following are some of the approaches we took to minimize network requests and reduce bandwidth on html5rocks:

Remove iframes - iframes are slow! A large amount of our latency came from 3rd party sharing widgets (Buzz, Google Friend Connect, Twitter, Facebook) on tutorial pages. These APIs were included via <script> tags and create iframes that diminish the speed of the page. The widgets were removed for mobile.
display:none - In certain cases, we were hiding markup if it didn't fit the mobile profile. A good example were the four rounded boxes at the top of the homepage:

Box buttons on homepage.

They're missing from the mobile site. It's important to remember that the browser still makes a request for each icon, despite their container being hidden with display:none. Therefore, it wasn't sufficient to simply hide these buttons. Not only would that be wasting bandwidth, but the user wouldn't even see the fruits of that wasted bandwidth! The solution was to create a "is_mobile" boolean in our Django template to conditionally omit sections of HTML. When the user is viewing the site on a smart device, the buttons are left out.

Application Cache - Not only does this give us offline support, but it also creates a faster startup.
CSS/JS compression - we're using YUI compressor instead of Closure compiler mainly because it handles both CSS and JS. One issue that we ran into was that inline media queries (media queries that appear inside a stylesheet) barfed in YUI compressor 2.4.2 (see this issue). Using YUI Compressor 2.4.4+ fixed the problem.
Used of CSS image sprites where possible.
Used pngcrush for image compression.
Used dataURIs for small images. Base64 encoding adds ~30%+ of size to the image but saves the network request.
Auto loaded Google Custom Search using a single script tag rather than dynamically loading it with google.load(). The latter makes an extra request.

<script src="//www.google.com/jsapi?autoload={"modules":[{"name":"search","version":"1"}]}"> </script>

Our code pretty printer and Modernizr were being included on every page, even if they were never used. Modernizr is great, but it runs a bunch of tests on every load. Some of those tests make costly modifications to the DOM and slows down page load. Now, we're only including these libraries on pages where they're actually needed. -2 requests :)

Additional performance tweaks:

Moved all JS to the bottom of the page (where possible).
Removed inline <style> tags.
Cached DOM lookups and minimized DOM manipulations - Every time you touch the DOM the browser performs a reflow. Reflows are even more costly on a mobile device.

Offloaded wasteful client-side code to the server. Specifically, the check to see set the navigation styling of the current page:

var lis = document.querySelectorAll('header nav li');
var i = lis.length;
while (i--) {
var a = lis[i].querySelector('a');
var section = a.getAttribute("data-section");
if (new RegExp(section).test(document.location.href)) {
    a.className = 'current';
}
}

Elements with fixed widths were replaced with fluid width:100% or width:auto.

Application Cache #

The mobile version of html5rocks uses Application Cache to speed up initial load and allows users to read content offline.

When implementing AppCache on your site, it's very important that you don't cache your manifest file (either explicitly in the manifest file itself or implicitly with heavy cache control headers). If your manifest gets cached by the browser it's a nightmare to debug. iOS and Android do a particularly good job of caching this file but don't provide a convenient way to flush the cache like the desktop browsers do.

To prevent said caching for our site, we first set App Engine to never cache manifest files:

- url: /(.*\.(appcache|manifest))
  static_files: \1
  mime_type: text/cache-manifest
  upload: (.*\.(appcache|manifest))
  expiration: "0s"

Second, we used the JS API to inform the user when a new manifest has finished downloading. They're prompted to refresh the page:

window.applicationCache.addEventListener('updateready', function(e) {
  if (window.applicationCache.status == window.applicationCache.UPDATEREADY) {
    window.applicationCache.swapCache();
    if (confirm('A new version of this site is available. Load it?')) {
      window.location.reload();
    }
  }
}, false);

To save network traffic keep your manifest simple. That is to say, don't call out every page on your site. Just list the important images, CSS, and JavaScript files. The last thing you want to do is force the mobile browser to download a large number of assets on every appcache update. Instead, remember that the browser will implicitly cache an html page when the user visits (and it includes a <html manifest="..."> attribute).

Stream Updates with Server-Sent Events

2010-11-30T00:00:00Z

Introduction #

I wouldn't be surprised if you've stumbled on this article wondering, "What the heck are Server-Sent Events (SSEs)?" Many people have never heard of them, and rightfully so. Over the years, the specification has seen significant changes, and the API has taken somewhat of a backseat to newer, sexier communication protocols such as the WebSocket API. The idea behind SSEs may be familiar: a web app "subscribes" to a stream of updates generated by a server and, whenever a new event occurs, a notification is sent to the client. But to really understand Server-Sent Events, we need to understand the limitations of its AJAX predecessors, which includes:

Polling is a traditional technique used by the vast majority of AJAX applications. The basic idea is that the application repeatedly polls a server for data. If you're familiar with the HTTP protocol, you know that fetching data revolves around a request/response format. The client makes a request and waits for the server to respond with data. If none is available, an empty response is returned. So what's the big deal with polling? Extra polling creates greater HTTP overhead.

Long polling (Hanging GET / COMET) is a slight variation on polling. In long polling, if the server does not have data available, the server holds the request open until new data is made available. Hence, this technique is often referred to as a "Hanging GET". When information becomes available, the server responds, closes the connection, and the process is repeated. The effect is that the server is constantly responding with new data as it becomes available. The shortcoming is that the implementation of such a procedure typically involves hacks such as appending script tags to an 'infinite' iframe. We can do better than hacks!

Server-Sent Events on the other hand, have been designed from the ground up to be efficient. When communicating using SSEs, a server can push data to your app whenever it wants, without the need to make an initial request. In other words, updates can be streamed from server to client as they happen. SSEs open a single unidirectional channel between server and client.

The main difference between Server-Sent Events and long-polling is that SSEs are handled directly by the browser and the user simply has to listen for messages.

Server-Sent Events vs. WebSockets #

Why would you choose Server-Sent Events over WebSockets? Good question.

One reason SSEs have been kept in the shadow is because later APIs like WebSockets provide a richer protocol to perform bi-directional, full-duplex communication. Having a two-way channel is more attractive for things like games, messaging apps, and for cases where you need near real-time updates in both directions. However, in some scenarios data doesn't need to be sent from the client. You simply need updates from some server action. A few examples would be friends' status updates, stock tickers, news feeds, or other automated data push mechanisms (e.g. updating a client-side Web SQL Database or IndexedDB object store). If you'll need to send data to a server, XMLHttpRequest is always a friend.

SSEs are sent over traditional HTTP. That means they do not require a special protocol or server implementation to get working. WebSockets on the other hand, require full-duplex connections and new Web Socket servers to handle the protocol. In addition, Server-Sent Events have a variety of features that WebSockets lack by design such as automatic reconnection, event IDs, and the ability to send arbitrary events.

JavaScript API #

To subscribe to an event stream, create an EventSource object and pass it the URL of your stream:

if (!!window.EventSource) {
var source = new EventSource('stream.php');
} else {
// Result to xhr polling :(
}

Next, set up a handler for the message event. You can optionally listen for open and error:

source.addEventListener('message', function(e) {
console.log(e.data);
}, false);

source.addEventListener('open', function(e) {
// Connection was opened.
}, false);

source.addEventListener('error', function(e) {
if (e.readyState == EventSource.CLOSED) {
// Connection was closed.
}
}, false);

When updates are pushed from the server, the onmessage handler fires and new data is be available in its e.data property. The magical part is that whenever the connection is closed, the browser will automatically reconnect to the source after ~3 seconds. Your server implementation can even have control over this reconnection timeout. See Controlling the reconnection-timeout in the next section.

That's it. Your client is now ready to process events from stream.php.

Event stream format #

Sending an event stream from the source is a matter of constructing a plaintext response, served with a text/event-stream Content-Type, that follows the SSE format. In its basic form, the response should contain a data: line, followed by your message, followed by two "\n" characters to end the stream:

data: My message\n\n

Multiline data #

If your message is longer, you can break it up by using multiple data: lines. Two or more consecutive lines beginning with data: will be treated as a single piece of data, meaning only one message event will be fired. Each line should end in a single "\n" (except for the last, which should end with two). The result passed to your message handler is a single string concatenated by newline characters. For example:

data: first line\n
data: second line\n\n</pre>

will produce "first line\nsecond line" in e.data. One could then use e.data.split('\n').join('') to reconstruct the message sans "\n" characters.

Send JSON data #

Using multiple lines makes it easy to send JSON without breaking syntax:

data: {\n
data: "msg": "hello world",\n
data: "id": 12345\n
data: }\n\n

and possible client-side code to handle that stream:

source.addEventListener('message', function(e) {
var data = JSON.parse(e.data);
console.log(data.id, data.msg);
}, false);

Associating an ID with an event #

You can send a unique id with an stream event by including a line starting with id::

id: 12345\n
data: GOOG\n
data: 556\n\n

Setting an ID lets the browser keep track of the last event fired so that if, the connection to the server is dropped, a special HTTP header (Last-Event-ID) is set with the new request. This lets the browser determine which event is appropriate to fire. The message event contains a e.lastEventId property.

Controlling the reconnection-timeout #

The browser attempts to reconnect to the source roughly 3 seconds after each connection is closed. You can change that timeout by including a line beginning with retry:, followed by the number of milliseconds to wait before trying to reconnect.

The following example attempts a reconnect after 10 seconds:

retry: 10000\n
data: hello world\n\n

Specifying an event name #

A single event source can generate different types events by including an event name. If a line beginning with event: is present, followed by a unique name for the event, the event is associated with that name. On the client, an event listener can be setup to listen to that particular event.

For example, the following server output sends three types of events, a generic 'message' event, 'userlogon', and 'update' event:

data: {"msg": "First message"}\n\n
event: userlogon\n
data: {"username": "John123"}\n\n
event: update\n
data: {"username": "John123", "emotion": "happy"}\n\n

With event listeners setup on the client:

source.addEventListener('message', function(e) {
var data = JSON.parse(e.data);
console.log(data.msg);
}, false);

source.addEventListener('userlogon', function(e) {
var data = JSON.parse(e.data);
console.log('User login:' + data.username);
}, false);

source.addEventListener('update', function(e) {
var data = JSON.parse(e.data);
console.log(data.username + ' is now ' + data.emotion);
}, false);

Server examples #

A basic server implementation in PHP:

<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache'); // recommended to prevent caching of event data.

/**
* Constructs the SSE data format and flushes that data to the client.
*
* @param string $id Timestamp/id of this connection.
* @param string $msg Line of text that should be transmitted.
**/

function sendMsg($id, $msg) {
  echo "id: $id" . PHP_EOL;
  echo "data: $msg" . PHP_EOL;
  echo PHP_EOL;
  ob_flush();
  flush();
}

$serverTime = time();

sendMsg($serverTime, 'server time: ' . date("h:i:s", time()));
?>

Here's a similiar implementation using Node JS:

var http = require('http');
var sys = require('sys');
var fs = require('fs');

http.createServer(function(req, res) {
//debugHeaders(req);

if (req.headers.accept && req.headers.accept == 'text/event-stream') {
if (req.url == '/events') {
    sendSSE(req, res);
} else {
    res.writeHead(404);
    res.end();
}
} else {
res.writeHead(200, {'Content-Type': 'text/html'});
res.write(fs.readFileSync(__dirname + '/sse-node.html'));
res.end();
}
}).listen(8000);

function sendSSE(req, res) {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive'
});

var id = (new Date()).toLocaleTimeString();

// Sends a SSE every 5 seconds on a single connection.
setInterval(function() {
constructSSE(res, id, (new Date()).toLocaleTimeString());
}, 5000);

constructSSE(res, id, (new Date()).toLocaleTimeString());
}

function constructSSE(res, id, data) {
res.write('id: ' + id + '\n');
res.write("data: " + data + '\n\n');
}

function debugHeaders(req) {
sys.puts('URL: ' + req.url);
for (var key in req.headers) {
sys.puts(key + ': ' + req.headers[key]);
}
sys.puts('\n\n');
}

sse-node.html #

<!DOCTYPE html>
<html>
    <head>
        <meta charset=&quot;utf-8&quot; />
    </head>
    <body>
        <script>
        var source = new EventSource('/events');
        source.onmessage = function(e) {
            document.body.innerHTML += e.data + '<br>';
        };
        </script>
    </body>
</html>

Cancel an event stream #

Normally, the browser auto-reconnects to the event source when the connection is closed, but that behavior can be canceled from either the client or server.

To cancel a stream from the client, simply call:

source.close();

To cancel a stream from the server, respond with a non text/event-stream Content-Type or return an HTTP status other than 200 OK (e.g. 404 Not Found).

Both methods will prevent the browser from re-establishing the connection.

A word on security #

From the WHATWG's section on Cross-document messaging security:

So, as an extra level of precaution, be sure to verify e.origin in your message handler matches your app's origin:

source.addEventListener('message', function(e) {
if (e.origin != 'http://example.com') {
alert('Origin was not http://example.com');
return;
}
...
}, false);

Another good idea is to check the integrity of the data you receive.

Even after checking the origin attribute, authors should also check that the data in question is of the expected format.

References #

Using the HTML5 Drag and Drop API

2010-09-30T00:00:00Z

The HTML5 Drag and Drop API means that we can make almost any element on our page draggable. In this post we'll explain the basics of drag and drop.

Creating draggable content #

It's worth noting that in most browsers, text selections, images, and links are draggable by default. For example, if you drag a link on a web page you will see a small box with a title and URL. You can drop this box on the address bar or the desktop to create a shortcut or to navigate to the link. To make other types of content draggable you need to use the HTML5 Drag and Drop APIs.

To make an object draggable set draggable=true on that element. Just about anything can be drag-enabled: images, files, links, files, or any markup on your page.

Our example creates an interface to rearrange columns that have been laid out with CSS Grid. The basic markup for the columns looks like this, with each column having the draggable attribute set to true.

<div class="container">
  <div draggable="true" class="box">A</div>
  <div draggable="true" class="box">B</div>
  <div draggable="true" class="box">C</div>
</div>

Here's the CSS for the container and box elements. Note that the only CSS related to drag and drop functionality is the cursor: move property. The rest of the code just controls the layout and styling of the container and box elements.

.container {
  display: grid;
  grid-template-columns: repeat(5, 1fr);
  gap: 10px;
}

.box {
  border: 3px solid #666;
  background-color: #ddd;
  border-radius: .5em;
  padding: 10px;
  cursor: move;
}

At this point you will find that you can drag the items, however nothing else will happen. To add the drag and drop functionality we need to use the JavaScript API.

Listening for dragging events #

There are a number of different events to attach to for monitoring the entire drag and drop process.

To handle the drag and drop flow, you need some kind of source element (where the drag originates), the data payload (what you're trying to drop), and a target (an area to catch the drop). The source element can be an image, list, link, file object, block of HTML, etc. The target is the drop zone (or set of drop zones) that accepts the data the user is trying to drop. Keep in mind that not all elements can be targets, for example an image can't be a target.

Starting and ending a drag and drop sequence #

Once you have draggable="true" attributes defined on your content, attach a dragstart event handler to kick off the drag and drop sequence for each column.

This code will set the column's opacity to 40% when the user begins dragging it, then return it to 100% when the dragging event ends.

function handleDragStart(e) {
  this.style.opacity = '0.4';
}

function handleDragEnd(e) {
  this.style.opacity = '1';
}

let items = document.querySelectorAll('.container .box');
items.forEach(function (item) {
  item.addEventListener('dragstart', handleDragStart);
  item.addEventListener('dragend', handleDragEnd);
});

The result can be seen in the Glitch demo below. Drag an item and it becomes opaque. As the dragstart event's target is the source element, setting this.style.opacity to 40% gives the user visual feedback that the element is the current selection being moved. Once you drop the item, although the drop functionality is not in place, the source element returns to 100% opacity.

Add additional visual cues with `dragenter`, `dragover`, and `dragleave` #

To help the user understand how to interact with your interface, use the dragenter, dragover and dragleave event handlers. In this example the columns are drop targets in addition to being draggable. Help the user to understand this by making the border dashed as they hold a dragged item over a column. For example, in your CSS you might create an over class to represent elements that are drop targets:

.box.over {
  border: 3px dotted #666;
}

Then, in your JavaScript set up the event handlers, add the over class when the column is dragged over, and remove it when we leave. In the dragend handler we also make sure to remove the classes at the end of the drag.

document.addEventListener('DOMContentLoaded', (event) => {

  function handleDragStart(e) {
    this.style.opacity = '0.4';
  }

  function handleDragEnd(e) {
    this.style.opacity = '1';

    items.forEach(function (item) {
      item.classList.remove('over');
    });
  }

  function handleDragOver(e) {
    e.preventDefault();
    return false;
  }

  function handleDragEnter(e) {
    this.classList.add('over');
  }

  function handleDragLeave(e) {
    this.classList.remove('over');
  }

  let items = document.querySelectorAll('.container .box');
  items.forEach(function(item) {
    item.addEventListener('dragstart', handleDragStart);
    item.addEventListener('dragover', handleDragOver);
    item.addEventListener('dragenter', handleDragEnter);
    item.addEventListener('dragleave', handleDragLeave);
    item.addEventListener('dragend', handleDragEnd);
    item.addEventListener('drop', handleDrop);
  });
});

There are a couple of points worth covering in this code:

The default action for dragover event is to set the dataTransfer.dropEffect property to "none". The dropEffect property is covered later in this article. For now, just know that it prevents the drop event from being fired. To override this behavior, call e.preventDefault(). Another good practice is to return false in that same handler.
The dragenter event handler is used to toggle the over class instead of dragover. If you use dragover, the CSS class would be toggled many times as the event dragover continued to fire on a column hover. Ultimately, that would cause the browser's renderer to do a large amount of unnecessary work. Keeping redraws to a minimum is always a good idea. If you need to use the dragover event for something, consider throttling or debouncing your event listener.

Completing the drop #

To process the actual drop, add an event listener for the drop event. In the drop handler, you'll need to prevent the browser's default behavior for drops, which is typically some sort of annoying redirect. You can prevent the event from bubbling up the DOM by calling e.stopPropagation().

function handleDrop(e) {
  e.stopPropagation(); // stops the browser from redirecting.
  return false;
}

Be sure to register the new handler in amongst the other handlers:

  let items = document.querySelectorAll('.container .box');
  items.forEach(function(item) {
    item.addEventListener('dragstart', handleDragStart);
    item.addEventListener('dragover', handleDragOver);
    item.addEventListener('dragenter', handleDragEnter);
    item.addEventListener('dragleave', handleDragLeave);
    item.addEventListener('dragend', handleDragEnd);
    item.addEventListener('drop', handleDrop);
  });

If you run the code at this point, the item will not drop to the new location. To achieve this you need to use the DataTransfer object.

The dataTransfer property is where all the drag and drop magic happens. It holds the piece of data sent in a drag action. dataTransfer is set in the dragstart event and read/handled in the drop event. Calling e.dataTransfer.setData(mimeType, dataPayload) lets you set the object's MIME type and data payload.

In this example, we're going to allow users to rearrange the order of the columns. To do that, first you need to store the source element's HTML when the drag starts:

function handleDragStart(e) {
  this.style.opacity = '0.4';

  dragSrcEl = this;

  e.dataTransfer.effectAllowed = 'move';
  e.dataTransfer.setData('text/html', this.innerHTML);
}

In the drop event you process the column drop, setting the source column's HTML to the HTML of the target column that you dropped on, first checking that the user is not dropping back onto the same column they dragged from.

function handleDrop(e) {
  e.stopPropagation();

  if (dragSrcEl !== this) {
    dragSrcEl.innerHTML = this.innerHTML;
    this.innerHTML = e.dataTransfer.getData('text/html');
  }

  return false;
}

You can see the result in the following demo. For this to work, you'll need a desktop browser. The Drag and Drop API isn't supported on mobile. Drag and release the A column on top of the B column and notice how they change places:

More dragging properties #

The dataTransfer object exposes properties to provide visual feedback to the user during the drag process. These properties can also be used to control how each drop target responds to a particular data type.

dataTransfer.effectAllowed restricts what 'type of drag' the user can perform on the element. It is used in the drag-and-drop processing model to initialize the dropEffect during the dragenter and dragover events. The property can be set to the following values: none, copy, copyLink, copyMove, link, linkMove, move, all, and uninitialized.
dataTransfer.dropEffect controls the feedback that the user is given during the dragenter and dragover events. When the user hovers over a target element, the browser's cursor indicates what type of operation is going to take place (e.g. a copy, a move, etc.). The effect can take one of the following values: none, copy, link, move.
e.dataTransfer.setDragImage(imgElement, x, y) means that instead of using the browser's default 'ghost image' feedback, you can optionally set a drag icon.

File upload with drag and drop #

This simple example uses a column as both the drag source and drag target. This might be seen in a UI where the user is asked to rearrange the items. In some situations the drag target and source may be different, such as an interface where the user needs to select one image to be the main image for a product by dragging the selected image onto a target.

Drag and Drop is frequently used to allow users to drag items from their desktop into an application. The main difference is in your drop handler. Instead of using dataTransfer.getData() to access the files, their data will be contained in the dataTransfer.files property:

function handleDrop(e) {
  e.stopPropagation(); // Stops some browsers from redirecting.
  e.preventDefault();

  var files = e.dataTransfer.files;
  for (var i = 0, f; (f = files[i]); i++) {
    // Read the File objects in this FileList.
  }
}

You can find more information about this in Custom drag-and-drop.

More resources #

A beginner's guide to using the application cache

2010-06-18T00:00:00Z

Introduction #

It's becoming increasingly important for web-based applications to be accessible offline. Yes, all browsers can cache pages and resources for long periods if told to do so, but the browser can kick individual items out of the cache at any point to make room for other things. HTML5 addresses some of the annoyances of being offline with the ApplicationCache interface. Using the cache interface gives your application three advantages:

Offline browsing - users can navigate your full site when they're offline
Speed - resources come straight from disk, no trip to the network.
Resilience - if your site goes down for "maintenance" (as in, someone accidentally breaks everything), your users will get the offline experience

The Application Cache (or AppCache) allows a developer to specify which files the browser should cache and make available to offline users. Your app will load and work correctly, even if the user presses the refresh button while they're offline.

The cache manifest file #

The cache manifest file is a simple text file that lists the resources the browser should cache for offline access.

Referencing a manifest file #

To enable the application cache for an app, include the manifest attribute on the document's html tag:

<html manifest="example.appcache">
  ...
</html>

The manifest attribute should be included on every page of your web application that you want cached. The browser does not cache a page if it does not contain the manifest attribute (unless it is explicitly listed in the manifest file itself. This means that any page the user navigates to that includes a manifest will be implicitly added to the application cache. Thus, there's no need to list every page in your manifest. If a page points to a manifest, there's no way to prevent this page being cached.

You can see the urls that are controlled by the application cache by visiting about://://appcache-internals/ in Chrome. From here you can clear caches and view the entries. There are similar developer tools in Firefox.

The manifest attribute can point to an absolute URL or relative path, but an absolute URL must be under the same origin as the web application. A manifest file can have any file extension, but needs to be served with the correct mime-type (see below).

<html manifest="http://www.example.com/example.mf">
  ...
</html>

A manifest file must be served with the mime-type text/cache-manifest. You may need to add a custom file type to your web server or .htaccess configuration.

For example, to serve this mime-type in Apache, add this line to your config file:

AddType text/cache-manifest .appcache

Or, in your app.yaml file in Google App Engine:

- url: /mystaticdir/(.*\.appcache)
  static_files: mystaticdir/\1
  mime_type: text/cache-manifest
  upload: mystaticdir/(.*\.appcache)

This requirement was dropped from the specification some time ago, and no longer required by the latest versions of Chrome, Safari and Firefox, but you'll need the mime-type to work in older browsers and IE11.

Structure of a manifest file #

The manifest is a separate file you link to via the manifest attribute on the html element. A simple manifest looks something like this:

CACHE MANIFEST
index.html
stylesheet.css
images/logo.png
scripts/main.js
http://cdn.example.com/scripts/main.js

This example will cache four files on the page that specifies this manifest file.

There are a couple of things to note:

The CACHE MANIFEST string is the first line and is required.
Files can be from another domain
Some browsers place restrictions on the amount of storage quota available to your app. In Chrome for example, AppCache uses a shared pool of TEMPORARY storage that other offline APIs can share. If you are writing an app for the Chrome Web Store, using the unlimitedStorage removes that restriction.
If the manifest itself returns a 404 or 410, the cache is deleted.
If the manifest or a resource specified in it fails to download, the entire cache update process fails. The browser will keep using the old application cache in the event of failure.

Lets take a look at a more complex example:

CACHE MANIFEST
# 2010-06-18:v2

# Explicitly cached 'master entries'.
CACHE:
/favicon.ico
index.html
stylesheet.css
images/logo.png
scripts/main.js

# Resources that require the user to be online.
NETWORK:
*

# static.html will be served if main.py is inaccessible
# offline.jpg will be served in place of all images in images/large/
# offline.html will be served in place of all other .html files
FALLBACK:
/main.py /static.html
images/large/ images/offline.jpg

Lines starting with a '#' are comment lines, but can also serve another purpose. An application's cache is only updated when its manifest file changes. So for example, if you edit an image resource or change a JavaScript function, those changes will not be re-cached. You must modify the manifest file itself to inform the browser to refresh cached files.

Avoid using a continually-updating timestamp or random string to force updates every time. The manifest is checked twice during an update, once at the start and once after all cached files have been updated. If the manifest has changed during the update, it's possible the browser fetched some files from one version, and other files from another version, so it doesn't apply the cache and retries later.

Although the cache updates, the browser won't use those files until the page is refreshed, because updates happen after the page is loaded from the current version of the cache.

A manifest can have three distinct sections: CACHE, NETWORK, and FALLBACK.

CACHE:: This is the default section for entries. Files listed under this header (or immediately after the CACHE MANIFEST) will be explicitly cached after they're downloaded for the first time. NETWORK:; Files listed in this section may come from the network if they aren't in the cache, otherwise the network isn't used, even if the user is online. You can white-list specific URLs here, or simply "", which allows all URLs. Most sites need "". FALLBACK:; An optional section specifying fallback pages if a resource is inaccessible. The first URI is the resource, the second is the fallback used if the network request fails or errors. Both URIs must from the same origin as the manifest file. You can capture specific URLs but also URL prefixes. "images/large/" will capture failures from URLs such as "images/large/whatever/img.jpg".

The following manifest defines a "catch-all" page (offline.html) that will be displayed when the user tries to access the root of the site while offline. It also declares that all other resources (e.g. those on remote a site) require an internet connection.

CACHE MANIFEST
# 2010-06-18:v3

# Explicitly cached entries
index.html
css/style.css

# offline.html will be displayed if the user is offline
FALLBACK:
/ /offline.html

# All other resources (e.g. sites) require the user to be online.
NETWORK:
*

# Additional resources to cache
CACHE:
images/logo1.png
images/logo2.png
images/logo3.png

Updating the cache #

Once an application is offline it remains cached until one of the following happens:

The user clears their browser's data storage for your site.
The manifest file is modified. Note: updating a file listed in the manifest doesn't mean the browser will re-cache that resource. The manifest file itself must be altered.

Cache status #

The window.applicationCache object is your programmatic access the browser's app cache. Its status property is useful for checking the current state of the cache:

var appCache = window.applicationCache;

switch (appCache.status) {
case appCache.UNCACHED: // UNCACHED == 0
return 'UNCACHED';
break;
case appCache.IDLE: // IDLE == 1
return 'IDLE';
break;
case appCache.CHECKING: // CHECKING == 2
return 'CHECKING';
break;
case appCache.DOWNLOADING: // DOWNLOADING == 3
return 'DOWNLOADING';
break;
case appCache.UPDATEREADY:  // UPDATEREADY == 4
return 'UPDATEREADY';
break;
case appCache.OBSOLETE: // OBSOLETE == 5
return 'OBSOLETE';
break;
default:
return 'UKNOWN CACHE STATUS';
break;
};

To programmatically check for updates to the manifest, first call applicationCache.update(). This will attempt to update the user's cache (which requires the manifest file to have changed). Finally, when the applicationCache.status is in its UPDATEREADY state, calling applicationCache.swapCache() will swap the old cache for the new one.

var appCache = window.applicationCache;

appCache.update(); // Attempt to update the user's cache.

...

if (appCache.status == window.applicationCache.UPDATEREADY) {
appCache.swapCache();  // The fetch was successful, swap in the new cache.
}

The good news: you can automate this. To update users to the newest version of your site, set a listener to monitor the updateready event on page load:

// Check if a new cache is available on page load.
window.addEventListener('load', function(e) {

window.applicationCache.addEventListener('updateready', function(e) {
if (window.applicationCache.status == window.applicationCache.UPDATEREADY) {
    // Browser downloaded a new app cache.
    if (confirm('A new version of this site is available. Load it?')) {
    window.location.reload();
    }
} else {
    // Manifest didn't changed. Nothing new to server.
}
}, false);

}, false);

AppCache events #

As you may expect, additional events are exposed to monitor the cache's state. The browser fires events for things like download progress, updating the app cache, and error conditions. The following snippet sets up event listeners for each type of cache event:

function handleCacheEvent(e) {
//...
}

function handleCacheError(e) {
alert('Error: Cache failed to update!');
};

// Fired after the first cache of the manifest.
appCache.addEventListener('cached', handleCacheEvent, false);

// Checking for an update. Always the first event fired in the sequence.
appCache.addEventListener('checking', handleCacheEvent, false);

// An update was found. The browser is fetching resources.
appCache.addEventListener('downloading', handleCacheEvent, false);

// The manifest returns 404 or 410, the download failed,
// or the manifest changed while the download was in progress.
appCache.addEventListener('error', handleCacheError, false);

// Fired after the first download of the manifest.
appCache.addEventListener('noupdate', handleCacheEvent, false);

// Fired if the manifest file returns a 404 or 410.
// This results in the application cache being deleted.
appCache.addEventListener('obsolete', handleCacheEvent, false);

// Fired for each resource listed in the manifest as it is being fetched.
appCache.addEventListener('progress', handleCacheEvent, false);

// Fired when the manifest resources have been newly redownloaded.
appCache.addEventListener('updateready', handleCacheEvent, false);

If the manifest file or a resource specified in it fails to download, the entire update fails. The browser will continue to use the old application cache in the event of such a failure.

References #

ApplicationCache API specification
Application Cache is a douchebag - covering the gotchas and issues with AppCache.

Eric Bidelman on web.dev

Network Error Logging (NEL)

Introduction #

Overview of the legacy Reporting API #

The legacy Report-To Header #

Explanation of header fields #

How does the browser send reports? #

When does the browser send reports? #

Configuring multiple endpoints #

Failover and load balancing #

Setting up Network Error Logging #

Setup #

Subresources #

Debugging report configurations #

What about ReportingObserver? #

Example server #

Further reading #

Custom Elements v1 - Reusable Web Components

Introduction #

Defining a new element #

Defining an element's JavaScript API #

Rules on creating custom elements #

Custom element reactions #

Properties and attributes #

Reflecting properties to attributes #

Observing changes to attributes #

Element upgrades #

Progressively enhanced HTML #

Element-defined content #

Creating an element that uses Shadow DOM #

Creating elements from a <template> #

Styling a custom element #

Pre-styling unregistered elements #

Extending elements #

Extending a custom element #

Extending native HTML elements #

Misc details #

Unknown elements vs. undefined custom elements #

API reference #

History and browser support #

Browser support #

Polyfill #

Conclusion #

Shadow DOM v1 - Self-Contained Web Components

Summary #

Introduction #

fancy-tabs demo #

What is shadow DOM? #

Background on DOM #

DOM… in the shadows #

Creating shadow DOM #

Creating shadow DOM for a custom element #

Composition and slots #

Terminology: light DOM vs. shadow DOM #

The <slot> element #

Styling #

Component-defined styles #

Styling based on context #

Styling distributed nodes #

Styling a component from the outside #

Creating style hooks using CSS custom properties #

Advanced topics #

Creating closed shadow roots (should avoid) #

Working with slots in JS #

slotchange event #

What elements are being rendering in a slot? #

What slot is an element assigned to? #

The Shadow DOM event model #

Using custom events #

Handling focus #

Tips & Tricks #

Use CSS containment #

Resetting inheritable styles #

Finding all the custom elements used by a page #

Creating elements from a <template> #

History & browser support #

Browser support #

Polyfill #

Conclusion #

Further reading #

The legacy `Report-To` Header #

Creating elements from a `<template>` #

`fancy-tabs` demo #