John McCutchan on web.dev

Use forensics and detective work to solve JavaScript performance mysteries

2013-06-13T00:00:00Z

Introduction #

In recent years, web applications have been sped up considerably. Many applications now run fast enough that I've heard some developers wonder aloud "is the web fast enough?". For some applications it may be, but, for the developers working on high performance applications, we know it is not fast enough. Despite the amazing advances in JavaScript virtual machine technology, a recent study showed that Google applications spend between 50% and 70% of their time inside V8. Your application has a finite amount of time, shaving cycles off of one system means another system can do more. Remember, applications running at 60fps only have 16ms per frame or else - jank. Read on, to learn about optimizing JavaScript and profile JavaScript applications, in a from the trenches story of the performance detectives on the V8 team tracking down an obscure performance problem in Find Your Way to Oz.

Google I/O 2013 Session #

I presented this material at Google I/O 2013. Check out the video below:

Why does performance matter? #

CPU cycles are a zero sum game. Making one part of your system use fewer allows you to use more in another or run smoother overall. Running faster and doing more are often competing goals. Users demand new features while also expecting your application to run smoother. JavaScript virtual machines keep getting faster but that is not a reason for ignoring performance problems that you can fix today, as the many developers, dealing with performance problems in their web applications already know. In real-time, high frame rate, applications the pressure to be jank free is paramount. Insomniac Games produced a study which showed that a solid, sustained frame rate is important to the success of a game: "A solid frame-rate is still a sign of professional, well-made product." Web developers take note.

Solving Performance Problems #

Solving a performance problem is like solving a crime. You need to carefully examine the evidence, check suspected causes, and experiment with different solutions. All along the way you must document your measurements so that you can be sure you've actually fixed the problem. There is very little difference between this method and how criminal detectives crack a case. Detectives examine evidence, interrogate suspects, and run experiments hoping to find the smoking gun.

V8 CSI: Oz #

The amazing wizards building Find Your Way to Oz approached the V8 team with a performance problem they couldn't solve on their own. Occasionally Oz would freeze, causing jank. The Oz developers had done some initial investigation using the Timeline Panel in Chrome DevTools. Looking at memory usage they encountered the dreaded saw tooth graph. Once per second the garbage collector was collecting 10MB of garbage and the garbage collection pauses corresponded with the jank. Similar to the following screenshot from the Timeline in Chrome Devtools:

The V8 detectives, Jakob and Yang took up the case. What took place was a long back and forth between Jakob and Yang from the V8 team and the Oz team. I've distilled this conversation down to the important events that helped track down this problem.

Evidence #

The first step is to collect and study the initial evidence.

What type of application are we looking at? #

The Oz demo is an interactive 3D application. Because of this, it is very sensitive to pauses caused by garbage collections. Remember, an interactive application running at 60fps has 16ms to do all JavaScript work and must leave some of that time for Chrome to process the graphics calls and draw the screen.

Oz performs a lot of arithmetic computation on double values and makes frequent calls to WebAudio and WebGL.

What kind of performance problem are we seeing? #

We are seeing pauses aka frame drops aka jank. These pauses correlate with garbage collection runs.

Are the developers following best practices? #

Yes, the Oz developers are well versed in JavaScript VM performance and optimization techniques. It's worth noting that the Oz developers were using CoffeeScript as their source language and producing JavaScript code via the CoffeeScript compiler. This made some of the investigation trickier because of the disconnect between the code being written by the Oz developers and the code being consumed by V8. Chrome DevTools now supports source maps which would have made this easier.

Why does the garbage collector run? #

Memory in JavaScript is automatically managed for the developer by the VM. V8 uses a common garbage collection system where memory is divided into two (or more) generations. The young generation holds objects that have recently been allocated. If an object survives long enough it is moved to the old generation.

The young generation is collected at a much higher frequency than the old generation is collected. This is by design, as young generation collection is much cheaper. It is often safe to assume that frequent GC pauses are caused by young generation collection.

In V8 the young memory space is divided into two equally sized contiguous blocks of memory. Only one of these two blocks of memory is in use at any given time and it is called the to space. While there is remaining memory in the to space, allocating a new object is cheap. A cursor in the to space is moved forward the number of bytes needed for the new object. This continues until the to space is exhausted. At this point the program is stopped and collection begins.

At this point the from space and to space are swapped. What was the to space and is now the from space, is scanned from beginning to end and any objects which are still alive are copied into the to space or are promoted to the old generation heap. If you want details, I suggest you read about Cheney's Algorithm.

Intuitively you should understand that every time an object is allocated either implicitly or explicitly (via a call to new, [], or {}) your application is moving closer and closer to a garbage collection and the dreaded application pause.

Is 10MB/sec of garbage expected for this application? #

In short, no. The developer is not doing anything to expect 10MB/sec of garbage.

Suspects #

The next phase of the investigation is to determine potential suspects and then whittle them down.

Suspect #1 #

Calling new during the frame. Remember that each object that is allocated moves you ever closer to a GC pause. Applications running at high frame rates in particular should strive for zero allocations per frame. Usually this requires a carefully thought out, application specific, object recycling system. The V8 detectives checked with the Oz team and they were not calling new. In fact the Oz team was already well aware of this requirement and said "That would be embarrassing". Scratch this one off the list.

Suspect #2 #

Modifying the "shape" of an object outside of the constructor. This happens whenever a new property is added to an object outside of the constructor. This creates a new hidden class for the object. When optimized code sees this new hidden class a deopt will be triggered, unoptimized code will execute until the code is classified as hot and optimized again. This de-optimization,re-optimization churn will result in jank but does not strictly correlate with excessive garbage creation. After a careful audit of the code, it was confirmed that object shapes were static, thus suspect #2 was ruled out.

Suspect #3 #

Arithmetic in unoptimized code. In unoptimized code all computation results in actual objects being allocated. For example, this snippet:

var a = p * d;
var b = c + 3;
var c = 3.3 * dt;
point.x = a * b * c;

Results in 5 HeapNumber objects being created. The first three are for the variables, a, b, and c. The 4th is for the anonymous value (a * b) and the 5th is from #4 * c; The 5th is ultimately assigned to point.x.

Oz does thousands of these operations per frame. If any of these computations occur in functions which are never optimized, they could be the cause of the garbage. Because computations in unoptimized allocate memory even for temporary results.

Suspect #4 #

Storing a double precision number to a property. A HeapNumber object must be created to store the number and the property altered to point at this new object. Altering the property to point at the HeapNumber will not produce garbage. However, it is possible that there are many double precision numbers being stored as object properties. The code is full of statements like the following:

sprite.position.x += 0.5 * (dt);

In optimized code, every time x is assigned a freshly computed value, a seemingly innocuous statement, a new HeapNumber object is implicitly allocated, bringing us closer to a garbage collection pause.

Note that by using a typed array (or a regular array which only has held doubles) you can avoid this specific problem entirely as the storage for the double precision number is allocated only once and repeatedly changing the value does not require new storage to be allocated.

Suspect #4 is a possibility.

Forensics #

At this point the detectives have two possible suspects: storing heap numbers as object properties and arithmetic computation happening inside unoptimized functions. It was time to head to the lab and determine definitively which suspect was guilty. NOTE: In this section I will be using a reproduction of the problem found in the actual Oz source code. This reproduction is orders of magnitude smaller than the original code, thus easier to reason about.

Experiment #1 #

Checking for suspect #3 (arithmetic computation inside unoptimized functions). The V8 JavaScript engine has a logging system builtin which can provide great insight into what is happening under the hood.

Starting with Chrome not running at all, launching Chrome with the flags:

--no-sandbox --js-flags="--prof --noprof-lazy --log-timer-events"

and then fully quitting Chrome will result in a v8.log file in the current directory.

In order to interpret the contents of v8.log, you must download the same version of v8 that your Chrome is using (check about:version), and build it.

After successfully building v8, you can process the log using the tick processor:

$ tools/linux-tick-processor /path/to/v8.log

(Substitute mac or windows for linux depending on your platform.) (This tool must be run from the top level source directory in v8.)

The tick processor displays a text based table of JavaScript functions which had the most ticks:

[JavaScript]:
ticks  total  nonlib   name
167   61.2%   61.2%  LazyCompile: *opt demo.js:12
 40   14.7%   14.7%  LazyCompile: unopt demo.js:20
 15    5.5%    5.5%  Stub: KeyedLoadElementStub
 13    4.8%    4.8%  Stub: BinaryOpStub_MUL_Alloc_Number+Smi
  6    2.2%    2.2%  Stub: BinaryOpStub_ADD_OverwriteRight_Number+Number
  4    1.5%    1.5%  Stub: KeyedStoreElementStub
  4    1.5%    1.5%  KeyedLoadIC:  {12}
  2    0.7%    0.7%  KeyedStoreIC:  {13}
  1    0.4%    0.4%  LazyCompile: ~main demo.js:30

You can see demo.js had three functions: opt, unopt, and main. Optimized functions have an asterisk (*) next to their names. Observe that the function opt is optimized and unopt is unoptimized.

Another important tool in the V8 detective's tool bag is plot-timer-event. It can be executed like so:

$ tools/plot-timer-event /path/to/v8.log

After being run, a png file called timer-events.png is in the current directory. Opening it up you should see something that looks like this:

Aside from the graph along the bottom, data is displayed in rows. The X axis is time (ms). The left hand side includes labels for each row:

The V8.Execute row has black vertical line drawn on it at each profile tick where V8 was executing JavaScript code. V8.GCScavenger has a blue vertical line drawn on it at each profile tick where V8 was performing a new generation collection. Similarly for the rest of the V8 states.

One of the most important rows is the "code kind being executed". It will be green whenever optimized code is executing and mix of red and blue when unoptimized code is being executed. The following screenshot shows the transition from optimized to unoptimized and then back to optimized code:

Ideally, but never immediately, this line will be solid green. Meaning that your program has transitioned into an optimized steady state. Unoptimized code will always run slower than optimized code.

If you've gone to this length it's worth noting that you can work much quicker by refactoring your application so that it can run in the v8 debug shell: d8. Using d8 gives you faster iteration times with the tick-processor and plot-timer-event tools. Another side effect of using d8 is that it becomes easier to isolate actual problem, reducing the amount of noise present in the data.

Looking at the timer events plot from the Oz source code, showed a transition from optimized to unoptimized code and, while executing unoptimized code many new generation collections were triggered, similar to the following screenshot (note time has been removed in the middle):

If you look closely you can see that the black lines indicating when V8 is executing JavaScript code are missing at precisely the same profile tick times as the new generation collections (blue lines). This demonstrates clearly that while garbage is being collected, the script is paused.

Looking at the tick processor output from the Oz source code, the top function (updateSprites) was not optimized. In other words, the function in which the program spent the most time was also unoptimized. This strongly indicates that suspect #3 is the culprit. The source for updateSprites contained loops that looked like these:

function updateSprites(dt) {
    for (var sprite in sprites) {
        sprite.position.x += 0.5 * dt;
        // 20 more lines of arithmetic computation.
    }
}

Knowing V8 as well as they do, they immediately recognized that the for-i-in loop construct is sometimes not optimized by V8. In other words, if a function contains a for-i-in loop construct, it may not be optimized. This is a special case today, and will likely change in the future, that is, V8 may one day optimize this loop construct. Since we aren't V8 detectives and don't know V8 like the back of our hands, how can we determine why updateSprites was not optimized?

Experiment #2 #

Running Chrome with this flag:

--js-flags="--trace-deopt --trace-opt-verbose"

displays a verbose log of optimization and deoptimization data. Searching through the data for updateSprites we find:

[disabled optimization for updateSprites, reason: ForInStatement is not fast case]

Just as the detectives hypothesized, the for-i-in loop construct was the reason.

Case Closed #

After discovering the reason updateSprites was not optimized, the fix was simple, simply move the computation into its own function, that is:

function updateSprite(sprite, dt) {
    sprite.position.x += 0.5 * dt;
    // 20 more lines of arithmetic computation.
}

function updateSprites(dt) {
    for (var sprite in sprites) {
        updateSprite(sprite, dt);
    }
}

updateSprite will be optimized, resulting in far fewer HeapNumber objects, resulting in less frequent GC pauses. It should be easy for you to confirm this by performing the same experiments with new code. The careful reader will notice that double numbers are still being stored as properties. If profiling indicates it is worth it, changing position to be an array of doubles or a typed data array would further reduce the number of objects being created.

Epilogue #

The Oz developers didn't stop there. Armed with the tools and techniques shared with them by the V8 detectives, they were able to find a few other functions that were stuck in deoptimization hell and factored the computation code into leaf functions which were optimized, resulting in even better performance.

Get out there and start solving some performance crimes!

Effectively managing memory at Gmail scale

2013-06-12T00:00:00Z

Introduction #

While JavaScript employs garbage collection for automatic memory management, it is not a substitute for effective memory management in applications. JavaScript applications suffer from the same memory related problems that native applications do, such as memory leaks and bloat, yet they must also deal with garbage collection pauses. Large-scale applications like Gmail encounter the same problems facing your smaller applications. Read on to learn how the Gmail team used Chrome DevTools to identify, isolate, and fix their memory problems.

Google I/O 2013 Session #

We presented this material at Google I/O 2013. Check out the video below:

Gmail, we have a problem… #

The Gmail team was facing a serious problem. Anecdotes of Gmail tabs consuming multiple gigabytes of memory on resource-constrained laptops and desktops were being heard increasingly frequently, often with a conclusion of bringing the entire browser down. Stories of CPUs being pinned at 100%, unresponsive apps, and Chrome sad tabs ("He's dead, Jim."). The team was at a loss as to how to even begin diagnosing the problem, let alone fix it. They had no idea how widespread the problem was and the available tools didn't scale up to large applications. The team joined forces with the Chrome teams, and together they developed new techniques to triage memory problems, improved existing tools, and enabled the collection of memory data from the field. But, before getting to the tools, let's cover the basics of JavaScript memory management.

Memory Management Basics #

Before you can effectively manage memory in JavaScript you must understand the fundamentals. This section will cover primitive types, the object graph, and provide definitions for memory bloat in general and a memory leak in JavaScript. Memory in JavaScript can be conceptualized as a graph and because of this Graph theory plays a part in JavaScript memory management and the Heap Profiler.

Primitive Types #

JavaScript has three primitive types:

Number (e.g. 4, 3.14159)
Boolean (true or false)
String ("Hello World")

These primitive types cannot reference any other values. In the object graph these values are always leaf or terminating nodes, meaning they never have an outgoing edge.

There is only one container type: the Object. In JavaScript the Object is an associative array. A non-empty object is an inner node with outgoing edges to other values (nodes).

What About Arrays? #

An Array in JavaScript is actually an Object that has numeric keys. This is a simplification, because JavaScript runtimes will optimize Array-like Objects and represent them under the hood as arrays.

Terminology #

Value - An instance of a primitive type, Object, Array, etc.
Variable - A name that references a value.
Property - A name in an Object that references a value.

Object Graph #

All values in JavaScript are part of the object graph. The graph begins with roots, for example, the window object. Managing the lifetime of GC roots is not in your control, as they are created by the browser and destroyed when the page is unloaded. Global variables are actually properties on window.

When Does a Value Become Garbage? #

A value becomes garbage when there is no path from a root to the value. In other words, starting at the roots and exhaustively searching all Object properties and variables that are alive in the stack frame, a value cannot be reached, it has become garbage.

What is a Memory Leak in JavaScript? #

A memory leak in JavaScript most commonly occurs when there are DOM nodes that are not reachable from the page's DOM tree, but are still referenced by a JavaScript object. While modern browsers are making it increasingly difficult to inadvertently create leaks, it's still easier than one might think. Let's say you append an element to the DOM tree like this:

email.message = document.createElement("div");
displayList.appendChild(email.message);

And later, you remove the element from the display list:

displayList.removeAllChildren();

As long as email exists, the DOM element referenced by message will not be removed, even though it is now detached from the page's DOM tree.

What is Bloat? #

Your page is bloated when you're using more memory than necessary for optimal page speed. Indirectly, memory leaks also cause bloat but that is not by design. An application cache that does not have any size bound is a common source of memory bloat. Also, your page can be bloated by host data, for example, pixel data loaded from images.

What is Garbage Collection? #

Garbage collection is how memory is reclaimed in JavaScript. The browser decides when this happens. During a collection, all script execution on your page is suspended while live values are discovered by a traversal of the object graph starting at the GC roots. All the values which are not reachable are classified as garbage. Memory for garbage values is reclaimed by the memory manager.

V8 Garbage Collector in Detail #

To help further understand how garbage collection happens, let's take a look at the V8 garbage collector in detail. V8 uses a generational collector. Memory is divided into two generations: the young and the old. Allocation and collection within the young generation is fast and frequent. Allocation and collection within the old generation is slower and less frequent.

Generational Collector #

V8 uses a two generation collector. The age of an value is defined as the number of bytes allocated since it was allocated. In practice, the age of an value is often approximated by the number of young generation collections that it survived. After a value is sufficiently old it is tenured into the old generation.

In practice, freshly allocated values do not live long. A study of Smalltalk programs, showed that only 7% of values survive after a young generation collection. Similar studies across runtimes found that on average between, 90% and 70% of freshly allocated values are never tenured into the old generation.

Young Generation #

The young generation heap in V8 is split into two spaces, named from and to. Memory is allocated from the to space. Allocating is very fast, until, the to space is full at which point a young generation collection is triggered. Young generation collection first swaps the from and to space, the old to space (now the from space) is scanned and all live values are copied into the to space or tenured into the old generation. A typical young generation collection will take on the order of 10 milliseconds (ms).

Intuitively, you should understand that each allocation your application makes brings you closer exhausting the to space and incurring a GC pause. Game developers, take note: to ensure a 16ms frame time (required to achieve 60 frames per second), your application must make zero allocations, because a single young generation collection will eat up most of the frame time.

Old Generation #

The old generation heap in V8 uses a mark-compact algorithm for collection. Old generation allocations occur whenever a value is tenured from the young generation to the old generation. Whenever an old generation collection occurs a young generation collection is done as well. Your application will paused on the order of seconds. In practice this is acceptable because old generation collections are infrequent.

V8 GC Summary #

Automatic memory management with garbage collection is great for developer productivity, but, each time you allocate a value, you move ever closer to a garbage collection pause. Garbage collection pauses can ruin the feel of your application by introducing jank. Now that you understand how JavaScript manages memory, you can make the right choices for your application.

Fixing Gmail #

Over the past year, numerous features and bug fixes have made their way into the Chrome DevTools making them more powerful than ever. In addition, the browser itself made a key change to the performance.memory API making it possible for Gmail and any other application to collect memory statistics from the field. Armed with these awesome tools, what once seemed like an impossible task soon became an exciting game of tracking down culprits.

Tools and Techniques #

Field Data and performance.memory API #

As of Chrome 22, the performance.memory API is enabled by default. For long-running applications like Gmail, data from real users is invaluable. This information allows us to distinguish between power users-- those who spend 8-16 hours a day on Gmail, receiving hundreds of messages a day-- from more average users who spend a few minutes a day in Gmail, receiving a dozen or so messages a week.

This API returns three pieces of data:

jsHeapSizeLimit - The amount of memory (in bytes) that the JavaScript heap is limited to.
totalJSHeapSize - The amount of memory (in bytes) that the JavaScript heap has allocated including free space.
usedJSHeapSize - The amount of memory (in bytes) currently being used.

One thing to keep in mind is that the API returns memory values for the entire Chrome process. Although it is not the default mode, under certain circumstances, Chrome may open multiple tabs in the same renderer process. This means that the values returned by performance.memory may contain the memory footprint of other browser tabs in addition to the one containing your app.

Measuring Memory At Scale #

Gmail instrumented their JavaScript to use the performance.memory API to collect memory information approximately once every 30 minutes. Because many Gmail users leave the app up for days at a time, the team was able to track memory growth over time as well as overall memory footprint statistics. Within a few days of instrumenting Gmail to collect memory information from a random sampling of users, the team had enough data to understand how widespread the memory problems were among average users. They set a baseline and used the stream of incoming data to track progress toward the goal of reducing memory consumption. Eventually this data would also be used to catch any memory regressions.

Beyond tracking purposes, the field measurements also provide a keen insight into the correlation between memory footprint and application performance. Contrary to the popular belief that "more memory results in better performance", the Gmail team found that the larger the memory footprint, the longer latencies were for common Gmail actions. Armed with this revelation, they were more motivated than ever to rein in their memory consumption.

Identifying a Memory Problem with the DevTools Timeline #

The first step in solving any performance problem is to prove that the problem exists, create a reproducible test, and take a baseline measurement of the problem. Without a reproducible program, you cannot reliably measure the problem. Without a baseline measurement you don't know by how much you've improved performance.

The DevTools Timeline panel is an ideal candidate for proving that the problem exists. It gives a complete overview of where time is spent when loading and interacting with your web app or page. All events, from loading resources to parsing JavaScript, calculating styles, garbage collection pauses, and repainting are plotted on a timeline. For the purposes of investigating memory issues, the Timeline panel also has a Memory mode which tracks total allocated memory, number of DOM nodes, number of window objects, and the number of event listeners allocated.

Proving a problem exists #

Start by identifying a sequence of actions you suspect to be leaking memory. Start recording the timeline, and perform the sequence of actions. Use the trash can button at the bottom to force a full garbage collection. If, after a few iterations, you see a sawtooth shaped graph, you are allocating lots of shortly lived objects. But if the sequence of actions is not expected to result in any retained memory, and the DOM node count does not drop down back to the baseline where you began, you have good reason to suspect there is a leak.

Once you've confirmed that the problem exists, you can get help identifying the source of the problem from the DevTools Heap Profiler.

Finding Memory Leaks with the DevTools Heap Profiler #

The Profiler panel provides both a CPU profiler and a Heap profiler. Heap profiling works by taking a snapshot of the object graph. Before a snapshot is taken both the young and old generations are garbage collected. In other words, you will only see values which were alive when the snapshot was taken.

There is too much functionality in the Heap profiler to cover sufficiently in this article, but detailed documentation can be found on the Chrome Developers site. We'll focus here on the Heap Allocation profiler.

Using the Heap Allocation Profiler #

The Heap Allocation profiler combines the detailed snapshot information of the Heap Profiler with the incremental updating and tracking of the Timeline panel. Open the Profiles panel, start a Record Heap Allocations profile, perform a sequence of actions, then stop the recording for analysis. The allocation profiler takes heap snapshots periodically throughout the recording (as frequently as every 50 ms!) and one final snapshot at the end of the recording.

The bars at the top indicate when new objects are found in the heap. The height of each bar corresponds to the size of the recently allocated objects, and the color of the bars indicate whether or not those objects are still live in the final heap snapshot: blue bars indicate objects that are still live at the end of the timeline, gray bars indicate objects that were allocated during the timeline, but have since been garbage collected.

In the example above, an action was performed 10 times. The sample program caches five objects, so the last five blue bars are expected. But the leftmost blue bar indicates a potential problem. You can then use the sliders in the timeline above to zoom in on that particular snapshot and see the objects that were recently allocated at that point. Clicking on a specific object in the heap will show its retaining tree in the bottom portion of the heap snapshot. Examining the retaining path to the object should give you enough information to understand why the object was not collected, and you can make the necessary code changes to remove the unnecessary reference.

Resolving Gmail's Memory Crisis #

By using the tools and techniques discussed above, the Gmail team was able to identify a few categories of bugs: unbounded caches, infinitely growing arrays of callbacks waiting for something to happen that never actually happens, and event listeners unintentionally retaining their targets. By fixing these issues, the overall memory usage of Gmail was dramatically reduced. Users in the 99% percent used 80% less memory than before and the memory consumption of the median users dropped by nearly 50%.

Because Gmail used less memory the GC pause latency was reduced, increasing the overall user experience.

Also of note, with the Gmail team collecting statistics on memory usage, they were able to uncover garbage collection regressions inside Chrome. Specifically, two fragmentation bugs were discovered when Gmail's memory data began showing a dramatic increase in the gap between total memory allocated and live memory.

Call to Action #

Ask yourself these questions:

How much memory is my app using? It's possible that you are using too much memory which contrary to popular belief has a net negative on overall application performance. It's hard to know exactly what the right number is, but, be sure to verify that any extra caching your page is using has a measurable performance impact.
Is my page leak free? If your page has memory leaks it can not only impact your page's performance but other tabs as well. Use the object tracker to help narrow in on any leaks.
How frequently is my page GCing? You can see any GC pause using Timeline panel in Chrome Developer Tools. If your page is GCing frequently, chances are you are allocating too frequently, churning through your young generation memory.

Conclusion #

We started off in a crisis. Covered the core basics of memory management in JavaScript and V8 in particular. You learned how to use the tools, including the new object tracker feature available in the latest builds of Chrome. The Gmail team, armed with this knowledge, solved their memory usage problem and saw improved performance. You can do the same with your web apps!

Profiling mobile HTML5 apps with Chrome DevTools

2013-03-26T00:00:00Z

Introduction #

Today, the most important thing you can do for your site is make sure it performs well when visited from a phone or tablet. Read on and learn how to optimize your site for the mobile browser using Chrome DevTools and an Android device.

Why is Optimizing for the Mobile Web so Important? #

Performance #

Mobile devices are getting faster CPUs, more RAM, faster GPUs, and faster network access as we transition from 2G and 3G to 4G. Despite the drumbeat of progress, mobile devices are underpowered when compared to our computers. In more concrete terms, loading network resources takes longer, unpacking images takes longer, painting the page takes longer, executing scripts takes longer. It’s safe to assume that your page runs 5 to 10 times slower on a mobile device.

Battery #

Mobile devices are exclusively battery powered. Users of mobile devices want that battery to last as long as possible. A sub-optimal site will drain a battery much quicker than needed. Minimize network traffic and reduce paints to reduce battery drain. When you fetch a resource, the WiFi or cell radio must be on, which drains battery. When the browser paints an element, the CPU and GPU usage spikes, which also drains battery.

Engagement #

Performance is there to increase the metric that matters most to you. In Facebook we care about scrolling. In an A/B test, we slowed down scrolling from 60fps down to 30fps. Engagement collapsed. We said okay, therefore scrolling matters.

Facebook at Edge Conference

Mobile users expect to be able to get in and out quickly. The fastest site will get the most engagement.

Managing Performance #

Chrome ships with a great set of developer tools. This article teaches you how to use these tools to profile your mobile site. If you’re already familiar with Chrome DevTools, great! If not, check these great tutorials:

Now that you’re caught up, let’s see how to accelerate your mobile site with DevTools. If this is the first time you’ve used Chrome DevTools for Android check out the getting started guide at the bottom of the article.

Using Chrome DevTools Remotely #

With your Android device tethered to your computer. In desktop Chrome, navigate to http://localhost:9222 and on your Android device, open up your site. You will be taken to a list of open tabs on your Android device. Pick your page from the list of ‘Inspectable pages’.

and you will be taken to Chrome DevTools for that page.

Aww… that familiar Chrome DevTools toolbar is right there. The most important thing to understand about remote Chrome DevTools is that they are the same DevTools you’re using today on your desktop. The only difference isthat your Android device is only responsible for the page, while your desktop is responsible for DevTools. Under the hood, the same data is collected and the same functionality is available.

As an example, I visited www.sfgate.com/movies on my phone. Using Chrome DevTools on my desktop I hovered over a div in the Elements tool and, just like it is on the desktop, the div is visually highlighted on my Android device.

The Elements tool can also be used to toggle styles on and off, which will come in handy when we attempt to investigate paint times.

Shedding Light on Network Access #

Network performance is import, and it’s even more important on the mobile web. Mobile devices are often on slower connections than our desktop and laptop computers. To make sure you’re doing the right thing, take a network snapshot by going to the Network tool and pressing record.

The screenshot shows the network traffic resulting from a Google search. Observe the network requests your site makes and find ways to minimize them. If your site makes polling requests to the server, you may want to pay attention to user activity and avoid polling when the user has been idle. The Network tool allows you to view the raw HTTP headers, useful in case mobile networks are altering them at all.

Optimizing Paint Times #

One of the biggest bottlenecks in mobile web browsers is painting your page. Painting is the process of drawing an element on the page with the specified styling. When one element is expensive to paint it slows down painting of the whole page. Chrome makes an attempt at caching previously painted elements in an offscreen buffer. But, on mobile the amount of GPU RAM available is limited, limiting the number of elements that can be cached off screen. The side effect is more paints and each paint is slower than the desktop. In order to have responsive scrolling, you must minimize paint times.

Chrome 25 includes continuous page repaint mode. Continuous page repaint mode never caches painted elements and, instead, paints all elements each frame. By forcing all elements to be painted each frame, it is possible to perform A/B testing of paint times by toggling elements on and off, and styles on and off. While the process is manual, it’s an invaluable tool for tracking down how expensive painting each element on your page is. The first rule of optimization club is measure what you’re trying to optimize to get a baseline. Let’s work through a simple example.

First, enable continuous page repaint mode:

After enabling, a graph will be visible in the upper right corner of your Android device. The x-axis of the graph is time, divided into frames. The y-axis of the graph measures paint time, in milliseconds. You can see that, on my device, the page takes 14 milliseconds to paint. The minimum and maximum paint times are also shown along with GPU memory used.

As an experiment, I set the style on the selected element to be display: none. Let’s see how expensive the page is to paint now.

Paint times went from around 14 milliseconds per frame down to 4 milliseconds per frame. In other words, painting that one element took approximately 10 milliseconds. By following the process of toggling elements on and off and styles on and off you can quickly narrow in on the expensive parts of your page. Remember, faster paint times means less jank, a longer battery and more engagement from your users. When you’re ready to dig deeper, be sure to read this great article on continuous page repaint mode.

Advanced Features #

about:tracing #

Many of the more advanced developer features available in desktop Chrome are also available in Android Chrome too. For example, about:gpu-internals, about:appcache-internals, and about:net-internals are available. When investigating a particularly tricky problem you sometimes need more data in order to narrow in on the cause of your problem. On the desktop, you might be using about:tracing. If you’re not already familiar with about:tracing, watch my video on using and exploring the about:tracing profiling tool. It is possible to capture the same data from Android Chrome, follow these steps to get started:

Download adb_trace.py
Run adb_trace.py from the command line
Use Chrome on Android
Press enter on the command line, shutting down the adb_trace.py script.

After adb_trace.py completes you will have a JSON file that you can load in desktop Chrome’s about:tracing.

Starting Guide #

Now that we've reviewed what remote Chrome DevTools can do, let's cover how to get started in your remote debugging session. If you haven’t used them before,read detailed instructions on how to get started. If you’ve already used them, but have forgotten exactly how to use them, I’ve listed shortened instructions here as well.

1. Install Android SDK #

You might be wondering why you have to install the Android SDK when you are developing for the web. Included in the SDK is adb (Android Debug Bridge). Desktop Chrome needs to be able to communicate with your Android device. Chrome doesn’t talk directly with the Android device, instead it routes communication through adb.

2. Enable USB debugging on your device #

The option for enabling USB debugging can be found in Android Settings. Enable it.

3. Connect to the device #

If you haven’t already, connect your Android device to your desktop via USB. If this is the first time you’ve used USB debugging you will be given the following prompt:

If you will be doing remote debug sessions frequently, I recommend checking ‘Always allow from this computer’.

4. Verify that your device is properly connected #

Run adb devices from your command prompt. You should see your device listed.

5. Enable USB debugging in Chrome #

Open Settings > Advanced > DevTools and check the Enable USB Web debugging option as shown here:

6. Creating a DevTools connection to your Android device #

Run the following command:

adb forward tcp:9222 localabstract:chrome_devtools_remote

creates a bridge between your desktop machine and your Android device via adb. If you run into any issues getting to this point, read over the detailed setup instructions here.

7. Verifying you're good to go #

Verify that your device is properly connected by opening Chrome on your desktop and navigating to http://localhost:9222. If you get a 404, another error, or don’t see something like the following:

Read over the detailed setup instructions here.

Conclusion #

Mobile users are often in a hurry and need to quickly get that important piece of information from your page. It is your duty as a mobile site builder to ensure that the page loads quickly and performs well on mobile. If not, user engagement will drop. Remote Chrome DevTools are functionally equivalent to their desktop counterparts. The UI is similar enough that you don’t need to learn a new set of tools. In other words, your work flow carries over. Remember, Facebook isn’t invincible to performance problems and your site isn’t either. Performant sites get more user engagement.

Pointer lock and first person shooter controls

2012-08-17T00:00:00Z

Introduction #

The Pointer Lock API helps properly implement first-person shooter controls in a browser game. Without relative mouse movement the player's cursor could, for example, hit the right edge of the screen and any further movements to the right would be discounted - the view would not continue to pan right, and the player would not be able to pursue the bad guys and strafe them with his machine gun. The player is going to get fragged and become frustrated. With pointer lock this suboptimal behaviour can't happen.

The Pointer Lock API allows your application to do the following:

Get access to raw mouse data including relative mouse movements
Route all mouse events to a specific element

As a side effect of enabling pointer lock, the mouse cursor is hidden allowing you to choose to draw an application-specific pointer if you desire, or leave the mouse pointer hidden so that the user can move the frame with the mouse. Relative mouse movement is the mouse pointer position's delta from the previous frame regardless of absolute position. For example, if the mouse pointer moved from (640, 480) to (520, 490) the relative movement was (-120, 10). See below for an interactive example showing raw mouse position deltas.

This tutorial covers two topics: the nuts and bolts of activating and processing pointer lock events, and implementing the first-person shooter control scheme. That's right, when you're finished reading this article you will know how to use pointer lock and implement Quake-style controls for your very own browser game!

Browser compatibility #

Browser support

Chrome 37, Supported 37
Firefox 50, Supported 50
Edge 13, Supported 13
Safari 10.1, Supported 10.1

Source

Pointer Lock Mechanics #

Feature Detection #

To determine if the user's browser supports pointer lock you need to check for pointerLockElement or a vendor-prefixed version in the document object. In code:

var havePointerLock = 'pointerLockElement' in document ||
    'mozPointerLockElement' in document ||
    'webkitPointerLockElement' in document;

Currently pointer lock is only available in Firefox and Chrome. Opera and IE do not yet support it.

Activating #

Activating pointer lock is a two step process. First your application requests pointer lock be enabled for a specific element, and immediately after the user gives permission, a pointerlockchange event fires. The user can cancel pointer lock at any time by pressing the escape key. Your application can also progrmamatically exit pointer lock. When pointer lock is cancelled a pointerlockchange event fires.

element.requestPointerLock = element.requestPointerLock ||
			     element.mozRequestPointerLock ||
			     element.webkitRequestPointerLock;
// Ask the browser to lock the pointer
element.requestPointerLock();

// Ask the browser to release the pointer
document.exitPointerLock = document.exitPointerLock ||
			   document.mozExitPointerLock ||
			   document.webkitExitPointerLock;
document.exitPointerLock();

The above code is all it takes. When the browser locks the pointer a bubble will popup letting the user know that your application has locked the pointer and instructing them that they can cancel it by pressing the 'Esc' key.

Pointer Lock info bar in Chrome.

Event Handling #

There are two events that your application must add listeners for. The first is pointerlockchange, which fires whenever a change in pointer lock state occurs. The second is mousemove which fires whenever the mouse has moved.

// Hook pointer lock state change events
document.addEventListener('pointerlockchange', changeCallback, false);
document.addEventListener('mozpointerlockchange', changeCallback, false);
document.addEventListener('webkitpointerlockchange', changeCallback, false);

// Hook mouse move events
document.addEventListener("mousemove", this.moveCallback, false);

Inside your pointerlockchange callback you must check if the pointer has just been locked or unlocked. Determining if pointer lock was enabled is simple: check if document.pointerLockElement is equal to the element that pointer lock was requested for. If it is, your application successfully locked the pointer and if it is not, the pointer was unlocked by the user or your own code.

if (document.pointerLockElement === requestedElement ||
  document.mozPointerLockElement === requestedElement ||
  document.webkitPointerLockElement === requestedElement) {
  // Pointer was just locked
  // Enable the mousemove listener
  document.addEventListener("mousemove", this.moveCallback, false);
} else {
  // Pointer was just unlocked
  // Disable the mousemove listener
  document.removeEventListener("mousemove", this.moveCallback, false);
  this.unlockHook(this.element);
}

When pointer lock is enabled clientX, clientY, screenX, and screenY remain constant. movementX and movementY are updated with the number of pixels the pointer would have moved since the last event was delivered. In pseudo-code:

event.movementX = currentCursorPositionX - previousCursorPositionX;
event.movementY = currentCursorPositionY - previousCursorPositionY;

Inside the mousemove callback relative mouse motion data can be extracted from the event's movementX and movementY fields.

function moveCallback(e) {
  var movementX = e.movementX ||
      e.mozMovementX          ||
      e.webkitMovementX       ||
      0,
  movementY = e.movementY ||
      e.mozMovementY      ||
      e.webkitMovementY   ||
      0;
}

Catching errors #

If an error is raised by either entering or exiting pointer lock the pointerlockerror event fires. There is no data attached to this event.

document.addEventListener('pointerlockerror', errorCallback, false);
document.addEventListener('mozpointerlockerror', errorCallback, false);
document.addEventListener('webkitpointerlockerror', errorCallback, false);

Full-screen Required? #

Originally pointer lock was tied to the FullScreen API. Meaning that an element must be in fullscreen mode before it can have the pointer locked to it. That is no longer true and pointer lock can be used for any element in your application full-screen or not.

First-Person Shooter Controls Example #

Now that we have pointer lock enabled and receiving events, it's time for a practical example. Have you ever wanted to know how the controls in Quake work? Strap in because I'm about to explain them with code!

First-person shooter controls are built around four core mechanics:

Moving forward and backward along the current look vector
Moving left and right along the current strafe vector
Rotating the view yaw (left and right)
Rotating the view pitch (up and down)

A game implementing this control scheme needs only three pieces of data: camera position, camera look vector, and a constant up vector. The up vector is always (0, 1, 0). All four of the above mechanics just manipulate the camera position and camera look vector in different ways.

Movement #

First on deck is movement. In the demo below movement is mapped to the standard W, A, S, and D keys. The W and S keys drive the camera forward and backward. While the A and D keys drive the camera to the left and right. Moving the camera forward and backward is simple:

// Forward direction
var forwardDirection = vec3.create(cameraLookVector);
// Speed
var forwardSpeed = dt * cameraSpeed;
// Forward or backward depending on keys held
var forwardScale = 0.0;
forwardScale += keyState.W ? 1.0 : 0.0;
forwardScale -= keyState.S ? 1.0 : 0.0;
// Scale movement
vec3.scale(forwardDirection, forwardScale * forwardSpeed);
// Add scaled movement to camera position
vec3.add(cameraPosition, forwardDirection);

Strafing left and right requires a strafe direction. The strafe direction can be computed using the cross product:

// Strafe direction
var strafeDirection = vec3.create();
vec3.cross(cameraLookVector, cameraUpVector, strafeDirection);

Once you have the strafe direction, implementing strafe movement is the same as moving forward or backward.

Next up is rotating the view.

Yaw #

Yaw or the horizontal rotation of the camera view is just a rotation around the constant up vector. Below is general code for rotating the camera look vector around an arbitrary axis. It works by constructing a quaternion representing the rotation of deltaAngle radians around axis and then uses the quaternion to rotate the camera look vector:

// Extract camera look vector
var frontDirection = vec3.create();
vec3.subtract(this.lookAtPoint, this.eyePoint, frontDirection);
vec3.normalize(frontDirection);
var q = quat4.create();
// Construct quaternion
quat4.fromAngleAxis(deltaAngle, axis, q);
// Rotate camera look vector
quat4.multiplyVec3(q, frontDirection);
// Update camera look vector
this.lookAtPoint = vec3.create(this.eyePoint);
vec3.add(this.lookAtPoint, frontDirection);

Pitch #

Implementing pitch or the vertical rotation of the camera view is similar but instead of a rotation around the up vector you apply a rotation around the strafe vector. The first step is to compute the strafe vector and then rotate the camera look vector around that axis.

Summary #

The Pointer Lock API allows you to take control of the mouse cursor. If you're making web games your players will love it when they stop getting fragged because they excitedly moved mouse out of the window and your game stopped getting mouse updates. Usage is simple:

Add pointerlockchange event listener to track the state of pointer lock
Request pointer lock for a specific element
Add mousemove event listener to get updates

External Demos #

References #

Mozilla Developer Network API documentation