Marcin Wichary on web.dev

Jumping the hurdles with the Gamepad API

2012-08-06T00:00:00Z

Introduction #

Let the novices keep their keyboards for adventure games, their precious multi-touchy-feely fingertips for fruit cutting, and their fancy newfangled motion sensors for pretending they can dance like Michael Jackson. (Newsflash: They can't.) But you're different. You're better. You're a pro. For you, the games begin and end with a gamepad in your hands.

But wait. Aren't you out of luck if you want to support a gamepad in your web app? Not anymore. The brand new Gamepad API comes to the rescue, allowing you to use JavaScript to read the state of any gamepad controller attached to your computer. It's so fresh off the presses, that it only landed in Chrome 21 last week – and it's also on the verge of being supported in Firefox (currently available in a special build).

That turned out to be pretty great timing, because we get a chance to use it recently in the Hurdles 2012 Google doodle. This article will briefly explain how we added Gamepad API to the doodle, and what we learned during the process.

Hurdles 2012 Google doodle

Gamepad tester #

Ephemeral as they are, interactive doodles tend to be pretty complex under the hood. So that it's easier to demonstrate what we're talking about, we took the gamepad code from the doodle, and put together a simple gamepad tester. You can use it to see whether your USB gamepad works correctly – and also look under the hood to examine how it's done.

What browsers support it today? #

Browser support

Chrome 21, Supported 21
Firefox 29, Supported 29
Edge 12, Supported 12
Safari 10.1, Supported 10.1

Source

What gamepads can be used? #

Generally, any modern gamepad that is supported by your system natively should work. We tested various gamepads from off-brand USB controllers on a PC, through PlayStation 2 gamepads connected via a dongle to a Mac, all the way to Bluetooth controllers paired with a Chrome OS notebook.

Gamepads

This is a photo of some controllers we used for testing our doodle – "Yes, mom, that really is what I do at work." If your controller doesn't work, or if the controls are mapped incorrectly, please file a bug against Chrome or Firefox . (Please test in the absolutely newest version of each browser to make sure it's not already fixed.)

Feature Detecting the Gamepad API< #

Easy enough in Chrome:

var gamepadSupportAvailable = !!navigator.webkitGetGamepads || !!navigator.webkitGamepads;

It does not seem possible to detect this in Firefox just yet – everything is event-based, and all the event handlers need to be attached to window, which prevents a typical technique of detecting event handlers from working.

But we're sure that is temporary. The ever-so-awesome Modernizr already tells you about the Gamepad API, so we recommend this for all your current and future detecting needs:

var gamepadSupportAvailable = Modernizr.gamepads;

Finding out about connected gamepads #

Even if you connect the gamepad, it won't manifest itself in any way unless the user presses any of its buttons first. This is to prevent fingerprinting, although proves to be a bit of a challenge for user experience: you can't ask the user to press the button or provide gamepad-specific instructions because you have no idea whether they connected their controller.

Once you clear that hurdle (sorry…), however, more await.

Polling #

Chrome's implementation of the API exposes a function – navigator.webkitGetGamepads() – you can use to get a list of all the gamepads currently plugged into to the system, alongside with their current state (buttons + sticks). The first connected gamepad will be returned as the first entry in the array, and so on.

(This function call just recently replaced an array you could access directly – navigator.webkitGamepads[]. As of early August 2012, accessing this array is still necessary in Chrome 21, while the function call works in Chrome 22 and newer. Moving forward, the function call is the recommended way to use the API, and will slowly trickle to all the installed Chrome browsers.)

The so-far-implemented part of the spec requires you to continuously check the state of connected gamepads (and compare it to the previous one if necessary), instead of firing events when things change. We relied on requestAnimationFrame() to set up polling in the most efficient and battery-friendly way. For our doodle, even though we already have a requestAnimationFrame() loop to support animations, we created a second completely separate one – it was simpler to code and shouldn't affect performance in any way.

Here is the code from the tester:

/**
 * Starts a polling loop to check for gamepad state.
 */
startPolling: function() {
    // Don't accidentally start a second loop, man.
    if (!gamepadSupport.ticking) {
    gamepadSupport.ticking = true;
    gamepadSupport.tick();
    }
},

/**
 * Stops a polling loop by setting a flag which will prevent the next
 * requestAnimationFrame() from being scheduled.
 */
stopPolling: function() {
    gamepadSupport.ticking = false;
},

/**
 * A function called with each requestAnimationFrame(). Polls the gamepad
 * status and schedules another poll.
 */
tick: function() {
    gamepadSupport.pollStatus();
    gamepadSupport.scheduleNextTick();
},

scheduleNextTick: function() {
    // Only schedule the next frame if we haven't decided to stop via
    // stopPolling() before.
    if (gamepadSupport.ticking) {
    if (window.requestAnimationFrame) {
        window.requestAnimationFrame(gamepadSupport.tick);
    } else if (window.mozRequestAnimationFrame) {
        window.mozRequestAnimationFrame(gamepadSupport.tick);
    } else if (window.webkitRequestAnimationFrame) {
        window.webkitRequestAnimationFrame(gamepadSupport.tick);
    }
    // Note lack of setTimeout since all the browsers that support
    // Gamepad API are already supporting requestAnimationFrame().
    }
},

/**
 * Checks for the gamepad status. Monitors the necessary data and notices
 * the differences from previous state (buttons for Chrome/Firefox,
 * new connects/disconnects for Chrome). If differences are noticed, asks
 * to update the display accordingly. Should run as close to 60 frames per
 * second as possible.
 */
pollStatus: function() {
    // (Code goes here.)
},

If you only care about one gamepad, getting its data might be as simple as:

var gamepad = navigator.webkitGetGamepads && navigator.webkitGetGamepads()[0];

/ If you want to be a bit more clever, or support more than one player simultaneously, you will need to add a few more lines of code to react to more complex scenarious (two or more gamepads connected, some of them getting disconnected mid-way, etc.). You can look at the source code of our tester, function pollGamepads(), for one approach on how to solve this.

Events #

Firefox uses an alternative, better way described in the Gamepad API spec. Instead of asking you to poll, it exposes two events – MozGamepadConnected and MozGamepadDisconnected – which are fired whenever a gamepad gets plugged in (or, more precisely, plugged in and "announced" by pressing any of its buttons) or unplugged. The gamepad object that will continue to reflect its future state is passed as the .gamepad parameter of the event object.

From the tester source code:

/**
 * React to the gamepad being connected. Today, this will only be executed
 * on Firefox.
 */
onGamepadConnect: function(event) {
    // Add the new gamepad on the list of gamepads to look after.
    gamepadSupport.gamepads.push(event.gamepad);

    // Start the polling loop to monitor button changes.
    gamepadSupport.startPolling();

    // Ask the tester to update the screen to show more gamepads.
    tester.updateGamepads(gamepadSupport.gamepads);
},

Summary #

In the end, our initialization function in the tester, supporting both approaches, looks like this:

/**
 * Initialize support for Gamepad API.
 */
init: function() {
    // As of writing, it seems impossible to detect Gamepad API support
    // in Firefox, hence we need to hardcode it in the third clause.
    // (The preceding two clauses are for Chrome.)
    var gamepadSupportAvailable = !!navigator.webkitGetGamepads ||
        !!navigator.webkitGamepads ||
        (navigator.userAgent.indexOf('Firefox/') != -1);

    if (!gamepadSupportAvailable) {
    // It doesn't seem Gamepad API is available – show a message telling
    // the visitor about it.
    tester.showNotSupported();
    } else {
    // Firefox supports the connect/disconnect event, so we attach event
    // handlers to those.
    window.addEventListener('MozGamepadConnected',
                            gamepadSupport.onGamepadConnect, false);
    window.addEventListener('MozGamepadDisconnected',
                            gamepadSupport.onGamepadDisconnect, false);

    // Since Chrome only supports polling, we initiate polling loop straight
    // away. For Firefox, we will only do it if we get a connect event.
    if (!!navigator.webkitGamepads || !!navigator.webkitGetGamepads) {
        gamepadSupport.startPolling();
    }
    }
},

Gamepad info #

Every gamepad connected to the system will be represented by an object looking something like this:

id: "PLAYSTATION(R)3 Controller (STANDARD GAMEPAD Vendor: 054c Product: 0268)"
index: 1
timestamp: 18395424738498
buttons: Array[8]
    0: 0
    1: 0
    2: 1
    3: 0
    4: 0
    5: 0
    6: 0.03291
    7: 0
axes: Array[4]
    0: -0.01176
    1: 0.01961
    2: -0.00392
    3: -0.01176

Basic info #

The top few fields are simple metadata:

id: a textual description of the gamepad
index: an integer useful to tell different gamepads attached to one computer apart
timestamp: the timestamp of the last update of the button/axes state (currently only supported in Chrome)

Buttons and sticks #

Today's gamepads are not exactly what your grandpa might have used to save the princess in the wrong castle – they typically have at least sixteen separate buttons (some discrete, some analogue), in addition to two analogue sticks. Gamepad API will tell you about all the buttons and analogue sticks that are reported by the operating system.

Once you get the current state in the gamepad object, you can access the buttons via .buttons[] and sticks via .axes[] arrays. Here's a visual summary of what they correspond to:

Gamepad Diagram

The spec asks the browser to map first sixteen buttons and four axes to:

gamepad.BUTTONS = {
    FACE_1: 0, // Face (main) buttons
    FACE_2: 1,
    FACE_3: 2,
    FACE_4: 3,
    LEFT_SHOULDER: 4, // Top shoulder buttons
    RIGHT_SHOULDER: 5,
    LEFT_SHOULDER_BOTTOM: 6, // Bottom shoulder buttons
    RIGHT_SHOULDER_BOTTOM: 7,
    SELECT: 8,
    START: 9,
    LEFT_ANALOGUE_STICK: 10, // Analogue sticks (if depressible)
    RIGHT_ANALOGUE_STICK: 11,
    PAD_TOP: 12, // Directional (discrete) pad
    PAD_BOTTOM: 13,
    PAD_LEFT: 14,
    PAD_RIGHT: 15
};

gamepad.AXES = {
    LEFT_ANALOGUE_HOR: 0,
    LEFT_ANALOGUE_VERT: 1,
    RIGHT_ANALOGUE_HOR: 2,
    RIGHT_ANALOGUE_VERT: 3
};

The extra buttons and axes will be appended to the ones above. Please note that neither sixteen buttons nor four axes are guaranteed, though – be ready for some of them to simply be undefined.

The buttons can take values from 0.0 (not pressed) to 1.0 (pressed completely). The axes go from -1.0 (completely left or up) through 0.0 (center) to 1.0 (completely right or down).

Analogue or discrete? #

Ostensibly, every button could be an analogue one – this is somewhat common for shoulder buttons, for example. Therefore, it is best to set a threshold rather than just compare it bluntly to 1.00 (what if an analogue button happens to be slightly dirty? It might never reach 1.00). In our doodle we do it this way:

gamepad.ANALOGUE_BUTTON_THRESHOLD = .5;

gamepad.buttonPressed_ = function(pad, buttonId) {
    return pad.buttons[buttonId] &&
            (pad.buttons[buttonId] > gamepad.ANALOGUE_BUTTON_THRESHOLD);
};

You can do the same to turn analogue sticks into digital joysticks. Sure, there is always the digital pad (d-pad), but your gamepad might not have one. Here's our code to handle that:

gamepad.AXIS_THRESHOLD = .75;

gamepad.stickMoved_ = function(pad, axisId, negativeDirection) {
    if (typeof pad.axes[axisId] == 'undefined') {
    return false;
    } else if (negativeDirection) {
    return pad.axes[axisId] < -gamepad.AXIS_THRESHOLD;
    } else {
    return pad.axes[axisId] > gamepad.AXIS_THRESHOLD;
    }
};

Button presses and stick movements #

Events #

In some cases, like a flight simulator game, continuously checking and reacting to stick positions or button presses makes more sense… but for things like Hurdles 2012 doodle? You might wonder: Why do I need to check for buttons every single frame? Why can't I get events like I do for keyboard or mouse up/down?

Good news is, you can. Bad news is – in the future. It's in the spec, but not implemented in any browser yet.

Polling #

In the meantime, your way out is comparing the current and previous state, and calling functions if you see any difference. For example:

if (buttonPressed(pad, 0) != buttonPressed(oldPad, 0)) {
    buttonEvent(0, buttonPressed(pad, 0) ? 'down' : 'up');
}

for (var i in gamepadSupport.gamepads) {
    var gamepad = gamepadSupport.gamepads[i];

    // Don't do anything if the current timestamp is the same as previous
    // one, which means that the state of the gamepad hasn't changed.
    // This is only supported by Chrome right now, so the first check
    // makes sure we're not doing anything if the timestamps are empty
    // or undefined.
    if (gamepadSupport.prevTimestamps[i] &&
        (gamepad.timestamp == gamepadSupport.prevTimestamps[i])) {
    continue;
    }
    gamepadSupport.prevTimestamps[i] = gamepad.timestamp;

    gamepadSupport.updateDisplay(i);
}

The keyboard-first approach in the Hurdles 2012 doodle #

Since without a gamepad, our today's doodle's preferred input method is the keyboard, we decided for the gamepad to emulate it rather closely. This meant three decisions:

The doodle only needs three buttons – two for running, and one for jumping – but the gamepad is likely to have many more. We therefore mapped all the sixteen known buttons and two known sticks onto those three logical functions in a way we thought made most sense, so that people could run by: alternating A/B buttons, alternating shoulder buttons, pressing left/right on the d-pad, or swinging either stick violently left and right (some of those will, of course, be more efficient than the others). For example:

newState[gamepad.STATES.LEFT] =
    gamepad.buttonPressed_(pad, gamepad.BUTTONS.PAD_LEFT) ||
    gamepad.stickMoved_(pad, gamepad.AXES.LEFT_ANALOGUE_HOR, true) ||
    gamepad.stickMoved_(pad, gamepad.AXES.RIGHT_ANALOGUE_HOR, true),

newState[gamepad.STATES.PRIMARY_BUTTON] =
    gamepad.buttonPressed_(pad, gamepad.BUTTONS.FACE_1) ||
    gamepad.buttonPressed_(pad, gamepad.BUTTONS.LEFT_SHOULDER) ||
    gamepad.buttonPressed_(pad, gamepad.BUTTONS.LEFT_SHOULDER_BOTTOM) ||
    gamepad.buttonPressed_(pad, gamepad.BUTTONS.SELECT) ||
    gamepad.buttonPressed_(pad, gamepad.BUTTONS.START) ||
    gamepad.buttonPressed_(pad, gamepad.BUTTONS.LEFT_ANALOGUE_STICK),

We treated each analogue input as a discrete one, using the threshold functions described previously.

We went as far as bolting the gamepad input onto the doodle, instead of baking it in – our polling loop actually synthesizes necessary keydown and keyup events (with a proper keyCode) and sends them back to DOM:

// Create and dispatch a corresponding key event.
var event = document.createEvent('Event');
var eventName = down ? 'keydown' : 'keyup';
event.initEvent(eventName, true, true);
event.keyCode = gamepad.stateToKeyCodeMap_[state];
gamepad.containerElement_.dispatchEvent(event);

And that's all there is to it!

Tips and tricks #

Remember that the gamepad won't be visible in your browser at all before a button is pressed.
If you are testing the gamepad in different browsers simultaneously, note that only one of them can sense the controller. If you're not receiving any events, make sure to close the other pages that might be using it. Plus, from our experience, sometimes a browser can "hold onto" the gamepad even if you close the tab or quit the browser itself. Restarting the system is sometimes the only way to fix things.
As always, use Chrome Canary and the equivalents for other browsers to make sure you're experiencing the best support – and then act appropriately if you see older versions behaving differently.

The Future #

We hope this helps to shed some light on this new API – still a bit precarious, but already a lot of fun.

In addition to the missing pieces of the API (e.g. events) and broader browser support, we're also hoping to eventually see things like rumble control, access to built-in gyroscopes, etc. And, more support for different types of gamepads – please file a bug against Chrome and/or file a bug against Firefox if you see find one that works incorrectly or not at all.

But before that, go and play with our Hurdles 2012 doodle and see how much more fun it is on the gamepad. Oh, did you just say you could do better than 10.7 seconds? Bring it.

Case Study - Building the Stanisław Lem Google doodle

2011-08-06T00:00:00Z

Hello, (strange) world #

The Google homepage is a fascinating environment to code within. It comes with many challenging restrictions: particular focus on speed and latency, having to cater to all sorts of browsers and work under various circumstances, and… yes, surprise and delight.

I’m talking about Google doodles, the special illustrations that occasionally replace our logo. And while my relationship with pens and brushes has long had that distinctive flavour of a restraining order, I often contribute to the interactive ones.

Every interactive doodle I coded (Pac-Man, Jules Verne, World’s Fair)– and many I helped with – were in equal parts futuristic and anachronistic: great opportunities for pie-in-the-sky applications of cutting-edge Web features… and gritty pragmatism of cross-browser compatibility.

We learn a lot from each interactive doodle, and the recent Stanisław Lem mini-game was no exception, with its 17,000 lines of JavaScript code trying many things for the first time in doodle history. Today, I want to share that code with you – perhaps you'll find something interesting there, or point out my mistakes – and talk a bit about it.

View Stanisław Lem doodle code »

A thing worth keeping in mind is that Google's homepage is not a place for tech demos. With our doodles, we want to celebrate specific people and events, and we want to do that using the best art and the best technologies we can summon – but never celebrate technology for technology's sake. This means carefully looking at whatever part of broadly understood HTML5 is available, and whether it helps us make the doodle better without distracting from it or overshadowing it.

So, let's go through some of the modern Web technologies that found their place – and some that didn't – in the Stanisław Lem doodle.

Graphics via DOM and canvas #

Canvas is powerful and created for exactly the kind of things we wanted to do in this doodle. However, some of the older browsers we cared about didn't support it – and even though I am literally sharing an office with the person who put together an otherwise excellent excanvas, I decided to choose a different way.

I put together a graphic engine that abstracts away graphic primitives called “rects,” and then renders them using either canvas, DOM if canvas is unavailable.

This approach comes with some interesting challenges – for example, moving or changing an object in DOM has immediate consequences, whereas for canvas there’s a specific moment when everything’s drawn at the same time. (I decided to have just one canvas, and clear it and draw from scratch with every frame. Too many, literally, moving parts on one hand – and on the other not enough complexity to warrant splitting into multiple overlapping canvases and updating them selectively.)

Unfortunately, switching to canvas is not as simple as just mirroring CSS backgrounds with drawImage(): you lose a number of things that come for free when putting things together via DOM – most importantly layering with z-indexes, and mouse events.

I already abstracted away the z-index with a concept called “planes.” The doodle defined a number of plane – from the sky far behind, to the mouse pointer in front of everything – and every actor within the doodle had to decide which one it belonged to (small plus/minus corrections within a plane were possible by using planeCorrection).

Rendering through DOM, the planes are simply translated into z-index. But if we render via canvas, we need to sort rects based on their planes before drawing them. Since it’s costly to do this every time, the order is recalculated only when an actor is added or when it moves to another plane.

For mouse events, I abstracted that too… sort of. For both DOM and canvas, I used additional completely transparent floating DOM elements with high z-index, whose function is only to react to mouse over/out, clicks and taps.

One of the things we wanted to try with this doodle was breaking the fourth wall. The above engine allowed us to combine canvas-based actors with DOM-based actors. For example, the explosions in the finale are both in canvas for in-universe objects and in DOM for the rest of the Google homepage. The bird, normally flying around and clipped by our jagged mask like any other actor, decides to stay out of trouble during the shooting level, and sits on the I’m Feeling Lucky button. The way it’s done is for the bird to leave canvas and become a DOM element (and vice versa later), which I hoped to be completely transparent to our visitors.

The frame rate #

Knowing the current frame rate, and reacting to when it's too slow (and too fast!) was an important part of our engine. Since the browsers don't report back the frame rate, we have to calculate it ourselves.

I started with using requestAnimationFrame, falling back to the old-fashioned setTimeout if the former was not available. requestAnimationFrame cleverly saves the CPU in some situations – although we are doing some of that ourselves, as will be explained below – but also simply allows us to get a higher frame rate than setTimeout.

Calculating the current frame rate is simple, but is subject to drastic changes – for example it can drop quickly when another application hogs the computer for a while. Therefore, we calculate a “rolling” (averaged) frame rate only across every 100 physical ticks and make decisions based on that.

What kind of decisions?

If the frame rate is higher than 60fps, we throttle it. Currently, requestAnimationFrame on some versions of Firefox has no upper cap on the frame rate, and there's no point in wasting the CPU. Note that we actually cap at 65fps, because of the rounding errors that make the frame rate just a bit higher than 60fps on other browsers – we don't want do start throttling that by mistake.
If the frame rate is lower than 10fps, we simply slow down the engine instead of dropping frames. It’s a lose-lose proposition, but I felt that skipping frames excessively would be more confusing than simply having a slower (but still coherent) game. There’s another nice side effect of that – if the system gets slow temporarily, the user won’t experience a weird jump ahead as the engine is desperately catching up. (I did it slightly differently for Pac-Man, but the minimum frame rate is a better approach.)
Lastly, we can think of simplifying graphics when the frame rate gets dangerously low. We're not doing it for Lem doodle with the exception of mouse pointer (more on that below), but hypothetically we could lose some of the extraneous animations just so that the doodle feels fluid even on slower computers.

We also have a concept of a physical tick and a logical tick. The former comes from requestAnimationFrame/setTimeout. The ratio in normal gameplay is 1:1, but for fast-forwarding, we just add more logical ticks per a physical tick (up to 1:5). This allows us to do all the necessary calculations for every logical tick, but only designate the last one to be the one updating things on the screen.

Benchmarking #

An assumption can be (and indeed, early on, was) made that canvas will be faster than DOM whenever it's available. That is not always true. While testing, we found out that Opera 10.0–10.1 on a Mac, and Firefox on Linux are actually faster when moving DOM elements.

In the perfect world, the doodle would silently benchmark different graphic techniques – DOM elements moved using style.left and style.top, drawing on canvas, and maybe even DOM elements moved using CSS3 transforms

– and then switch to whichever one gave the highest frame rate. I started writing code for that, but found that at least my way of benchmarking was pretty unreliable and required a lot of time. Time that we don't have on our homepage – we care a lot about speed and we want the doodle to show up instantly and the gameplay to begin as soon as you click or tap.

In the end, Web development sometimes boils down to having to do what you gotta do. I looked behind my shoulder to make sure no one was looking, and then I just hard-coded Opera 10 and Firefox out of canvas. In the next life, I will come back as a <marquee> tag.

Conserving CPU #

You know that friend who comes to your house, watches the season finale of Breaking Bad, spoils it for you and then deletes it from your DVR? You don't want to be that guy, do you?

So, yes, the worst analogy ever. But we don't want our doodle to be that guy either – the fact we're allowed into someone's browser tab is a privilege, and hoarding CPU cycles or distracting the user would make us an unpleasant guest. Therefore, if no one’s playing with the doodle (no taps, mouse clicks, mouse movements, or key presses), we want it to eventually go to sleep.

When?

after 18 seconds on the homepage (arcade games called this the attract mode)
after 180 seconds if the tab has focus
after 30 seconds if the tab doesn’t have focus (e.g. the user switched to another window, but perhaps is still watching the doodle now in an inactive tab)
immediately if the tab becomes invisible (e.g. the user switched to another tab in the same window – no point in wasting cycles if we can't be seen)

How do we know the tab currently has focus? We attach ourselves to window.focus and window.blur How do we know the tab is visible? We’re using the new Page Visibility API and react to the appropriate event.

The time outs above are more forgiving than usual for us. I adapted them to this particular doodle, which has a lot of ambient animations (chiefly the sky and the bird). Ideally, the time outs would be gated on in-game interaction – e.g. right after landing, the bird could report back to the doodle that it can go to sleep now – but I didn't implement that in the end.

Since the sky is always in motion, when falling asleep and waking up the doodle doesn’t just stop or start – it slows down before pausing, and vice versa for resuming, increasing or decreasing the number logical ticks per a physical tick as necessary.

Transitions, transforms, events #

One of the powers of HTML has always been the fact you can make it better yourself: if something is not good enough in the regular portfolio of HTML and CSS, you can wrangle JavaScript into extending it. Unfortunately, it oftentimes means having to start from scratch. CSS3 transitions are great, but you cannot add a new transition type or use transitions to do anything else than styling elements. Another example: CSS3 transforms are great for DOM, but when you move to canvas, you're suddenly on your own.

These issues, and more, are why Lem doodle has its own transition and transform engine. Yeah, I know, 2000s called, etc. – the capabilities I built in are nowhere near as powerful as CSS3, but whatever the engine does, it does consistently and gives us much more control.

I started with a simple action (event) system – a timeline that fires events in the future without using setTimeout, since at any given point doodle time can become divorced from physical time as it gets faster (fast forward), slower (low frame rate or falling asleep to save CPU), or stops altogether (waiting for images to finish loading).

Transitions are just another type of actions. In addition to basic movements and rotation, we also support relative movements (e.g. move something 10 pixels to the right), custom things like shivering, and also keyframe image animations.

I mentioned rotations, and those are done manually too: we have sprites for various angles for the objects that need to be rotated. The main reason is that both CSS3 and canvas rotations were introducing visual artifacts that we found unacceptable – and on top of that, those artifacts varied per platform.

Given that some objects which rotate are attached to other rotating objects – one example is a robot's hand connected to the lower arm, which itself is attached to a rotating upper arm – this meant I also needed to create a poor man’s transform-origin in form of pivots.

All of this is a solid amount of work that ultimately covers the ground already taken care of by HTML5 – but sometimes native support is not good enough and it's that time for wheel reinvention.

Dealing with images and sprites #

An engine is not just for running the doodle – it's also for working on it. I shared some debug parameters above: you can find the rest in engine.readDebugParams.

Spriting is a well-known technique that we too use for doodles. It allows us to save bytes and decrease load times, plus it makes pre-loading easier. However, it also makes development harder – every change to imagery would require re-spriting (largely automated, but still cumbersome). Therefore, the engine supports running on raw images for development as well as sprites for production via engine.useSprites – both are included with the source code.

Sprites used by the Pac-Man doodle.

We also support pre-loading images as we go along and halting the doodle if the images didn’t load in time – complete with a faux progress bar! (Faux because, unfortunately, not even HTML5 can tell us how much of an image file has already been loaded.)

A screenshot of loading graphic with the rigged progress bar.

For some scenes, we use more than one sprite not as much to speed up loading using parallel connections, but simply because of 3/5 million pixel limitation for images on iOS.

Where does HTML5 fit into all this? There's not much of it above, but the tool I wrote for spriting/cropping was all new Web tech: canvas, blobs, a[download]. One of the exciting things about HTML is that it slowly subsumes things that previously had to be done outside of the browser; the only part we needed do there was optimizing PNG files.

Saving state in between games #

Lem's worlds always felt big and alive and realistic. His stories typically started without much in terms of explanation, the first page starting in medias res, with the reader having to find her or his way around.

The Cyberiad was no exception and we wanted to replicate that feeling for the doodle. We start with trying not to over-explain the story. Another large part is randomization which we felt befitted the mechanic nature of the universe of the book; we have a number of helper functions dealing with randomness that we use in many, many places.

We also wanted to increase replayability in other ways. For that, we needed to know how many times the doodle was finished before. The historically correct technological solution to that is a cookie, but that doesn't work for Google homepage – every cookie increases every page's payload, and again, we care quite a lot about speed and latency.

Fortunately, HTML5 gives us Web Storage, trivial in use, allowing us to save and recall the general play count and the last scene played by the user – with much more grace than cookies would ever allow for.

What do we do with this information?

we show a fast-forward button, allowing to zip through the cutscenes the user already saw before
we show different N items during the finale
we slightly increase the difficulty of the shooting level
we show a little easter egg probability dragon from a different story on your third and subsequent plays

There is a number of debug parameters controlling this:

?doodle-debug&doodle-first-run – pretend it’s a first run
?doodle-debug&doodle-second-run – pretend it’s a second run
?doodle-debug&doodle-old-run – pretend it’s an old run

Touch devices #

We wanted the doodle to feel right at home on touch devices – the most modern ones are powerful enough so that the doodle runs really well, and experiencing the game via tapping is so much more fun than with clicking.

Some upfront changes to the user experience needed to be made. Originally, our mouse pointer was the only place that communicated a cutscene/non-interactive part is taking place. We later added a little indicator in the lower-right corner, so we didn't have to rely on mouse pointer alone (given that those don't exist on touch devices).

	Normal	Busy	Clickable	Clicked
Work-in-progress
Final	v

Mouse pointers during development, and final equivalents.

Most of the stuff worked out of the box. However, quick impromptu usability tests of our touch experience showed two problems: some of the targets were too hard to press, and quick taps were ignored since we just overrode mouse click events.

Having separate clickable transparent DOM elements helped a lot here, as I could resize them independently of the visuals. I introduced extra 15-pixel padding for touch devices and used it whenever clickable elements were created. (I added 5-pixel padding for mouse environments too, just to make Mr. Fitts happy.)

As for the other problem, I just made sure to attach and test proper touch start and end handlers, instead of relying on mouse click.

We’re also using more modern style properties to remove some touch features that WebKit browsers add by default (tap highlight, tap callout).

And how do we detect whether a given device running the doodle supports touch? Lazily. Instead of figuring it out a priori, we just used our combined IQs to deduct that the device supports touch… after we get the first touch start event.

Customizing the mouse pointer #

But not everything is touch-based. One of our guiding principles was to put as many things as we could within the universe of the doodle. The little sidebar UI (fast-forward, question mark), the tooltip, and even, yes, the mouse pointer.

How to customize a mouse pointer? Some browsers allow changing the mouse cursor by linking to a bespoke image file. However, this is not supported well and it’s also somewhat restrictive.

If not this, then what? Well, why not making a mouse pointer just another actor in the doodle? This works, but comes with a number of caveats, chiefly:

you need to be able to remove the native mouse pointer
you need to be pretty good at keeping your mouse pointer in sync with the “real” one

The former is tricky. CSS3 allows for cursor: none, but it too is not supported in some browsers. We needed to resort to some gymnastics: using empty .cur file as a fallback, specifying concrete behavior for some browsers, and even hard-coding others out of the experience whatsoever.

The other is relatively trivial on its face, but with the mouse pointer being just another part of the universe of the doodle, it will inherit all its problems too. The biggest one? If the frame rate of the doodle is low, the frame rate of the mouse pointer will be low too – and that has dire consequences since the mouse pointer, being a natural extension of your hand, needs to feel responsive no matter what. (People who used Commodore Amiga in their past are now nodding vigorously.)

One somewhat complex solution to that problem is decoupling the mouse pointer from the regular update loop. We did just that – in an alternate universe where I don't need to sleep. A simpler solution for this one? Just reverting to the native mouse pointer if the rolling frame rate drops below 20fps. (This is where the rolling frame rate comes in handy. If we reacted to the current frame rate, and if it happened to oscillate around 20fps, the user would see the custom mouse pointer hiding and showing all the time.) This brings us to:

Frame rate range	Behaviour
>10fps	Slow down the game so that more frames are not dropped.
10–20fps	Use native mouse pointer instead of custom one.
20–60fps	Normal operation.
>60fps	Throttle so that the frame rate doesn't exceed this value.

Summary of frame rate-dependent behaviour.

Oh, and our mouse pointer is dark on a Mac, but white on a PC. Why? Because platform wars need fuel even in fictional universes.

Conclusion #

This is not a perfect engine, but it doesn't try to be one. It was developed alongside the Lem doodle, and is very specific to it. That's okay. “Premature optimization is the root of all evil,” as Don Knuth famously said, and I don't believe writing an engine in isolation first, and only applying it later makes sense – the practice informs theory just as much as theory informs practice. In my case, code was thrown away, several parts rewritten over and over again, and many common pieces noticed post, rather than ante factum. But in the end, what we have here allowed us to do what we wanted – celebrate the career of Stanisław Lem and the drawings by Daniel Mróz in the best way we could think of.

I hope the above sheds light on some of the design choices and trade-offs that we needed to make – and how we used HTML5 in a specific, real-life scenario. Now, play with the source code, take it for the spin, and let us know what you think.

I did that myself – this below was live in the last days, counting down to the early hours of the 23th of November 2011 in Russia, which was the first time zone that saw the Lem doodle. A goofy thing, perhaps, but just like doodles, things that appear insignificant sometimes have a deeper meaning – this counter was really a nice “stress test” for the engine.

A screenshot of Lem doodle in-universe countdown clock.

And that's one way of looking at the life of a Google doodle – months of work, weeks of testing, 48 hours of baking it in, all for something that people play for five minutes. Every one of those thousands of JavaScript lines is hoping that those 5 minutes will be time well spent. Enjoy.

Marcin Wichary on web.dev

Jumping the hurdles with the Gamepad API

Introduction #

Gamepad tester #

What browsers support it today? #

What gamepads can be used? #

Feature Detecting the Gamepad API< #

Finding out about connected gamepads #

Polling #

Events #

Summary #

Gamepad info #

Basic info #

Buttons and sticks #

Analogue or discrete? #

Button presses and stick movements #

Events #

Polling #

The keyboard-first approach in the Hurdles 2012 doodle #

Tips and tricks #

The Future #

Further reading #

Case Study - Building the Stanisław Lem Google doodle

Hello, (strange) world #

Graphics via DOM and canvas #

The frame rate #

Benchmarking #

Conserving CPU #

Transitions, transforms, events #

Dealing with images and sprites #

Saving state in between games #

Touch devices #

Customizing the mouse pointer #

Conclusion #