Jecelyn Yeen on web.dev

Four common types of code coverage

2023-09-06T00:00:00Z

Have you heard the phrase "code coverage"? In this post, we will explore what code coverage in tests is and four common ways to measure it.

What is code coverage? #

Code coverage is a metric that measures the percentage of source code your tests execute. It helps you identify areas that may lack proper testing.

Often, recording these metrics looks like this:

File	% Statements	% Branch	% Functions	% Lines	Uncovered lines
file.js	90%	100%	90%	80%	89,256
coffee.js	55.55%	80%	50%	62.5%	10-11, 18

As you add new features and tests, increasing code coverage percentages can give you more confidence that your application has been thoroughly tested. However, there is more to discover.

Four common types of code coverage #

There are four common ways to collect and calculate code coverage: function, line, branch, and statement coverage.

To see how each type of code coverage calculates its percentage, consider the following code example for calculating coffee ingredients:

/* coffee.js */

export function calcCoffeeIngredient(coffeeName, cup = 1) {
  let espresso, water;

  if (coffeeName === 'espresso') {
    espresso = 30 * cup;
    return { espresso };
  }

  if (coffeeName === 'americano') {
    espresso = 30 * cup; water = 70 * cup;
    return { espresso, water };
  }

  return {};
}

export function isValidCoffee(name) {
  return ['espresso', 'americano', 'mocha'].includes(name);
}

The tests that verify the calcCoffeeIngredient function are:

/* coffee.test.js */

import { describe, expect, assert, it } from 'vitest';
import { calcCoffeeIngredient } from '../src/coffee-incomplete';

describe('Coffee', () => {
  it('should have espresso', () => {
    const result = calcCoffeeIngredient('espresso', 2);
    expect(result).to.deep.equal({ espresso: 60 });
  });

  it('should have nothing', () => {
    const result = calcCoffeeIngredient('unknown');
    expect(result).to.deep.equal({});
  });
});

You can run the code and tests on this live demo or check out the repository.

Function coverage #

Code coverage: 50%

/* coffee.js */

export function calcCoffeeIngredient(coffeeName, cup = 1) {
  // ...
}

function isValidCoffee(name) {
  // ...
}

Function coverage is a straightforward metric. It captures the percentage of functions in your code that your tests call.

In the code example, there are two functions: calcCoffeeIngredient and isValidCoffee. The tests only call the calcCoffeeIngredient function, so the function coverage is 50%.

Line coverage #

Code coverage: 62.5%

/* coffee.js */

export function calcCoffeeIngredient(coffeeName, cup = 1) {
  let espresso, water;

  if (coffeeName === 'espresso') {
    espresso = 30 * cup;
    return { espresso };
  }

  if (coffeeName === 'americano') {
    espresso = 30 * cup; water = 70 * cup;
    return { espresso, water };
  }

  return {};
}

export function isValidCoffee(name) {
  return ['espresso', 'americano', 'mocha'].includes(name);
}

Line coverage measures the percentage of executable code lines that your test suite executed. If a line of code remains unexecuted, it means that some part of the code hasn't been tested.

The code example has eight lines of executable code (highlighted in red and green) but the tests don’t execute the americano condition (two lines) and the isValidCoffee function (one line). This results in a line coverage of 62.5%.

Note that line coverage doesn’t take into account declaration statements, such as function isValidCoffee(name) and let espresso, water;, because they are not executable.

Branch coverage #

Code coverage: 80%

/* coffee.js */

export function calcCoffeeIngredient(coffeeName, cup = 1) {
  // ...

  if (coffeeName === 'espresso') {
    // ...
    return { espresso };
  }

  if (coffeeName === 'americano') {
    // ...
    return { espresso, water };
  }

  return {};
}
…

Branch coverage measures the percentage of executed branches or decision points in the code, such as if statements or loops. It determines whether tests examine both the true and false branches of conditional statements.

There are five branches in the code example:

Calling calcCoffeeIngredient with just coffeeName
Calling calcCoffeeIngredient with coffeeName and cup
Coffee is Espresso
Coffee is Americano
Other coffee

The tests cover all branches except the Coffee is Americano condition. So branch coverage is 80%.

Statement coverage #

Code coverage: 55.55%

/* coffee.js */

export function calcCoffeeIngredient(coffeeName, cup = 1) {
  let espresso, water;

  if (coffeeName === 'espresso') {
    espresso = 30 * cup;
    return { espresso };
  }

  if (coffeeName === 'americano') {
    espresso = 30 * cup; water = 70 * cup;
    return { espresso, water };
  }

  return {};
}

export function isValidCoffee(name) {
  return ['espresso', 'americano', 'mocha'].includes(name);
}

Statement coverage measures the percentage of statements in your code that your tests execute. At first glance, you might wonder, “isn’t this the same as line coverage?” Indeed, statement coverage is similar to line coverage but takes into account single lines of code that contain multiple statements.

In the code example, there are eight lines of executable code, but there are nine statements. Can you spot the line containing two statements?

Check your answer

It's the following line: `espresso = 30 * cup; water = 70 * cup;`

The tests cover only five of the nine statements, therefore the statement coverage is 55.55%.

If you always write one statement per line, your line coverage will be similar to your statement coverage.

What type of code coverage should you choose? #

Most code coverage tools include these four types of common code coverage. Choosing which code coverage metric to prioritize depends on specific project requirements, development practices, and testing goals.

In general, statement coverage is a good starting point because it is a simple and easy-to-understand metric. Unlike statement coverage, branch coverage and function coverage measure whether tests call a condition (branch) or a function. Therefore, they are a natural progression after statement coverage.

Once you have achieved high statement coverage, you can then move on to branch coverage and function coverage.

Is test coverage the same as code coverage? #

No. Test coverage and code coverage are often confused but they are different:

Test coverage: A qualitative metric that measures how well the test suite covers the features of the software. It helps determine the level of risk involved.
Code coverage: A quantitative metric that measures the proportion of code executed during testing. It is about how much code the tests cover.

Here is a simplified analogy: imagine a web application as a house.

Test coverage measures how well the tests cover the rooms in the house.
Code coverage measures how much of the house the tests have walked through.

100% code coverage doesn’t mean no bugs #

While it is certainly desirable to achieve high code coverage in testing, 100% code coverage doesn’t guarantee the absence of bugs or flaws in your code.

A meaningless way to achieve 100% code coverage #

Consider the following test:

/* coffee.test.js */

// ...
describe('Warning: Do not do this', () => {
  it('is meaningless', () => { 
    calcCoffeeIngredient('espresso', 2);
    calcCoffeeIngredient('americano');
    calcCoffeeIngredient('unknown');
    isValidCoffee('mocha');
    expect(true).toBe(true); // not meaningful assertion
  });
});

This test achieves 100% function, line, branch, and statement coverage, but it doesn’t make sense because it doesn’t actually test the code. The expect(true).toBe(true) assertion will always pass regardless of whether the code works correctly.

A bad metric is worse than no metric #

A bad metric can give you a false sense of security, which is worse than having no metric at all. For example, if you have a test suite that achieves 100% code coverage but the tests are all meaningless, then you may get a false sense of security that your code is well tested. If you accidentally delete or break a part of the application code, the tests will still pass, even though the application no longer works correctly.

To avoid this scenario:

Test review. Write and review tests to make sure they are meaningful and test the code in a variety of different scenarios.
Use code coverage as a guideline, not as the only measure of test effectiveness or code quality.

Using code coverage in different types of testing #

Let’s take a closer look at how you can use code coverage with the three common types of test:

Unit tests. They are the best test type for gathering code coverage because they are designed to cover multiple small scenarios and testing paths.
Integration tests. They can help collect code coverage for integration tests, but use them with caution. In this case, you calculate the coverage of a larger portion of the source code, and it can be difficult to determine which tests actually cover which parts of the code. Nonetheless, calculating code coverage of integration tests may be useful for legacy systems that don’t have well-isolated units.
End-to-end (E2E) tests. Measuring code coverage for E2E tests is difficult and challenging due to the intricate nature of these tests. Instead of using code coverage, requirement coverage might be the better way to go. This is because the focus of E2E tests is to cover the requirements of your test, not to focus on the source code.

Conclusion #

Code coverage can be a useful metric for measuring the effectiveness of your tests. It can help you to improve the quality of your application by ensuring that the crucial logic in your code is well tested.

However, remember that code coverage is just one metric. Make sure to also consider other factors, such as the quality of your tests and your application requirements.

Aiming for 100% code coverage is not the goal. Instead, you should use code coverage along with a well-rounded testing plan that incorporates a variety of testing methods, including unit tests, integration tests, end-to-end tests, and manual tests.

See the full code example and tests with good code coverage. You can also run the code and tests with this live demo.

/* coffee.js - a complete example */

export function calcCoffeeIngredient(coffeeName, cup = 1) {
  if (!isValidCoffee(coffeeName)) return {};

  let espresso, water;

  if (coffeeName === 'espresso') {
    espresso = 30 * cup;
    return { espresso };
  }

  if (coffeeName === 'americano') {
    espresso = 30 * cup; water = 70 * cup;
    return { espresso, water };
  }

  throw new Error (`${coffeeName} not found`);
}

function isValidCoffee(name) {
  return ['espresso', 'americano', 'mocha'].includes(name);
}

/* coffee.test.js - a complete test suite */

import { describe, expect, it } from 'vitest';
import { calcCoffeeIngredient } from '../src/coffee-complete';

describe('Coffee', () => {
  it('should have espresso', () => {
    const result = calcCoffeeIngredient('espresso', 2);
    expect(result).to.deep.equal({ espresso: 60 });
  });

  it('should have americano', () => {
    const result = calcCoffeeIngredient('americano');
    expect(result.espresso).to.equal(30);
    expect(result.water).to.equal(70);
  });

  it('should throw error', () => {
    const func = () => calcCoffeeIngredient('mocha');
    expect(func).toThrowError(new Error('mocha not found'));
  });

  it('should have nothing', () => {
    const result = calcCoffeeIngredient('unknown')
    expect(result).to.deep.equal({});
  });
});

What are source maps?

2023-03-31T00:00:00Z

Today, we are talking about source maps, a crucial tool in modern web development that makes debugging significantly easier. In this article, we will explore the basics of source maps, how they are generated, and how they improve the debugging experience.

The need for source maps #

Back in the good old days, we built web applications with pure HTML, CSS, and JavaScript and deployed the same files to the web.

However, as we are building more complex web applications nowadays, your development workflow may involve using various tools. For example:

Templating languages and HTML preprocessors: Pug, Nunjucks, Markdown.
CSS preprocessors: SCSS, LESS, PostCSS.
JavaScript frameworks: Angular, React, Vue, Svelte.
JavaScript meta frameworks: Next.js, Nuxt, Astro.
High-level programming languages: TypeScript, Dart, CoffeeScript.
And more. The list can go on and on!

These tools require a build process to transpile your code into standard HTML, JavaScript, and CSS that browsers can understand. Additionally, to optimize performance, it's common practice to compress (for example, using Terser to minify and mangle JavaScript) and combine these files, reducing their size and making them more efficient for the web.

For example, using build tools, we can transpile and compress the following TypeScript file into a single line of JavaScript. You can play with the demo in my GitHub repository.

/* A TypeScript demo: example.ts */

document.querySelector('button')?.addEventListener('click', () => {
  const num: number = Math.floor(Math.random() * 101);
  const greet: string = 'Hello';
  (document.querySelector('p') as HTMLParagraphElement).innerText = `${greet}, you are no. ${num}!`;
  console.log(num);
});

A compressed version would be:

/* A compressed JavaScript version of the TypeScript demo: example.min.js  */

document.querySelector("button")?.addEventListener("click",(()=>{const e=Math.floor(101*Math.random());document.querySelector("p").innerText=`Hello, you are no. ${e}!`,console.log(e)}));

However, this optimization can make debugging more challenging. Compressed code with everything in a single line and shorter variable names can make it difficult to pinpoint the source of an issue. That's where source maps come in—they map your compiled code back to the original code.

Generating source maps #

Source maps are files with names ending with .map (for example, example.min.js.map and styles.css.map). They can be generated by most build tools, for example, Vite, webpack, Rollup, Parcel, esbuild, and more.

Some tools include source maps by default, while others may need additional configuration to produce them.

/* Example configuration: vite.config.js */
/* https://vitejs.dev/config/ */

export default defineConfig({
  build: {
    sourcemap: true, // enable production source maps
  },
  css: {
    devSourcemap: true // enable CSS source maps during development
  }
})

Understanding the source map #

These source map files contain essential information about how the compiled code maps to the original code, enabling developers to debug with ease. Here is an example of a source map.

{
  "mappings": "AAAAA,SAASC,cAAc,WAAWC, ...",
  "sources": ["src/script.ts"],
  "sourcesContent": ["document.querySelector('button')..."],
  "names": ["document","querySelector", ...],
  "version": 3,
  "file": "example.min.js.map"
}

To understand each of these fields, you can read the source map specification or this classic article on the anatomy of a source map.

The most critical aspect of a source map is the mappings field. It uses a VLQ base 64 encoded string to map lines and locations in the compiled file to the corresponding original file. This mapping can be visualized using a source map visualizer like source-map-visualization and Source Map Visualization.

The image shows the visualization of our previous code example above, generated by a visualizer.

The generated column on the left shows the compressed content and the original column shows the original source.

The visualizer color codes each line in the original column and its corresponding code in the generated column.

The mappings section shows the decoded mappings of the code. For example, the entry 65-> 2:2 means:

Generated code: The word const starts at position 65 in the compressed content.
Original code: The word const starts at line 2 and column 2 in the original content.

This way, developers can quickly identify the relationship between the minified code and the original code, making debugging a smoother process.

Browser Developer Tools apply these source maps to help you pinpoint your debugging issues quicker, right in the browsers.

The image shows how browser Developer Tools applies source maps and shows the mappings between files.

Source map extensions #

Source maps support extensions. Extensions are custom fields that start with the x_ naming convention. One example is the x_google_ignoreList extension field proposed by Chrome DevTools. See x_google_ignoreList to learn more on how these extensions help you focus on your code.

It's not perfect #

In our example, the variable greet was optimized away during the build process. The value was directly embedded into the final string output.

In this case, when you debug the code, developer tools might not be able to infer and display the actual value. This isn't just the challenges for browser's developer tools. It also makes code monitoring and analysis harder.

This of course is a solvable problem. One of the ways is to include the scope information in the source maps like other programming languages do with their debug information.

However, this requires the whole ecosystem to work together to improve the source maps specification and implementation. There is an active discussion on improving the debuggability with source maps.

We look forward to improving source maps and making debugging even less overwhelming!