How to write tests

This guide explains how we structure, write, and run visual regression tests using BackstopJS. It merges and streamlines our previous docs.

Overview

We use BackstopJS to:

capture screenshots of pages/components,
compare them against a reference,
and highlight unintended visual changes early in development.

Our test assets live in a project‑root test/ folder. The most important files are:

test/backstop.config.js — our main BackstopJS configuration
test/scenarios/*.test.json — individual scenario definitions
test/backstop_data/engine_scripts/puppet/onReady.js — script that eagerly loads lazy images and runs helpers

If backstop.json is missing, you can bootstrap defaults with npx backstop init — but our setup uses backstop.config.js as the single source of truth.

Folder Structure

project/
├─ src/ …
├─ blocks/ …
├─ test/
│  ├─ backstop.config.js
│  ├─ scenarios/
│  │  └─ example.test.json
│  └─ backstop_data/
│     ├─ bitmaps_reference/
│     ├─ bitmaps_test/
│     ├─ ci_report/
│     ├─ html_report/
│     └─ engine_scripts/
│        └─ puppet/
│           ├─ onBefore.js
│           └─ onReady.js
└─ package.json

Engine Script: `onReady.js`

We force‑load images that use lazy/async hints so screenshots are consistent across runs.

module.exports = async (page, scenario) => {
    console.log('SCENARIO > ' + scenario.label);

    await page.evaluate(() => {
        document.querySelectorAll('[loading="lazy"]').forEach((el) => {
            el.loading = 'eager';
        });

        document.querySelectorAll('[decoding="async"]').forEach((el) => {
            el.decoding = 'sync';
        });
    });

    await require('./clickAndHoverHelper')(page, scenario);
    // add more ready handlers here...
};

Backstop Config (`test/backstop.config.js`)

Global defaults live here. Scenarios are auto‑loaded from test/scenarios and shallow‑merged with the base scenario.

const fs = require('fs');
const path = require('path');

const BASE_URL =
    process.env.BACKSTOP_TEST_URL ||
    'http://everyday-base-theme.local/komponentenubersicht/';

// Base configuration with global defaults
const baseScenario = {
    cookiePath: 'backstop_data/engine_scripts/cookies.json',
    url: BASE_URL,
    referenceUrl: '',
    readyEvent: '',
    readySelector: '',
    delay: 2000,
    hideSelectors: [],
    removeSelectors: [],
    hoverSelector: '',
    clickSelector: '',
    postInteractionWait: 0,
    selectorExpansion: true,
    expect: 0,
    misMatchThreshold: 0.1,
    requireSameDimensions: true,
};

// Load and extend scenarios from /test/scenarios
function loadScenarios(dir) {
    return fs
        .readdirSync(dir)
        .filter((file) => file.endsWith('.json'))
        .map((file) => {
            const scenario = require(path.join(__dirname, dir, file));
            return { ...baseScenario, ...scenario };
        });
}

module.exports = {
    id: 'backstop_default',
    viewports: [
        { label: 'phone', width: 320, height: 480 },
        { label: 'tablet', width: 1024, height: 768 },
        { label: 'desktop', width: 1920, height: 1080 },
        { label: 'desktop-large', width: 2560, height: 1440 },
    ],
    onBeforeScript: 'puppet/onBefore.js',
    onReadyScript: 'puppet/onReady.js',
    scenarios: loadScenarios('scenarios'),
    paths: {
        bitmaps_reference: 'backstop_data/bitmaps_reference',
        bitmaps_test: 'backstop_data/bitmaps_test',
        engine_scripts: 'backstop_data/engine_scripts',
        html_report: 'backstop_data/html_report',
        ci_report: 'backstop_data/ci_report',
    },
    report: ['browser'],
    engine: 'puppeteer',
    engineOptions: { args: ['--no-sandbox'] },
    asyncCaptureLimit: 5,
    asyncCompareLimit: 50,
    debug: false,
    debugWindow: false,
};

Writing a Scenario

Create a file in test/scenarios/ with a meaningful label and one or more selectors. We typically prefix component hooks with bs- in markup.

{
    "label": "component-name",
    "selectors": [".bs-component-name"]
}

<!-- Example markup hook in your dynamic PHP -->
<div class="bs-component-name">
  <!-- Component HTML -->
</div>

You can also target full pages by omitting selectors (Backstop will default to document), or capture multiple regions by adding more selectors.

Commands

These are the standard scripts we use. Add them to your package.json.

{
    "scripts": {
        "build:test": "NODE_ENV=production wp-scripts build src/js/index.js blocks/utilities/**/*.jsx blocks/components/**/*.jsx && cd test && backstop test --configPath=backstop.config.js",
        "test:reference": "cd test && backstop reference --configPath=backstop.config.js",
        "test:approve": "cd test && backstop approve --configPath=backstop.config.js",
        "test": "cd test && backstop test --configPath=backstop.config.js"
    }
}

Typical workflow

Create/Update scenarios in test/scenarios.
Generate references (first run or after intentional UI changes):
Terminal window
```
pnpm run test:reference
```
Run tests:
Terminal window
```
pnpm run test
```
Approve expected diffs (only if changes are desired):
Terminal window
```
pnpm run test:approve
```

You can also run Backstop directly: cd test && backstop test --configPath=backstop.config.js.

Best Practices

Stable fixtures: Ensure data and network calls are deterministic (mock if needed).
Consistent states: Use onBefore/onReady to open menus, set cookies, or log in.
Explicit hooks: Add .bs-* classes for robust, layout‑independent selectors.
Viewport coverage: Keep viewports minimal but meaningful; avoid over‑testing.
Small thresholds: Prefer misMatchThreshold: 0.1 or lower; raise only when justified.

Troubleshooting

Element not found for capturing: Verify the selector exists at test time, ensure delay is sufficient, or use readySelector.
Blank/partial images: Ensure lazy images are eagerly loaded (see onReady.js).
Font or anti‑aliasing noise: Consider requireSameDimensions: true and keep the same OS/rendering stack in CI.
Wrong base URL: Set BACKSTOP_TEST_URL to the correct environment.