Model-based testing

Docs for XState v4 are no longer maintained

This page is no longer maintained as XState V5 has been released.

Read about testing in XState V5.

Adopting model-based testing can lead to self-documenting, easy-to-maintain tests which are far more DRY than regular tests.

You create a visual model, tell it how to interact with your app, and execute the model to test your app automatically.

Imperative testing

If you’ve done any automated application testing before, you’ve likely come across code like this:

describe('Search box', () => {
  it('Should let you search for a page and go there', () => {
    // Visit the home page
    cy.visit('/');

    // Search for ‘Model-based testing‘ in the search box
    cy.findByPlaceholderText('Search').type('Model-based testing');

    // Ensure the text is visible
    cy.findByText('Model-based testing').should('be.visible');

    // Click on the result that’s shown
    cy.findByText('Model-based testing').click();

    // Assert the URL has changed
    cy.url().should('include', '/model-based-testing/intro');
  });
});

In the example above, we’re using Cypress syntax. The test visits the homepage, searches for an item, clicks on that item, then asserts that the URL has changed. We’ll call this style of testing imperative testing. You’re telling the test suite exactly what to do and in what order.

You might model this flow with a statechart as follows:

import { createTestMachine } from '@xstate/test';

const machine = createTestMachine({
  initial: 'onHomePage',
  states: {
    onHomePage: {
      on: {
        SEARCH_FOR_MODEL_BASED_TESTING: 'searchResultsVisible',
      },
    },
    searchResultsVisible: {
      on: {
        CLICK_MODEL_BASED_TESTING_RESULT: 'onModelBasedTestingPage',
      },
    },
    onModelBasedTestingPage: {},
  },
});

In the example above, we use three states to represent the different states the application can be in, and transition between them using events.

If we were to run a test using this model, the sequence would be as follows:

1. Check we’re in state onHomePage

cy.visit('/');

2. Perform event SEARCH_FOR_MODEL_BASED_TESTING

cy.findByPlaceholderText('Search').type('Model-based testing');

3. Check we’re in state searchResultsVisible

cy.findByText('Model-based testing').should('be.visible');

4. Perform event CLICK_MODEL_BASED_TESTING_RESULT

cy.findByText('Model-based testing').click();

5. Check we’re in state onModelBasedTestingPage

cy.url().should('include', '/model-based-testing/intro');

Each of the assertions and actions in the original test map neatly onto this model.

This approach is model-based testing. You can create a model representing the part of your app you want to test, and use it to execute your tests.

Many tests, one model

The benefits of model-based testing over imperative testing come when you need to test multiple, complex paths in your app.

Imagine if we wanted to add a new test for “when we press ‘escape’ with the search box open, the box closes.” With our initial test suite, we’d need to add a new test:

describe('Search box', () => {
  it('Should let you search for a page and go there', () => {
    // ...
  });

  it('Should close when you press ESC', () => {
    // Visit the home page
    cy.visit('/');

    // Search for ‘Model-based testing’ in the search box
    cy.findByPlaceholderText('Search').type('Model-based testing');

    // Ensure the text is visible
    cy.findByText('Model-based testing').should('be.visible');

    // Press escape
    cy.realPress('{escape}');

    // Ensure the text is NOT visible
    cy.findByText('Model-based testing').should('not.be.visible');
  });
});

Note how much code is duplicated between this test and the previous test. The first three steps of “visit home page,” “type in search box,” and “ensure text is visible” are exactly the same. You might extract the code to a setupSearchBoxTest() function, but that would make the tests hard to decipher.

In our model-based test, you just add a new state and event to the model:

import { createTestMachine } from '@xstate/test';

const machine = createTestMachine({
  initial: 'onHomePage',
  states: {
    onHomePage: {
      on: {
        SEARCH_FOR_MODEL_BASED_TESTING: 'searchResultsVisible',
      },
    },
    searchResultsVisible: {
      on: {
        CLICK_MODEL_BASED_TESTING_RESULT: 'onModelBasedTestingPage',
        PRESS_ESCAPE: 'searchBoxClosed',
      },
    },
    searchBoxClosed: {},
    onModelBasedTestingPage: {},
  },
});

Now, there are two possible test paths for the model:

• Aim for searchBoxClosed
• Aim for onModelBasedTestingPage

We then just need to add instructions to our model on how to:

Press escape:

cy.realPress('{escape}');

And check we’re in the searchBoxClosed state:

cy.findByText('Model-based testing').should('not.be.visible');

Summary

You might be starting to see the power of model-based testing. You can create a visual model of your app using our Stately Studio editor. You can tell the model how to interact with your application; we’ll look at the syntax in depth later. Finally, you can let the model test your app for you. It’ll calculate the fewest possible number of test paths through your model, then run the tests for you.

Adopting model-based testing leads to a test suite that is:

• Self-documenting: you can inspect the visual models, give them descriptions, and share them with non-devs.
• DRY: you write the minimum amount of setup code for each test.
• Easy to maintain: adding new tests becomes as easy as adding a new state or event to an existing model.

This approach also works with any existing test framework, alongside your existing test suite.