Skip to main content

Testing Levels

Table Of Contents ​

Unit Tests​

Unit tests are the most granular level of testing, focusing on individual functions or methods within a module or module subcomponent. These tests are used to verify that each unit of code behaves as expected in isolation.

Module Integration Tests​

Module integration tests focus on testing the interaction between modules in the appchain without mocking individual components. This level of testing ensures that cross-module interactions can be exercised but without the overhead of the full appchain.

Unit Test Example​

// TODO_DOCUMENT(@bryanchriswhite): Add example

Unit Test - Good Fit​

  • Exercising a Keeper method
  • Code has dependencies on other module Keepers

Unit Test - Bad Fit​

  • Test depends on network events
  • Test depends on Tx assertions

Unit Test - Limitations​

  • No transactions
  • No events
  • No message server

App Integration Tests​

App integration tests focus on testing the behavior of the fully integrated appchain from a common message server interface. This level of testing ensures message handling logic is exercise while fully integrated with cosmos-sdk but without the overhead of the cometbft engine and networking.

Integration Test Example​

// TODO_DOCUMENT(@bryanchriswhite): Add example

NOTE: See App Integration Suites for organizing larger or higher-level app integration tests.

Integration Test - Good Fit​

  • Exercising a user story involving multiple messages
  • Exercising a scenario involving multiple messages
  • Exercising cross-module dependencies & interactions
  • Asserting against a new integrated state

Integration Test - Bad Fit​

  • Code under test depends/asserts on networking operations
  • Code under test depends/asserts on consensus operations
  • Code under test requires setup which would be simpler to do with direct keeper interaction.

Integration Test - Limitations​

  • No networking
  • No consensus
  • No keeper API access (intentional)

In-Memory Network Integration Tests​

In-memory network integration tests focus on testing the behavior of a multi-validator network from the perspective of the ABCI message interface. This level of testing ensures that the appchain behaves as expected in a multi-validator environment.

In-Memory Network Example​

// TODO_DOCUMENT(@bryanchriswhite): Add example

In-Memory Network - Good Fit​

  • Exercising CometBFT RPC
  • Exercising consensus scenarios
  • Exercising multi-validator scenarios
  • Integrating with external tools via network

In-Memory Network - Bad Fit​

  • Most cases; use sparingly
  • Prefer other levels unless it's clearly appropriate

In-Memory Network - Limitations​

  • No parallelization
  • No Keeper module access
  • Depends on cosmos-sdk APIs (less customizable)
  • Slow startup time (per network).

End-to-End Tests​

End-to-end tests focus on testing the behavior of a network containing both on- and off-chain actors; typically exercising "localnet".

E2E Test Example​

// TODO_DOCUMENT(@bryanchriswhite): Add example

E2E Test - Good Fit​

  • Asserts or dependent on off-chain assertions
  • Asserts or dependent on off-chain actors
  • Asserts or dependent on off-chain behavior

E2E Test - Bad Fit​

  • Scenarios which require many blocks/sessions to complete
  • Scenarios which are not idempotent
  • Scenarios which assume specific/complex network states

E2E Test - Limitations​

  • Depends on LocalNet to be running and healthy
  • Depends on other environments (DevNet/TestNet) to be running and healthy
  • Shared mutable network state on-chain
  • Shared mutable network state off-chain
  • Intolerant of non-idempotent operations (CI re-runnability).