Use scripted tests when you want to reuse tests or require a clear track record of covered areas:
For regression testing, write scripted tests so they can be automated. Then you can enable efficient and repeatable testing of existing functionalities to avoid regression from updates to the system.
For regulatory or compliance testing, execute scripted tests to get records of specific coverage.
While scripted tests can offer advantages in certain scenarios, you should also consider other testing techniques, such as exploratory testing, which allow for more flexibility and creativity in uncovering unexpected issues. Select appropriate testing techniques depending on your specific testing objectives and context.
Test Types
Tests can be either function- or scenario-based.
Function-Based Tests
These tests should focus on one logical function or functionality at a time for the following benefits:
Effective targeting of coverage
Effective Regression Testing: Function-based tests enable regression coverage targeted at a specific function that's updated. Instead of executing large test runs covering multiple functionalities, you can focus solely on the affected functions, ensuring faster and more efficient testing.
Clear Tester Focus: Function-based tests cover a specific functionality in depth, providing a clear focus without distracting testers by other factors.
Alternation in Coverage: Creating various tests for a single underlying function means you can alternate coverage from run to run while remaining confident the core underlying function is covered.
Clearer reporting of the outcome
With function-based tests, when a test fails, it becomes easier to pinpoint the exact function detail causing the failure.
Scenario-Based Tests
These tests should focus on one logical user/business flow. Their goal is to verify the integration of functions.
The user/business flow represents the path taken by an app user or a business process that is triggered by certain conditions.
Scenario-based testing can include several different functions (for example, a scenario for a money transfer app could include: signing in, topping up a digital wallet, sending funds).
Using Function- and Scenario-Based Tests
Fused Testing (Enabling Automation)
Fallback to Manual Testing in Fused Testing Flows: All automated tests must have parity with scripted tests so manual tests can be run when automated tests can’t (for example, flaky tests, problems with the automation environment). Clearly written function- or scenario-based tests make it easier to not only automate such tests, but also seamlessly fall back to manual tests. This is a core principle for our underlying concept of fused testing.
Not all tests are a fit for automation right away because of complex implementation or potential instability for such automated tests. But if scripted tests are written in a standardized way, they can be automated as soon as such impediments are cleared.
Combining Function- and Scenario-Based Checks
Don’t include function-based verifications in scenario-based tests as this can lead to:
Misleading result reports (such as a minor function-level check failing an entire scenario with a pass/fail test)
A loss of focus for the tester
Increased maintenance overhead
Likewise, don’t rely on combining multiple function-based tests into a single test plan for scenario-based coverage. The extensive focus on functional details and lack of integration checks between functions mean that the test plan won’t cover scenarios very efficiently. Instead, design a separate scenario-based test.
You can use both function-based tests and scenario-based tests within a single test plan.
A good testing strategy involves using both function-based and scenario-based tests to complement the coverage. Especially for regression coverage, using only function-based tests or only scenario-based tests leaves coverage gaps in either function integration or specific function details.
What's Next
Other guidelines on writing tests