This guidelines are part of the Test Writing Guidelines.
The exact use of test metadata can depend on the workspace/engagement. This example presents one possible option.
Test Title
The test title should indicate the feature (or a logical grouping of the functionality under test) with any additional specifications about the testable function or flow. As the ability to group and present metadata about the test in the platform is currently limited, the title is used to give quick information to the stakeholder reviewing the test list.
The naming convention for titles is as follows:
[Tag][Main feature under test][Optional: Sub-feature under test] Description of the test
Convention | Description |
Tag | An example of tags to give quick information about the type of test:
|
Main feature under test | Represents the name of the main functionality. Must be aligned with a mindmap node. |
Sub-feature under test | Optional: leave out if the test only relates to the main feature. Must be aligned with a mindmap node. |
Description of the test | Short assertive description of the coverage, also mentioning any constraints. |
Example Title
[F][Tests][Export] Export multiple selected tests
Platforms
When creating a test, it is essential to specify the platforms (including operating systems, devices, browsers) that the test applies to. Only include those platforms that are relevant to the specific test. If the test is applicable to all platforms, it should be marked as such.
Test Priority
Test priority is determined based on the potential risk associated with failure of the functionality being tested. A higher priority indicates a higher business risk if the tested feature doesn't perform as expected or encounters issues.
Multiple Tests for the Same Functionality: You can have multiple tests covering the same functionality, each assigned different priorities. Higher priority tests focus on critical functionalities that pose significant business risks if they fail. Lower priority tests extend the coverage of higher priority tests by exploring additional scenarios or edge cases.
Input for Test Coverage Decisions: Test priorities serve as valuable inputs when deciding on the coverage for different testing needs. For example, a smoke test might only contain high priority tests and extended regression testing might include tests with all three priorities.
Priority | What failure means | Execution criteria | Release criteria* |
High | Unacceptable loss of functionality from a business perspective. Users might be blocked from completing essential activities. | High priority tests should be executed before any deployment to production and should be part of all regression runs. | Decide on a workspace level |
Medium | A loss in functionality that would potentially disrupt but not fully block relevant stakeholders. | Medium priority tests are candidates for regular scheduled regression execution. | Decide on a workspace level |
Low | A minor loss of functionality that can cause some inconvenience to stakeholders but would not significantly affect their main actions. | Low priority tests are candidates for one-time execution (such as on new feature releases) and/or for execution in scenarios where extended test coverage is required. | Decide on a workspace level |
* Release criteria is not mandatory and something for the client to decide if they drive release criteria based on test results (meaning the product isn’t released if any high-priority tests fail).
Execution time
Net execution time refers to the actual time we expect the tester to spend executing a test in minutes, excluding any time spent reporting potential issues.
The execution time that is added to the test itself does not appear for testers. It is used together with other tests in a plan to generate the expected time for run tasks, but testers don’t see it for a given test when testing.
Guidelines for Scripted Tests
For scripted tests, net execution time should be measured by recording the time taken to run the test steps and validate the expected results. Another approach is taking a simpler test as a baseline (for example, if a test on signing in is estimated to take 5 minutes) and estimate how much more complex and lengthy the other test is against this baseline. For example, in a social media app, a sign-in test might be estimated to take 5 minutes to cover a few negative scenarios and one positive scenario. Assuming the messaging feature of the app is 3x as complex and lengthy, you could estimate a messaging feature test as taking 15 minutes.
Guidelines for Structured Exploratory Tests
Given that structured exploratory tests allow additional exploration in comparison to scripted tests, you need to be clear on the boundaries of how long one test should run. Tests should be aimed to be timeboxed between 5-15 minutes.
Remember to explicitly include the time limit in the prerequisites so testers can see.
Guidelines for Exploratory Tests
Since exploratory tests are creative tests, testers could spend several hours exploring. So you need to set a timebox for all exploratory tests and explicitly write the limit in the test step name so testers can see.
Feature
The main feature of a test refers to the primary functionality or aspect being tested. Also identify sub-features, providing a hierarchical structure for tests related to the parent feature. This hierarchical approach allows for quick and precise selection of relevant coverage, whether targeting a specific sub-feature or all tests related to the main feature.
All feature names have to correlate with mindmap nodes (the mindmap should use the same names as in the product).
Labels
Labels are optional but valuable for categorizing and filtering tests. They provide additional information for grouping and organizing tests based on specific criteria. By using labels, coordinators can quickly identify and retrieve tests that share common characteristics or attributes.
Agreement on Basic Labels
Labels provide additional categorization and thus filtering opportunities for tests. Labels are optional but highly beneficial for more precise filtering and grouping of tests.
To ensure consistency and effectiveness, it is important to agree on basic labels when starting the process of writing tests in a particular workspace. Establishing a standard set of labels will facilitate precise filtering and grouping of tests, making it easier for the Coordinators team to manage and organize test repositories efficiently.
An example:
Label category | Description | Values |
Test type | Determining the test type: scenario- or function-based or embedded. | Scenario Function Embedded |
Automated On
When tests have been automated for a given platform, add the platform to the Automated on field.