Within LeoCore, a test plan refers to a collection of tests that are run. Think of it as a container holding all the tests.
Creating a test plan on LeoCore involves the following steps:
Defining which tests are included
Specifying the devices to be used for testing
Using the algorithm or assigning testers to carry out the testing
There are three types of test plans:
Static plans
Tests are included manually.
Automatically updated only when an included test is archived.
Filter-based plans
Tests are included based on defined criteria.
Automatically updated whenever tests matching the criteria are added or removed.
Temporary plans
Plans created when retesting automated runs.
Exist only in the run they were created for.
Automatically archived as soon as the run they are in is marked as finished.
There is also a deprecated distinction in LeoCore between functional and localization plans. Always create functional plans.
Following a few principles helps ensure that your test plans are logically structured, focused, clear to testers, and reviewed for any issues or inconsistencies before execution, leading to effective and efficient testing processes.
Focus on a Logical Set of Functions/Flows
Test plans should contain a logical grouping of functions or flows that need to be verified from both the end-user and coordinator perspective. Ideally, create one plan for a logical set of features / functionality to maintain clarity and focus.
Include All Necessary Information
Testers only have access to the specific plan they are assigned to, without seeing the larger picture. Any information needed to understand the tests should be embedded within the tests themselves so testers have all they require to execute the assigned plan.
Separate Test Types
Keep exploratory tests and scripted tests in different test plans. This allows testers to have a clear focus and mindset for each type of testing. Mixing scripted and exploratory tests within a plan often leads to suboptimal results. Additionally, targeting different testers for each type of testing can lead to better outcomes.
Use Preview To Review
After editing a test plan, use the preview feature to review the entire plan. This helps you catch potential inconsistencies and problems before the plan is executed, ensuring quality and accuracy.
Test Order in Static Plans
Ordering tests in static plans in a logical and efficient manner leads to a smooth testing flow, optimized coverage, and timely identification of critical issues. This section presents some tips for good ordering.
Include an Instructions Test
If you need to provide additional context or information to testers about the engagement, workspace, or product, consider starting the plan with an "Instructions Test". This test includes details such as specific accounts to be used and relevant guides/voiceovers to provide the necessary context. This is optional and can be helpful when documentation is otherwise missing or run instructions are inconvenient.
Order Logically
Order the tests in a logical manner within the plan. If a plan includes simple and complex features, start with the more simple ones. Allow testers to progress through the app naturally, considering their feature knowledge and complexity.
Consider the Run Focus and Critical Functionality
Be mindful of the particular run’s focus. Critical functionality such as new features, impacted areas, or critical business flows should be covered earlier in the run. This means potential showstoppers can be identified earlier in the testing process.
Optimize for Data Generation and Prerequisites
While tests should be independent, you can often optimize for data generation and prerequisites through test sequencing. For example, if a test requires using a newly created account and there's a separate test for account creation, put the account creation test first.
Consider Internal Logic
When testers spend more time on certain features at the beginning of a test plan (such as troubleshooting or reporting bugs), they tend to cover tests towards the end of the plan with potentially narrower focus. So for tests toward the end of the plan, either be more detailed or include tests involving less critical functionality, such as account settings.