Skip to main content
Testing Windows
Doris Sooläte avatar
Written by Doris Sooläte
Updated over a week ago

What Is a Testing Window?

From a tester’s point of view, a Testing Window is a time-slot where testing activities in a run occur. It has a clear start: the moment when the tasks become available for execution and we expect testers to start testing, and a clear end: the latest moment by which all testing tasks in a run have to be completed.

Why Use Testing Windows

Previously at Testlio, one customer request was often tested within a single long testing period - sometimes ranging from 12- 48 hours. We are re-shaping this approach so one customer request can be served by one or multiple testing windows which start sooner, have a shorter duration, and have predictable start and end times.

How Testing Windows Work

To handle a client testing request, a coordinator will schedule one or multiple testing slots with a duration of up to 4 hours. The schedule will be decided based on the coverage and turnaround time requirements accounting with availability of testers during particular times.

Expectations on Testers

We expect you to show up and start testing within the first hour of the testing window and generally complete the testing by the beginning of the last hour of the window to make sure you have enough buffer time to complete activities, file issues, and respond to input. Our coordinators will do their best to ensure that every tester has the time to complete their task to account for unforeseen circumstances or extensive issue reporting time. In general, the testing window duration will be two times the duration of the longest testing task. While for now this approach is a recommendation, it may become mandatory in the future.

Testing windows should make time management more straightforward. Having clear time-slots within a day where you will be invited to test should allow you to focus on one task at a time (noting our Freelancer Services Agreement prohibits parallel work/double billing).

We encourage you to share your observations about the builds and products you are testing with others in the relevant Workspace chat to share experiences and ensure great results for the testing team and our customers.


Do all workspaces use Testing Windows?

While most workspaces take advantage of testing windows, not all do. Refer to the instructions in the given workspace.

Do all tasks occur in a Testing Window?

Not all tasks are limited to testing windows. Some engagements require longer testing times or may not be suitable for the testing window approach. Many tasks associated with a run (build intake, instruction preparation, results analysis, client reporting) intentionally happen outside of the window. We strive to conduct the majority of our planned and scheduled testing work within the testing windows in most workspaces.

What is the recommended and maximum task duration within a window?

A recommended task duration within a window is up to 1h and 30 minutes. Maximum task length within a window is 2 hours.

Is it possible for me to participate only in the Testing Windows runs that are scheduled in my region?

You are free to choose whichever testing windows that work best for you, regardless of your location. However, we ask that you only accept invitations to testing windows when you can be productive.

Is it possible to accept invites to multiple runs that are happening within the same 4h testing window?

Today, our Platform does not prevent this. However, we request that you only accept invitations for testing work in a window that:

  • You can do with high-focus (e.g. not context shift and multi-task)

  • You can initiate within the first hour of the window

  • You can complete by beginning of the final hour

You also can accept invitations for other tasks in the window, like issue reproduction, once you complete testing.

Does the 4-hour testing window include issue reproduction tasks?

Issue reproduction tasks are additional tasks. We aim to start reproducing issues as soon as they are reported during the testing window. In general, these are invited and compensated separately, meaning you may have more work in the window than the initial scope of testing.

Will invitations that expire during my night time affect my rating?

Invites sent and expired between 11 PM to 7 AM in your local time zone will not affect your ratings negatively. Although you may still receive invites during this time, the expired ones within this 8-hour range will not be considered in the rating's calculation.


Did this answer your question?