The rating system measures your availability, responsiveness, efficiency, and effectiveness along multiple dimensions.
Your rating is based on your work in all workspaces across Testlio. The data used to calculate your rating goes back up to 2 years and is updated daily.
Metrics Used in the Rating System
The rating includes the following metrics:
Issue Acceptance: % of your reported issues that have been accepted by a coordinator
Run Attendance: % of accepted runs completed
High Priority Issues: % of issues that have been approved as high severity issues
Invite Response: % of invites that result in a response (accept or decline)
Issues Per Hour: # of approved issues per hour
Issue Acceptance
Why It’s Important
The goal is to ensure that issues reported to clients are accurate, relevant, and aligned with expectations. If you follow the provided instructions and guidelines, your issue will be accepted, which helps reduce the time coordinators spend analyzing invalid issues. This greater focus on delivering value efficiently and maintaining strong relationships. Issues that are closed with certain resolutions are not considered accepted and negatively affect your rating.
How to Succeed
Always read the workspace, run, and task instructions before starting. These are tailored to meet client needs.
Understand out-of-scope areas to avoid reporting issues that do not add value.
Check for duplicates to save time and maintain report clarity.
Think critically about whether an issue should be fixed by considering the product’s purpose and target audience.
Finally, if your issues get closed without approval, use this feedback in future runs. Understanding why it happened helps you avoid making similar mistakes in the future.
What Counts for Issue Acceptance
An issue counts as not accepted if it is closed for any of the following resolutions without prior approval:
Duplicate
Works as Designed
Out of scope
Invalid
This means that if the issue is reported following workspace, run, and task instructions, issue closure does not affect the rating negatively. For example, if the issue is reported following the task instructions and during issue validation the coordinator identifies outdated instructions, then the issue may be closed with the resolution Spec change and this does not affect the creator's rating.
Run Attendance
Why It’s Important
Make sure that you complete all tasks that you have accepted within the agreed timeframe. When you accept a task and fail to complete it, coordinators must spend extra time finding replacements. This can delay result delivery, negatively affecting the success of the project.
How to Succeed
Accept tasks only when you can commit to completing them within the agreed timeframe.
Manage your workload and avoid overcommitting during busy periods.
Communicate proactively if you foresee a conflict or delay.
Time management is essential.
High Priority Issues
Why It’s Important
Not all runs are the same, so understanding the purpose of each run helps you look for issues. Clients may want to check new areas or areas that have changed, and it’s likely that you find new issues.
How to Succeed
Identify areas that have recently changed or are new, as these are likely to contain high-priority issues. This insight allows you to focus on what matters most when executing the task. Exploring around the test steps may uncover them.
Understand the client's business context and the purpose of the app. Think like the end user:
How would they use this feature?
What issues would disrupt their experience?
Invite Response
Why It’s Important
Tasks are time-sensitive and quick responses to invites help in assembling a team promptly, ensuring test execution can meet client deadlines. Whenever you receive an invite, make sure to respond. Your free time is respected and you're encouraged to take time for yourself in between tasks.
How to Succeed
Respond to invites promptly, whether you accept or decline.
If you’re unsure, assess your availability as soon as possible to avoid letting the invite expire.
Take time to rest. Invites sent and expired between 11 pm and 7 am (in your local time) do not affect your rating negatively.
Issues per Hour
Why It’s Important
It's important that as many issues as possible are spotted during runs. To help with this, you are expected to understand the app you are testing, which may include looking deeply into error-prone areas. The number of approved issues you report reflects the additional value you contribute to clients besides covering the task scope.
How to Succeed
Start your testing run early to maximize the time available for finding issues.
Dive deeper into error-prone areas and, if assigned time allows, explore beyond the given test steps.
Read more about various testing types and approaches to stay on top of testing techniques.
How the Rating Is Calculated
All of these metrics are calculated at the workspace level, which means that you are compared to other freelancers within the same workspace. Your score on a specific metric indicates your current standing among your peers in that workspace.
To understand the meaning of a score on a specific metric, take the example of the Invite Response. To calculate this score, first the number of invites sent to you is divided by the number of invites you respond to, which results in your invite response rate. Your rate is then compared to the rates of other freelancers in the same workspace, with the result used as your rating.
For example, a score of 60 means that you have a higher invite response rate than 60% of the other freelancers in the same workspace and 40% of freelancers have a higher rate than you.
The Two Parts of the Rating System
Workspace-Specific Rating: A representation of your performance in comparison to your peers in a specific workspace.
Overall Rating: A representation of your performance in comparison to your peers in all workspaces combined into one number.
The Overall Rating reflects the percentage of your time spent testing in a specific workspace. For example, if you have spent 10% of your testing time in a particular workspace, that workspace makes up 10% of your Overall Rating.
The following image presents an example of calculating a rating for issues per hour across two workspaces.
Important Points to Consider
Ratings can be found on your Testlio Platform dashboard or when accessing your profile overview page. Learn how to improve your rating.
For established workspaces, freelancers can expect stable ratings and fewer fluctuations in the score.
The rating score is a comparison of freelancers, so it is possible for an individual's score to change even if you have not participated in a run. This can occur when the performance of other freelancers has improved or declined during a run.
#TestlioBot