The rating system measures your availability, responsiveness, efficiency, and effectiveness along multiple dimensions.
Your rating is based on your work in all workspaces across Testlio. The data used to calculate your rating goes back up to 2 years and is updated daily.
Metrics Used in the Rating System
The rating includes the following metrics:
Issue Acceptance: % of your reported issues that have been accepted by a coordinator
Run Attendance: % of accepted runs completed
High Priority Issues: % of issues that have been approved as high severity issues
Invite Response: % of invites that result in a response (accept or decline)
Issues Per Hour: # of approved issues per hour
What Counts for Issue Acceptance
An issue counts as not accepted if it is closed for any of the following reasons without prior approval:
Duplicate
Works as Designed
Out of scope
Invalid
This means other reasons for closing don't affect the rating. For example, if the task changes after the issue is created, the issue may closed with the reason "Spec change" and this does not affect the creator's rating.
How the Rating Is Calculated
All of these metrics are calculated at the workspace level, which means that you are compared to other freelancers within the same workspace. Your score on a specific metric indicates your current standing among your peers in that workspace.
To understand the meaning of a score on a specific metric, take the example of the Invite Response. To calculate this score, first the number of invites sent to you is divided by the number of invites you respond to, which results in your invite response rate. Your rate is then compared to the rates of other testers in the same workspace, with the result used as your rating.
For example, a score of 60 means that you have a higher invite response rate than 60% of the other testers in the same workspace and 40% of testers have a higher rate than you.
The Two Parts of the Rating System
Workspace-Specific Rating: A representation of your performance in comparison to your peers in a specific workspace.
Overall Rating: A representation of your performance in comparison to your peers in all workspaces combined into one number.
The Overall Rating reflects the percentage of your time spent testing in a specific workspace. For example, if you have spent 10% of your testing time in a particular workspace, that workspace makes up 10% of your Overall Rating.
The following image presents an example of calculating a rating for issues per hour across two workspaces.
Important Points to Consider
Ratings can be found on your Testlio Platform dashboard or when accessing your profile overview page. Learn how to improve your rating.
Invites sent and expired between 11 pm and 7 am (in your local time) have no negative affect on your rating.
In some workspaces, issues must be reported on an external platform and are not considered when calculating the rating. The rating system only takes into account your activity on the Testlio Platform.
For stable workspaces, the testers can expect stable ratings and fewer fluctuations in the score.
The rating score is a comparison of testers, so it is possible for an individual's score to change even if you have not participated in a run. This can occur when the performance of other testers has improved or declined during a run.
#TestlioBot