In order to evaluate if reported bugs follow Testlio standards, all bug reports are automatically graded.
1. What affects the bug report rating?
By default, all bug reports get a maximum grade if there are no changes made in the title nor in content, descriptions are clear and correct attachments are added. The rating decreases when test leads either requests changes or makes bigger changes themselves before approving the issue.
2. Do every small change in the bug report decrease the rating?
No, if only a small typo if fixed, the rating won't be affected. However, if there are many typos in the report, the rating will decrease.
3. What happens if my issue is closed due to merging?
Merging won't affect the rating.
4. Do the changes after approval affect issue rating?
No, they don't. Only the changes done before approving are calculated in.
5. What happens if I get a low issue rating?
Nothing happens straight away. Your pay rate won't be lowered and you won't be removed from the project if you learn from the mistakes (this is why issue history is so useful) and develop your skills constantly. If you make the same mistakes constantly and there's no sign of progress, there's a lower chance to get new invites.
6. But phrasing changes are so subjective!
Indeed, they can be, but you shouldn't worry about it. If you find valuable bugs and report them correctly, your performance is still valued highly.
7. My issue was closed. How does it affect my results?
If a single issue is closed, it doesn't have any direct effect. However, if none of your reported bugs in the cycle were approved, it will reflect in your overall results.
8. Some corrections are made due to client requirements, it's not my fault.
The issue rating gives just a generic insight on how well are testers following the requirements. It's not the only metric that is considered and therefore shouldn't be taken too seriously. Just do the best job you can and it will be noticed!
9. How do I know how am I doing?
You can keep an eye on your results from the "Your feedback" view on the platform. The issue statuses reflect how well they were reported. There are 4 different statuses:
- Approved - Well done! Your issue was approved without any changes.
- Approved with changes - The issue was approved but you should review the history to see what were the changes. Learn from them! ;)
- Closed - Unfortunately, the issue wasn't approved. Please check the close reason.
- Approved and Highlighted - Your reported issue was one of the top issues in that cycle, highlighted in the client report.
10. What should I do if I still have questions about some specific changes in my bug reports?
First, reach out to your test lead directly to discuss the cases through. If the solution is not clear, feel free to ask help from the community team.
Most importantly - don't worry solely about the ratings. Aim for creating value and you will do great!