What is issue reproduction and why we are doing it?
Issue reproduction activity's target is to provide additional context to the bug report - whether the bug is also reproducible on an assigned device/configuration for the tester. This helps to narrow down the seriousness of the bug (whether it's reproducible on larger variety of configurations) as well as gives an extra layer of confidence that another tester is able to follow listed issue reproduction steps and understand them.
Issue reproducing is a mandatory step in most of our projects and is performed prior to issue approval.
Some questions to keep in mind during issue reproduction:
- Do I understand the issue?
- Does the issue have a valid point?
- Does the issue reproduce with given steps?
- Does the video match with given steps?
- Does the issue need improvements in the structure and/or steps?
- Could this issue be described in a more general or more specific way?
When are you expected to reproduce issues?
1. Specific issue gets assigned to you during or after the cycle ends.
- In most cases, test leads ask from the project chat if someone is available for issue reproducing tasks and then assign these issues to available testers. But this approach can differ based on a project.
- In that case you can't choose which issues you reproduce - the list is already set by the test lead. Proceed with reproducing and add the result as a comment for those issues (‘Is reproducible’ or ‘Is not reproducible’).
2. Reproducing is a part of the task list
- In some projects there are task lists that require you to pick and reproduce a number of other tester's bugs. In this case you will have to choose the appropriate reports from the ongoing cycle that don't yet have reproduction attempts, reproduce them and add the reproduced issue IDs as comments to the task list step. Please note that the task list might contain additional info on how to perform this task, so make sure to follow the guidelines given there.
- If you are one of the first people who has finished testing in the test cycle, you might not be able to find enough issues to reproduce (there are no other issues except yours; or other issues have issue reproduction comments already). In these cases it would be great if you could check back in 2-3 hours and see if there are additional bugs reported. If you can not come back later, please notify the test lead and state in the task list that there were no issues to reproduce.
3. You stumble upon an already reported bug while testing
- If you're testing and during your duplicate search you discover that the bug you found is already reported, provide the reproducing information in the comments. Then it’s possible to raise to the client that this issue is still reproducible.
- Do not reproduce or comment issues that were closed as "works as designed" or "low priority", unless stated otherwise in the project details.
Comment types to be used during the issue reproduction
- ‘Is reproducible’/’Is not reproducible’
a) When issue is reproducible an ‘Is reproducible’ comment should be added.
- ‘Is reproducible’ comment template is available for all projects. The comment template can differ based on project needs as it can contain some specific info.
- Example of the most common ‘Is reproducible’ comment template:
b) When issue is not reproducible an ‘Is not reproducible’ comment should be added.
- ‘Is not reproducible’ comment template is available for all projects. The comment template can differ based on project needs as it can contain some specific info.
- Example of ‘Is not reproducible’ comment template:
c) Also include a video, screenshot or log, as per project requirements.
d) Make sure that all relevant info is included in this comment.
2. Regular additional comments
- Should be added to issues when it needs additional information that can be useful for the test lead and/or client (some examples are available in the next sections).
What are the steps of reproducing?
1. Make sure to understand the issue correctly.
a) Read all provided information in the issue, also review added attachments.
- Please avoid reproducing only based on the title or only reading part of the steps. It’s important to see all of the information that has been provided by the reporter.
b) Make sure that everything is understandable to you
c) If you can’t understand the issue, for example you don’t know how the feature should work or the issue report is missing vital information or you don’t have needed device, please contact the test lead or issue reporter :
- Via comment
- Via RocketChat
2. Make sure that this issue is valid
a) It’s not a duplicate:
- Maybe you have seen similar or the same issue before in the same project.
- If you have seen something similar - please try to find the original issue report.
- Please make sure not to spend too much time on that as the duplicate check should be performed by the reporter before submitting the issue.
- If you have found the duplicate, add a corresponding regular comment under the issue then test lead can see this information.
b) It’s a valid case for this project:
- Think about the app and the experience that you have with it. Is this issue actually valid and worth fixing? If it isn’t in your reasoned opinion, add a corresponding regular comment under the issue then test lead can see this information and count with it.
- Always make sure to provide a reasoned elaboration on why you think the issue is not valid, in order to avoid wasted efforts on the Test Lead side.
c) It has all the information that is needed for this project:
- The steps and the description are understandable and easy to follow.
- The video is relevant and matches the steps.
- If something needs to be changed or improved, add a corresponding regular comment under the issue, then test lead and reporter can see this information.
3. Read through the comments of the issue.
- There might be some valuable info or some additional information that you need to account during the issue reproduction.
4. Start reproducing
a) Follow the main rules of reproducing
- Use the provided build. In most cases it’s added to your task assignment details. But it can differ across projects.
- Follow the prerequisites.
- Follow provided steps. Make sure it’s possible to reproduce the issue with provided steps.
- Try to make sure if this issue can be more generic.
b) If the issue is reproducing:
- Add ‘Is reproducible’ comment under the issue with a video, screenshot or log, as per project requirements. Make sure to provide all relevant information in that comment.
- If there is something that test lead or reporter needs to know about this issue, please tell him/her and leave a note about that as a separate comment under the issue. For example if there is something vital missing.
c) If the issue is not reproducing:
- Please try again and make sure that you followed all of the prerequisites and steps.
- Think if this issue can be device, OS or browser specific. If your time allows, please try to reproduce also on another device.
- Maybe this issue can be account specific. That depends on a project if there are different types of accounts or not.
- If you have made sure that it’s not reproducing and you have followed all of the steps, please leave ‘Is not reproducible’ comment with video. The templates are available for all projects. If there is something that test lead needs to know about this issue, please tell it in RocketChat.
How to determine if this issue is more generic?
1. Is this issue reproducible with different accounts?
- As there can be accounts with different roles, it can be an indicator that it can be more generic.
2. Is this issue reproducible with different devices?
- If not stated otherwise in the issue reproduction task assignment, please try to use another device in comparison to the device used by the issue's reporter. The aim is to give the extra input on whether the issue is reproducible on another configuration.
- However, please think along with the issue - if it's clearly seen from the bug that it is tablet-specific, then there is no point to do the reproduction on the phone.