Things to keep in mind before creating a new bug report
*If you are new to Testlio, it might be useful to give a go at reproducing some issues before creating your own.
*Before creating your bug report, make sure that your bug isn't a duplicate. See how to look for duplicates.
Now that you are ready to report your bug, think of:
The Problem – Keep your eye on the prize, or in this case, the reason you’re writing a bug report in the first place. Make sure your title and description are focused on that problem and not the less important details.
Length – Detail is appreciated, but leaving a wall of text about an issue makes it harder to follow and understand. Keep your report short, simple, and focused!
Proof – Some bugs only exist on one platform or OS, some can’t be reproduced at all. Make sure to include screenshots, videos, and logs so we can confirm what happened.
Related Issues – Are there other actions that could cause the same problem? Are they relevant to mention in your report?
Creating a bug report with Testlio
The following is our preferred format for a bug report. Here are some guidelines and tips to help you succeed on our platform!
1. Title: [Profile] Uploaded photos are rotated sideways
- Between the [ ] brackets, first, specify what section of the app the bug was found. Be specific!
- Follow the brackets with a short description of the problem
- Please, do not put dot at the end of the title
2. The severity of a bug:
a) High/Critical: anything that blocks the use of the app or has a significant negative effect on normal user flow. See examples below
- [Login] User is not able to sign in after restoring the network.
- [Chat] App crashes and the user is logged out after unfriending user and sending a message
b) Medium: anything that will have a heavily negative effect on the user experience, in general, would be medium. See examples below
- [Notification Stream] New private message notification stays in-stream even when the user has read the message
- [Sign in] App freezes after tapping Next button to Sign In and twice changing the device orientation
c) Minor: anything else (typos, missing icons, layout issues, etc.) See examples below
- [Calendar] The background of the 'Delete' confirmation window becomes blank (white) after restoring the app from the background.
- [Game] When entering a name to a new game the text cursor is not shown clearly on the screen
3. Environment details.
In this section, we’re looking for details about your testing environment.
List your device, operating system, app version, browser version, and an internet connection, depending on relevance.
When you click to create a new issue you will see a similar template:
4. Steps to Reproduce
List your exact steps to reproduce the issue, and don’t skip any steps!
- Use the ‘>’ symbol to signal navigating from one step to the next
- Keep it simple and easy to follow!
- The first step should be the prerequisites someone needs before they can follow your repro steps
What NOT to do
Steps to reproduce:
- Open app
- Log in with valid credentials
- Tap on Settings
- On the displayed options select Profile
- Tap on Edit in the top right corner
- Tap on Profile Image
- Select an image saved on the phone
- Select Upload
- Tap on Save Changes
Simplified! (Here's how to do it)
Steps to reproduce:
0. Prerequisite: Log in with valid credentials
1. Settings > Profile > Edit
2. Change Profile picture > Save
5. Expected Result
In shorthand, write down what should be happening in this situation
6. Actual Result
Here is where you record the problematic behavior. What’s broken?
We require attachments with every bug report! Not everything is reproducible on every device or environment, so it’s extremely important to include proof and detailed documentation of how the bug is occurring on your end.
Use only letters, numbers, and underscores in attachment file names.
General naming convention should be:
- <issue number>_<attachment type>.<file extension>
- For example: 38744_screenshot.png or 38744_log.txt
Visual bugs (UI)
- Visual bugs that are stationary are often easy to display with a screenshot which highlights the problematic behavior
For issues that are to do with how the application functions, we require two attachments with every bug report.
- Many visual bugs happen when switching between screens, or need a motion to be understood. When this is the case, screenshots are not sufficient.
- Video has to be in .mp4 format and under 10mb and with a minimum resolution of 720p
- Make sure that no background sound is recorded on the video.
- Make the video as short as possible, but make sure to include important reproducing steps (e.g, if the issue is not related to logging in, do not include logging in in the video).
- If you are on iOS 11 and above, try using iPhone’s own screen recorder, you can watch a video on how to do that here.
- We prefer screen recordings over videos recorded with a camera, but if you are having issues with the software you can use this option.
- On Android, we suggest AZ Screen Recorder, you can read more about how to use it here.
- Logs are mandatory for reporting crashes
- Logs allow developers to dig deep into the root of the issue to see exactly what went wrong on your device. Many crashes and errors are not easy to reproduce. Without a crash or console log, your report will not provide much value.
- Retrieving iOS logs: iOS Console. Retrieving Windows logs: iTools. Retrieving Android logs: Tutorial
- See articles about retrieving logs.
- NB! In some specified cases, you need to use Charles log files. In such a case this will be communicated in Project details. This guide can be found here.
Think you've got the hang of it? Double-check by taking a look at an example of a good bug report.