Monitor Issues and Severity Levels

These indicators in the Tests Report provide a detailed look into how effectively we’re identifying and managing issues throughout the testing process. The issues in these metrics refer only to issues reported as part of test execution, not any reported outside of tests.

Example issue indicators in the Testlio Platform

A chart showing issue count by priority and open issues, both over time

Indicator	What It Tracks	What Signals It Might Bring
New Issues	The total number of new issues identified and approved during the period. Automatically compares to how many issues were found during the same time frame immediately before the period.	This metric helps track trends in issue discovery over time. Increasing numbers of found issues may mean new features are introducing complexities, requiring more focused testing. Decreasing numbers of found issues may signal stability, allowing the team to focus on resolving existing issues or automating tests.
Breakdown by Severity	The percentages of approved issues across severity levels (high, medium, low).	The breakdown by status helps confirm that testing efforts are aligned with business needs, ensuring the most important issues are addressed first. Many high-severity issues might indicate a need to reassess priorities and refocus efforts on areas that pose the greatest risk. Many low-severity issues might signal readiness to move forward with less critical tasks or optimizations.
Issues per Run	The average number of issues identified and approved per test run. Automatically compares to the average during the same time frame immediately before the period.	This metric helps gauge how well the testing process is catching problems. A higher average can indicate deeper testing efforts. An increase in the average might signal that new features are introducing regressions, which may require additional focus or adjustments. Consistently low averages might reflect either system stability or a potential gap in testing coverage, meaning a deeper look at the current strategy is needed to ensure that no important areas are being missed.
Issues per Hour	The rate at which issues are being identified and approved. Automatically compares to the rate during the same time frame immediately before the period.	This metrics helps in evaluating the effectiveness of the testing pace. A high rate could indicate a testing phase where a lot of problems are being uncovered, signaling a need for quick action. A low rate suggests stability or that testing has matured. It could also mean that certain areas are being overlooked.
Open Issues	The total number of issues that were created during the period or the prior period and that are still open today. Automatically compares to the equivalent number during the same time frame immediately before the period.	Seeing how many issues are still unresolved can help indicate where ongoing attention is needed. Many open issues signals a need to review whether the focus is on the issues that make the most impact. Few open issues signals a need to assess whether the testing process is effectively catching all potential problems or if certain areas are being overlooked.
Open High-Severity Issues	The total number of high-severity issues that were created during the period or the prior period and that are still open today. Automatically compares to the equivalent number during the same time frame immediately before the period.	Internally, the team must prioritize high-severity issues, ensuring that critical system risks are being addressed quickly. Many open high-severity issues indicates that there are areas that need urgent focus. Alternatively, it may signal a need to reassess whether these issues represent critical risks. Few open high-severity issues suggests that the system may be stable or that the current testing strategy is effectively managing critical risks.
New Issues Chart	A visual representation of newly identified and approved issues by severity (high, medium, and low) and open issues over time.	This chart helps in tracking tends in how the testing process is surfacing critical problems, allowing adjustments to the testing strategy to ensure resources are allocated where they’re needed most. Increases in high-severity issues may signal regressions or newly introduced risks that need urgent attention. Increases in low-severity issues may allow a focus on areas with greater impact. Increases in open issues may signal potential bottlenecks, helping prioritize which issues need resolution first.

Monitor Execution Trends

Analyze Results and Quality

Track Issues

Monitor Issue Trends

Monitor Manual Testers