Use the Automation Report to follow how successful your automated tests are and where you might need to direct extra effort.
Indicator | What It Tracks | What Signals It Might Bring |
Active Tests | The total number of tests that have been automated. Automatically compares to the total during the same time frame immediately before the period. | This indicator offers insights into progress toward automation goals and the scaling of automation efforts. |
Percentage Automated | The percent of all tests that have been automated during the period. Automatically compares to the percentage during the same time frame immediately before the period.
Note that current data may differ from expectations as deleted and archived tests are included. This difference should resolve over time with more granular metadata. | This metric provides a snapshot of automation coverage and progress toward targets. |
Test Automations | The total number of times that tests were automated during the period. Automatically compares to the total during the same time frame immediately before the period.
Note that current data may differ from expectations as deleted and archived tests are included. This difference should resolve over time with more granular metadata. | This indicator offers insights into progress toward automation goals and the scaling of automation efforts. |
Automation Pass Rate | The percentage of automated test runs that passed during the period. Automatically compares to the percentage during the same time frame immediately before the period. | This indicator provides a snapshot of automation stability and reliability. |
Automation Fail Rate | The percentage of automated test runs that failed during the period. Automatically compares to the percentage during the same time frame immediately before the period. | This metric offers recurring failures and areas for improvement. |
Automation Error Rate | The percentage of automated test runs that had an error during the period. Automatically compares to the percentage during the same time frame immediately before the period. | This indicator provides insights into recurring errors and areas for improvement. |
Automation Pass Rate | The percentage of automated test runs that passed over time. | This chart helps stakeholders identify fluctuations, reliability issues, and potential instability in the automation framework. |
Automation Fail Rate | The percentage of automated test runs that failed over time. | This chart complements the pass rate chart by focusing on problem areas, enabling teams to prioritize fixes and optimize test reliability. |
Issues by Automation | The total number of issues identified by automated test runs during the period. Automatically compares to the number during the same time frame immediately before the period. | This indicator showcases the role of automation role in detecting defects. |
Addressed Issues | The total number of issues detected by automation that were addressed during the period. Automatically compares to the total during the same time frame immediately before the period. | This metric emphasizes action taken on defects. |
Addressed Issues Rate | The percentage of issues detected by automation that were addressed during the period. Automatically compares to the percentage during the same time frame immediately before the period. | This indicator provides insight into resolution efficiency. |