Skip to main content
Signals module
Doris Sooläte avatar
Written by Doris Sooläte
Updated over 6 months ago

Overview

Signal-driven testing is focused on detecting quality signals from various data sources and actioning them quickly and concisely to resolve quality that matters to application end users. Signals module incorporates event sources, detection rules, mapped actions, notifications and keeping track of signal lifecycle.

Setting up signals

Prerequisite

  • App Reviews must be set up first as source for signal generation as Signals currently only support App Reviews. We will introduce more sources in 2024.

  • User has a role Admin or Coordinator to set up signals

Add sources

Sources are hard requirement and connected to rules. In order to create rules, we first need to establish the source of information we want to keep scanning.

  1. Go to Settings and select Sources

  2. Click on "Add source" button

  3. Select a source (we only support App Store and Google Play Store at this phase)

  4. Select the available app from the source (prerequisite needs to be completed). Example if we have Netflix Inc app setup for App Reviews.

  5. Add as source

  6. Source now is listed as available source to connect to rules

Create rules

Rules are important part of Signals. Rules are needed for scanning sources and if we have a match, we generate signals. Consider it like monitoring rule that triggers and creates an alert if an event matches our search and scan criteria.

Important

  • Rules take effect on only future events, not profiling events backwards

  • App Reviews are imported daily, once a day at 5am CT

  • Rules are scanning once new reviews are imported

  • Profiling happens only to new app reviews from the import

  • If rules are modified, paused and put to live, then it applies to next import content

Setting up rules

  1. Create a new rule in Signals > Rules

  2. Define rule matching criteria

    1. Rule name – provide a meaningful and identifiable name

    2. Data source – select defined sources in previous step. Rule is linked to source and there can only be one source per rule. Important: you can create many separate rules per source, but you cannot share rule across sources.

    3. Filters

      1. Ratings – select one or many app review ratings that apply to the rule

      2. Search term – comma separate keywords that need to be matched from the app review given by users. Each keyword is standalone condition

    4. Signal creation behaviour

      1. One signal per rule – option to sum all matching events into single signal. Example, when signal is scanned for new reviews and 3 matches, those are reported as 1 signal and all 3 reviews are shown as signal source events

      2. One signal per event – each matching event is considered standalone signal and contains 1-1 mapping to the review.

    5. Example rule setting

  3. Created signal now is available in Rules listing.

    Note – all newly created rules are by default in draft mode.

  4. Edit, set live, delete and pause a rule

    1. Select rule created in Signals > Rules

    2. Rule detail view is opened

    3. Edit – rule editing flow is opened and all defined settings can be modified. All edits affect only future events.

    4. Set live – to active rule based event monitoring and scanning, rule must be set to live. Once it is live, we can see "LIVE" tag and Pause button

    5. Pause – instead of deleting rules, we can pause them if the rule is needed or other needs tweaking, reviews by others. Once paused, user can set to live any time.

    6. Delete – if rule no longer effective or needed, it is wiser to remove it.

    7. Important 1– putting rule to pause or deleting one, does not effect already created signals. These statuses only affect future source scanning

Configure Slack notifications

We can check signals in the module and what have been created or closed, then most efficient way is to configure Slack notifications that will inform newly created signals.

  1. Go to Signals > Settings > Notifications

  2. This redirects us to workspace Slack notification settings

  3. Select Slack workspace and select slack channel we want signals notification to be sent to

  4. Select Signals:new and this will inform all in the channel of new signals

  5. Add connection and we can see now notification is set up

Using signals module

Module overivew

Signals is based on monitoring events and depend if we have any matches to new reviews or not. By default it may be empty as our rules are not effective or the app is performing really well and users just give praise.

In case we do have signals generated, we do see them based on Open or Closed via our filters

In the list view we can search or fitlers and go to see detailed view of a signal.

Signals actions

Signals offers currently 2 options - close or create an issue to action it to understand if the feedback is causing impact to the application and is this quality gap in testing strategy or there is an issue we should try to reproduce and suggest to address it.

Create an issue

Create an issue from the detail view opens Issue module and connection to the issue.

Creating the issue offers the title and connects to signal itself, however, it does not auto-fill the issue itself. Does expect the user to fill in relevant information.

The issue is now linked to the Signal and we can see the corresponding reference as part of our signal in detail view

Connect as many issues to signal as relevant.

Close signal

Once the signal is actioned, we can close it with relevant reasoning and offering data points for future analysis.

Did this answer your question?