Email alerts, improved analytics, and more!

June 14, 2017

This month we build on the recent success criteria feature by adding email notifications on test completion or test failure. When combined with test scheduling you can now use Testable as an external monitoring solution. Read on for full details on this and all the other new features.

Email Notifications

Email notifications can be configured organization wide via Account => Settings or per test configuration. They can be received under two conditions:

  1. On All Test Completions: Sends an email every time a test finishes

  2. On Test Failure: Sends an email only if all success criteria are not met.

The email alerts include the success criteria outcomes and high level details on the specific test that was run.

This now opens up the possibility of using Testable for some interesting new use cases. For example, use your load test scenario as an external monitor of infrastructure health:

  • Define a test scenario

  • Define success criteria

  • Schedule a test to run on a regular basis with one concurrent user per region

  • In the test configuration, configure email notifications on test failure

Define your test scenario once and then use it for both load testing and monitoring!

Per Resource/URL Analytics

Testable has long shown and aggregated metrics per resource. A “resource” is usually the URL or a label. With this release you can now see how the metrics compare with a weighted average of the same resource during recent test runs.

Click on a resource and all the charts and metrics in the results view will drill down to that resource only.

These fine grain analytics can be really useful for identifying performance bottlenecks and issues.

Improved Result View

The “Metrics” and “Chart” widgets have now been combined into one more concise interactive view. Toggle the metrics shown in the chart by clicking them. This opens up the possibility of customizing the results view to show a bunch of metrics but only graph a few of them by default.

AWS Test Runners — VPC and Subnet Defaults

If you use your own AWS account to spin up test runner instances you can now configure a default VPC and Subnet for each AWS region. These choices can be locked across tests or simply used as a default. Go to Account => Test Runners to create or update your test runner settings.

Try out the new features and let us know if you have any feedback or concerns. Happy testing!