Mobile Network Emulation and more
September 5, 2017
This month Testable introduces network emulation for different network conditions including various mobile networks (e.g. LTE, 4G, 3G, etc). Other new features include email alert improvements, network call filters, and more. Read on for all the details.
Simulate users on various types of networks including mobile devices, cable, DSL, as well as any custom configuration.
When creating a test configuration, go to Advanced Options => Network Connection to select the desired network.
Many common networks are preconfigured in the list. Feel free to add your own custom network as well.
Network Call Filters
Sometimes during load tests we want to avoid generating load on 3rd party services like Google Analytics, a CDN, APM, etc. We now support a blacklist/whitelist configuration for filtering out certain network calls.
Add blocked hostnames, IP addresses, and IP ranges to the blacklist and any exceptions to the whitelist. This can be specified organization wide under Account => Settings or within a single test configuration under Advanced Options.
Blocked network calls still appear in the test results as failing to connect unless they are filtered out using result collection filters.
Result Collection Filters
Result collection filters exclude certain network calls from being included in the test results. There are two common use cases for this:
1. Filtering out blocked network calls from appearing in the results
2. For 3rd party services where we might want to make the network call but not collect results and report metrics. For example, you may want to allow Google Analytics reporting but not necessarily clutter your test results with performance monitoring for those network calls.
Similar to network filters this works with a blacklist/whitelist approach except in this case we use glob wildcard matching on the full URL.
Improved Email Alerts
New options for when to send email alerts have been added. For example, instead of sending an email on every successful test execution, only send when it resolves a previous test failure. On failure you can choose to send an email alert only on the first failure and not on subsequent consecutive failures.
Taken together with test scheduling this allows you to get notified when a test fails and then get another notification when the failure is resolved without cluttering your inbox.
This can be configured at the organization level under Account => Settings or per test configuration under Advanced Options.
HAR Replay Scenario Type
HAR file replay is now available as a first class scenario type on the Testable platform. HAR files can still be replayed in Node.js scripts and recordings as before. When creating a new scenario choose the Replay HAR File option and upload your HAR file to the scenario.
WebSocket connections now work in browsers launched by WebDriver.io via Selenium. All standard metrics are captured and are visible in the test results.
Test Spec Editor
The primary test specification is now viewable and editable directly on the scenario page making it easier to understand exactly what the scenario does. If you have multiple specs there is a dropdown to switch between them. Also included are handy templates that allow you to insert common operations into your script quickly.
In addition to the new features there were numerous infrastructure and library updates, bug fixes, performance improvements, and capacity increases during the last month.
Try out the new features and let us know if you have any feedback or concerns. Happy testing!