Test Runners

Test Runners actually run your tests in various types of ways. They are what actually spin up concurrent clients and execute your defined scenario as configured. We currently support three types of test runners:

  1. A public shared grid distributed across the globe on the infrustructure of various cloud providers.
  2. A private shared grid distributed across your own infrastructure. This private grid can span both your own data centers and any cloud provider. This grid is then shared across tests run in your account only.
  3. On demand grid for a single test execution ensures full isolation. Currently this supports AWS EC2 instances only, either using Testable's account or your own. Other cloud platforms will be added in the future.

Test Configuration

When creating a new test configuration you must select one or more test runners to execute your test. The same test runner can be configured multiple times within a single configuration as long as you stay within your account limits.

So for example you could run your scenario on our shared public grid, inside your data center, and using on demand AWS instances all at the same time.

AWS On Demand Test Runner

Add

You can run your test on isolated AWS instances, either using your own AWS account or ours. Note: For the purposes of your monthly plan, on demand tests are rounded up to the nearest hour, inline with how AWS billing works. So a 5 minute test and a 59 minute test will both count as 1 hour of "duration" as far as your plan limits are concerned. Use either spot or on demand instances.

On the Test Configuration page there are a few options.

Option 1: EC2 Instances Within the Testable Account

Select AWS On Demand - Testable Account to have the EC2 instances started under the Testable account.

AWS On Demand Public

Option 2: EC2 Instances Within Your Testable Account

Configure your own AWS account to have the EC2 instances started there. Testable will automatically terminate them once the test is finished in that region. Click the New... option or check out the Management section of this guide below.

AWS On Demand Private

Notice the extra Key Pair option. Choose a key pair you have access to so that you can connect and monitor the instances we spun up as part of your test.

Public Shared Grid

The quickest way to run your test is on our public shared grid. Assuming there is no queue, your test will execute immediately in the regions selected.

Public Grid

As far as plan limits are concerned, shared grid usage is rounded to the nearest second (unlike AWS On Demand where rounding is up to the next hour).

Private Shared Grid

If you run on premises the My Shared Grid test runner will be automatically created. Whatever region names you provide when running your agents will appear in the list.

Private Grid

Management

Go to Settings -> Test Runners to add/remove/update your account's test runners. In the list you will see both public test runners (e.g. Public Shared Grid and AWS On Demand - Testable Account) as well as any private runners for your account.

Test Runners

Add New Test Runner

Press the Create button in the upper right to add a new test runner. Only AWS On Demand runners can be configured here. If you run an agent on premises the My Shared Grid runner will be created automatically and whatever region names you choose will appear in the list on this page.

Create Runner

Update Test Runner Configuration

For private AWS On Demand test runners you can update the AWS credentials by clicking the configure icon next to the test runner name.

Delete Test Runner

For private AWS On Demand test runners you can delete it by clicking the trash icon next to the test runner name.