Introducing Testable

January 21, 2016

To get this blog started let’s discuss the motivations for Testable, what makes it unique, and some of our founding principles.

Testable was founded with the idea of making performance testing easier for all types of developers and technology teams.  We wanted to build a tool that works well for websites and REST APIs but also supports anything that communicates over TCP or UDP including websockets, game backends, and any other streaming protocols.

Supporting all types of technology teams also includes ones that do not want to send their test results and data to Testable.  Towards that goal, we will soon launch Testable Enterprise which will allow companies to either:

1. Run the entire Testable stack, including the website/backend/agents, on their own infrastructure
2. Run the Testable agent on their own infrastructure (potentially behind a firewall) and report results back to the central Testable backend and website.

Flexibility in how test scenarios are defined is important. Test scenarios can be defined by either:

1. Creating a recording with our gateway technology
2. Write Javascript that runs in a sandboxed Node.js environment

Once defined, these scenarios can be executed at scale across Testable’s global pool of agents. Using the most popular language in the world (Javascript) makes it more accessible to developers. And since we support all the common Node.js modules, any Node developer can easily write their scenario without learning anything new. If you don’t know Javascript or Node, we provide quite a few templates to help get you started. Or even record some interactions via our gateway and convert that to a script as your starting point.

While our approach to scripting is great, we also want to support the most popular open source tools including JMeter (live), Gatling (in the pipeline), and WebDriver (in the pipeline). For teams that have invested significant time into building test cases using these tools we want Testable to be a place you can come execute your tests at scale easily.

Providing best in class metrics, reporting, and analysis was also high on our list. This includes customizing the way results are viewed, as well as some harder things like capturing your own metrics and percentiles and viewing them in the results along many different dimensions (e.g. overall, by resource, region, interval, etc). Maybe you don’t like the list of percentiles we capture when measuring latencies, no problem!

At the same time we know there are some fantastic data analytics solutions out there and we plan to integrate with as many of them as possible. For now, we started with New Relic, but plan to add more in the future.

Ease of integration includes more than just shipping test results to 3rd parties like New Relic though.  It also means being able to trigger tests from Continuous Integration tools and getting notified via webhook when actions of interest occur on the platform.

Finally, providing API access for everything has always been on our minds. Our website and agents use the same API that we will (soon) document and expose to our users.  This means that every operation can be performed programmatically if so desired.

That’s it for now! Hopefully this post helps give a better idea of what we are about and where things are headed.  We would love to get as much feedback as possible from our users so feel free to reach out any time.