Getting Started - Serenity BDD

Follow this guide for a basic example of running your Serenity BDD tests on the Testable platform. Our example project will open https://www.google.com, run various searches, and capture any screenshots.

Start by signing up and creating a new test case using the New Test Case button on the dashboard or Test Case -> New... in the left side panel.

Enter the test case name (e.g. Serenity Demo) and press Next.

Scenario

Select Selenium as the scenario type.

Selenium Scenario

Let's use the following settings:

  1. Bindings: Selenium Java
  2. Framework: Serenity
  3. Source: Build Project from Version Control. See the detailed documentation for all the options to get our code onto Testable.

Selenium Java Settings

We will use the serenity-example project as our source.

This project contains 3 tests that open Google and perform a search. By default Serenity will also capture screenshots after each step. See the serenity-example code for more details.

And that's it, we've now defined our scenario! To try it out before configuring a load test click the Smoke Test button in the upper right and watch Testable execute the scenario 1 time as 1 user. You should see all results including logging, assertions, screenshots, network traces, etc appear once the smoke test completes.

Next, click on the Configuration tab or press the Next button at the bottom to move to the next step.

Configuration

Now that we have the scenario for our test case we need to define a few parameters before we can execute our test:

  1. Load Profile: Select Flat to ramp up to a constant number of concurrent users for the test.
  2. Concurrent Users Per Region: Number of users that will execute your Serenity tests in parallel per region selected. Each concurrent user will run the tests using JUnit + Serenity Runner and communicate with a shared local Selenium server on the test runner.
  3. Test Runners: Choose the test runners that will run this test (e.g. on our public shared grid). Each test runner region will simulate the number of concurrent users defined above.
  4. Type: Select Iterations to have each concurrent user execute the Webdriver.io script sequentially a set number of times regardless of how long it takes. Choose Duration if you want each user to continue executing the script for a set amount of time (in minutes).

And that's it! Press Start Test and watch the results start to flow in. See the new configuration guide for full details of all configuration options.

For the sake of this example, let's use the following parameters:

Test Configuration

View Results

Once the test starts executing, Testable will distribute the work out to the selected test runners (e.g. Public Shared Grid in AWS N. Virginia).

Test Results

In each region, the test runners execute your project's tests in 3 separate processes concurrently for 1 minute or until it completes if it takes longer than a minute.

The results will include screenshots, assertions, traces, performance metrics, logging, breakdown by URL, analysis, comparison against previous test runs, and more.

Check out the Serenity guide for more details on running your Serenity code on the Testable platform.

We also offer integration (Settings -> Integration) with third party tools like New Relic. If you enable integration you can do more in depth analytics on your results there as well.

That's it! Go ahead and try these same steps with your own scripts and feel free to contact us with any questions.