Metrics Glossary

This page contains defintions for each standard metric that Testable captures during test execution. The name in parantheses is used in our API. On our website the name outside parantheses is used.

  • Active Concurrent Users (actualConcurrentClients) Counter: Number of concurrent users currently active within your test. Once the test completes this number will always be zero.

  • Bandwidth (bandwidth) Counter: Total bandwidth measured in bytes.

  • Breaking Point (breakingPoint) Counter: If your test hits a breaking point, this reports the number of concurrent users that were executing when the breaking point was hit.

  • Bytes Received (bytesReceived) Counter: Total bytes received in bytes.

  • Bytes Sent (bytesSent) Counter: Total bytes sent in bytes.

  • Connection Close (connectionCloseMs) Timing: The number of milliseconds from when a request starts until the connection is closed.

  • Connection Open (connectionOpenMs) Timing: The number of milliseconds from when a request starts until the connection is opened.

  • Count (count) Counter: The number of network requests that were executed during your test including HTTP requests, websockets, and TCP sockets opened.

  • CPU (cpu) Metered: The number of vCPUs used by test runners to run your test. This is available by test runner as well as aggregated across all test runners.

  • CPU % (cpuUtilization) Metered: The percent of all available vCPUs utilized while running your test. This is available by test runner as well as aggregated across all test runners.

  • DNS Lookup (dnsLookupMs) Timing: The number of milliseconds from when a request starts until the DNS lookup for the host name is resolved. This metric is not available for PhantomJS, SlimerJS, Gatling, and JMeter tests because the tools do not expose it.

  • HTTP Content Type (httpContentType) Histogram: A histogram that keeps track of how many responses of each content type is received, based on the Content-Type header. Only applicable for HTTP testing.

  • HTTP Method (httpMethod) Histogram: A histogram that keeps track of how many times each HTTP method is invoked. Only applicable for HTTP testing.

  • HTTP Response Code (httpResponseCode) Histogram: A histogram that keeps track of how many times each HTTP response code is received on a response. Only applicable for HTTP testing. Does not account for connection errors.

  • Iteration Time (iterationExecutedMs) Timing: The number of milliseconds to execute an entire iteration of your scenario.

  • Iterations Executed (iterationsExecuted) Counter: The number of iterations of your test scenario that were executed during the test.

  • JMeter Active Threads (jmeterConcurrents) Metered: The number of active JMeter threads across all instances/test runners.

  • Max Concurrent Users (maxConcurrentClients) Counter: Peak number of simulated users that were ever executing concurrently during your test.

  • Memory (memory) Metered: The memory used on the test runners, in bytes, while running your test. This is available by test runner as well as aggregated across all test runners.

  • Memory % (memoryUtilization) Metered: The percent of available test runner memory utilized while running your test. This is available by test runner as well as aggregated across all test runners.

  • Outcome (outcome) Histogram: This histogram has two buckets: success and failure. Each network call is either a success or a failure. For HTTP requests, it is successful if a response code < 400 is received. For all other types of connections (e.g. websocket, TCP), success is if the connection opens successfully and closes without any error occuring.

  • Packets Received (dataPacketsReceived) Counter: Number of packets received. For example, on an HTTP chunked response, each chunk counts as a packet. On a websocket, every frame received counts as a packet. On a raw TCP socket, every time the socket data event is fired counts as a packet.

  • Packets Sent (dataPacketsSent) Counter: Number of packets sent. For example, on an HTTP chunked request, each chunk counts as a packet. On a websocket, every frame sent counts as a packet. On a raw TCP socket, every time the socket.write() method is called counts as a packet.

  • Page Load (pageLoadMs) Timing: The number of milliseconds to complete a page load in PhantomJS or SlimerJS. Only applicable for PhantomJS and SlimerJS tests.

  • Peak Requests/Sec (peakRequestsPerSec) Counter: Peak number of requests per second measured during the test. A request can be any HTTP request, websocket, or TCP socket opened.

  • Requests/Sec (requestsPerSec) Counter: Average number of requests per second. A request can be any HTTP request, websocket, or TCP socket opened.

  • Response Code (responseCode): Histogram: Similar to HTTP Response Code except it has an Error bucket for connection failures and a Connected bucket for non-HTTP test scenarios.

  • Response Time (firstReceivedMs) Timing: The number of milliseconds from when a request starts until the first packet is received from the server. For an HTTP request this is the first packet (of potentially many with chunked encoding). For a websocket it is the time until the first frame AFTER the handshake is completed. For a raw TCP socket it is the time until the socket data event fires for the first time.

  • Test Runner Bandwidth (testRunnerBandwidth) Metered: Peak throughput, in bytes/sec, required by the test runners to run your test. This is available by test runner as well as aggregated across all test runners.

  • Test Runner Bytes Received (testRunnerReceived) Metered: Peak throughput received, in bytes/sec, required by the test runner to run your test. This is available by test runner as well as aggregated across all test runners.

  • Test Runner Bytes Sent (testRunnerSent) Metered: Peak throughput sent, in bytes/sec, required by the test runner to run your test. This is available by test runner as well as aggregated across all test runners.

  • Test Runner Connections (testRunnerConnections) Metered: Number of TCP connections open on the test runner including all TCP stats (ESTABLISHED, TIME_WAIT, CLOSED, etc). This is available by test runner as well as aggregated across all test runners.

  • Test Runner Connections Established (testRunnerConnectionsEstablished) Metered: Number of TCP connections open on the test runner whose current state is ESTABLISHED. This is available by test runner as well as aggregated across all test runners.

  • Throughput (throughput) Counter: Average throughput measured in bytes/sec.

  • Virtual Users Started (clientsStarted) Counter: Number of virtual users started during this test. This can be different than the reported Max Concurrent Users because not all virtual users necessarily run concurrently.

  • Virtual Users Finished (clientsFinished) Counter: Number of virtual users finished during this test. If the test is stopped early this number can be lower than the number of virtual users started.

  • Webdriver.io Assertions (wdioAssertions): Histogram: For Selenium Webdriver.io tests that are written with Mocha assertions this histogram tracks the number of assertions that were passed, skipped, and failed.