Write a Script

Introduction

Define a scenario using a script for maximum flexibility and customization. Scripts are written in Javascript and execute in a sandboxed Node.js environment.

A few notes about script execution:

  1. A full execution of the script is an iteration
  2. Any network operations (e.g. http, https, websocket, net, tls) will be measured (timing, bandwidth, success, etc) and the results captured and aggregated.
  3. The script page provides a dropdown to insert an example usage of common operations.

Create a Script

There are two ways to create a script:

  1. Click the 'New Test Case' button on the dashboard (or Test Case -> New... in the left nav) and select 'Script' during the 'Scenario' step.
  2. If a test case already exists, click on the test case name in the dashboard or left navigation, select the 'Scenarios' tab, and press 'New Scenario'. You can either start from scratch or use an existing recording as your starting point for the script.

Simple Example

In this example we call http://www.google.com in our script. This looks very simple:

http.get('http://wwww.google.com');

To check that the script works, click the 'Run' button in the top right. This will execute your script one time on an agent running in the AWS US East 1 region. The results will be logged in the 'Run Console' tab. Example output looks something like:

09/26/2015 06:34:08.536 PM Result Received for GET http://www.google.com
        Connected Successfully: true
        Response Code: 200
        Connection Opened Latency (ms): 13
        Connection Closed Latency (ms): 73
        Bytes Sent: 59 B
        Bytes Received: 184 KB
        Packets Sent: 1
        First Sent Packet (ms): 12
        Last Sent Packet (ms): 12
        Packets Received: 9
        First Received Packet (ms): 69
        Last Received Packet (ms): 70

Testable Utils

Testable provides several APIs as part of the testable-utils npm module. When run locally it will print to the console. When run via Testable it integrates seamlessly with the platform. More details can be found in the README or in the various sections below.

Logging

Logging from the script shows up in the 'Console' section of the test results. 5 logging levels are supported: fatal, error, info, debug, and trace. Fatal logging will cause the entire test to stop. Trace logging is only captured during smoke tests.

var log = require('testable-utils').log;

log.fatal("Use this to log the error and stop the entire test execution immediately.");
log.error("Use this to log errors to display in the results");
log.info("Use this to log info statements to display in the results");
log.debug("Use this for debugging. Remember that during a load test the scenario can get executed many times!");
log.trace("Only logged during a smoke test!");

Note: Keep in mind that your script can potentially execute many times depending on the test configuration. Each Testable account has a limit on logging per test and overall storage used. Check Settings -> Test Limits to see your limits. If any limit is breached the test will immediately stop executing.

Init/Teardown

The init and teardown scripts run exactly once globally per test execution. The init script runs before the test starts and the teardown script script runs after it completes.

The syntax and modules available are exactly the same as during the test.

Script Parameters

In some situations you may want to use the same script across multiple Test Configurations. In this case there may be certain parameters that need to be different for different configurations.

To define the parameters, use the Params section on the Script page. You can manage the expected parameters here.

To use a parameter in your script use the following syntax:

params[paramName]

Now create a new Test Configuration (it is one of the actions in the upper right of the Script page) and you will be prompted to specify values for any defined parameter.

Loading Additional Modules

If your script requires an NPM module that is not listed at the top of this guide, go ahead and try to use it and see if it is already available. For example:

var mysql = require('mysql');

Additional modules are not downloaded into our Node.js environment by default. When your script requires it, the module is installed and loaded dynamically for your use.

The full list of whitelisted modules is always changing. When writing your script select "Available NPM Modules" from the dropdown in the upper right to see the currently available full list. If you don't see your module there, please email support@testable.io to have it added.

All modules must be available on the public NPM module registry at https://npmjs.org to be eligible currently. Support for private repositories will be considered in the future.

Local Testing

Our testable-utils library is available on NPM and supports local execution. This allows you to run the tests locally before uploading them to Testable.

var utils = require('testable-utils');
var dataTable = utils.dataTable;
var log = utils.log;

// etc etc etc

Handing Async Flows

If you are using a Node.js module in your script that has async flows you need to indicate to Testable the start and finish of that flow. The following modules are exceptions where Testable instruments the module to handle the async flow: async, http, https, net, ws, socketio, engineio, tls, setTimeout, setInterval.

var testable = require('testable-utils').testable;

testable.start();
someModule.funcWithAsyncFlow('123', function() {
  // async callback
  console.log('do some stuff');
  testable.finish();
});

Or a callback flavor of the same API (using testable.execute(...)):

testable.execute(function(finished) {
  someModule.funcWithAsyncFlow('123', function() {
    // async callback
    console.log('do some stuff');
    finished();
  });
});

Remember the test script (.js file) must be uploaded on the Data block of the script.

Reading from a CSV

To provide different parameters for each iteration of the test script use this module. See the upload data page for more details about the API.

var testable = require('testable-utils').dataTable;

dataTable
  .open('demo.csv');
  .next()
  .then(function(rows) {
    http.get('http://sample.testable.io/stocks/' + rows[0].data['SYMBOL']);
  });

Capture Custom Metrics

To capture custom metrics in your test script use this module. See the custom metrics page for more details about the API.

var results = require('testable-utils').results;

results().counter('my-custom-counter', 2);
results().histogram('response-codes', '123');
results().timing('custom-timing', 225);

Stopwatch

Convenience function that executes your code, times how many milliseconds it takes, and captures it as a custom timing metric.

API

var stopwatch = require('testable-utils').stopwatch;
stopwatch(code, metricName[, resource]);

Example

timing(function(done) {
  // some operations go here
  done();
}, 'myCustomTimer');

Download a Remote File

To support downloading a file onto the agent that is required for running your script.

To get a readable stream to your file:

var dataFile = require('testable-utils').dataFile;

dataFile.open({ 
  name: 'http://mywebsite.com/somestuff.txt', 
  callback: function(file, finished) {
    // file is the result of caling the Nodejs fs.createReadStream(path) function
    console.log(file.path);
    // call finished once you are done with the file to clean it up proactively and so that the agent knows you are done processing it
    finished();
  }
});

To get the contents of your file as a buffer:

dataFile.get('http://myserver.com/someresource.txt', function (data) {
  console.log(data.toString());
});

NPM and Node.js Modules

Each section below details a Node.js or NPM module or package that is available for use in a script.

The following Node.js/NPM modules are always available to use during script execution:

  1. Request
  2. HTTP
  3. HTTPS
  4. Net
  5. TLS
  6. WebSocket
  7. socket.io-client
  8. engine.io-client
  9. Lodash
  10. Math
  11. moment
  12. setTimeout/clearTimeout
  13. process
  14. util
  15. url
  16. uuid
  17. async
  18. jsonfile
  19. har-replay

Testable allows for additional whitelisted NPM modules to be downloaded on demand. See the Loading Additional Modules section for more details.

Request Module

All options provided by the request NPM module are supported.

An example GET request:

request.get('http://sample.testable.io/stocks/IBM');

An example POST request:

var req = request.post('http://httpbin.org/post', { 
  headers: {
    'X-Test-Header': 'blablabla'
  }
}, function(err, res, body) {
  log.info('BODY: ' + body);
});
req.write('request body test');
req.end();

In the above example we call POST http://httpbin.org/post with a X-Test-Header header and request body test as the body. See the module documentation for the full range of options.

HTTP Module

All options provided by the client side of the Node.js HTTP module are supported. This includes http.get() and http.request().

An example POST request:

var req = http.request({ 
  hostname: 'httpbin.org', 
  path: '/post', 
  method: 'POST', 
  headers: {
    'X-Test-Header': 'blablabla'
  }
}, function(res) {
  res.on('data', function (chunk) {
    log.info('BODY: ' + chunk);
  });
});
req.write('request body test');
req.end();

In the above example we call POST http://httpbin.org/post with a X-Test-Header header and request body test as the body. See the Node.js documentation for full details of the options.

To make an HTTP request without Testable tracking and reporting metrics (e.g. reporting the start of a test to your servers):

httpNoTracking.get('https://myserver.com');
HTTPS Module

All options provided by the client side of the Node.js HTTPS module are supported. This includes http.get() and http.request().

https.get('https://www.google.com');

To make an HTTPS request without Testable tracking and reporting metrics (e.g. reporting the start of a test to your servers):

httpsNoTracking.get('https://myserver.com');
Net Module

All options provided by the client side of the Node.js Net module are supported.

var client = net.connect({ host: 'sample.testable.io', port: 8091 }, function() {
  // connected!
  client.write('test echo message');
});

client.on('data', function(data) {
  log.info(data.toString());
  client.end();
});

client.on('end', function() {
  log.info('disconnected from server');
});
TLS Module

All options provided by the client side of the Node.js TLS module are supported.

var wss = new WebSocket("wss://wss.websocketstest.com/service");

wss.on('open', function open() {
  wss.send('echo,test message');
});

wss.on('message', function(data, flags) {
  log.info(data);
  wss.close();
});

wss.on('error', function(error) {
  log.error(error);
  wss.close();
});
WebSocket Module

All options provided by the client side of the 'ws' NPM package module are supported.

In the below example we test the sample HTTP/WS service.

var ws = new WebSocket("ws://sample.testable.io/streaming/websocket");

ws.on('open', function open() {
  ws.send('{ "subscribe": "IBM" }');
});

ws.on('message', function(data, flags) {
  log.info(data);
  ws.close();
});

ws.on('error', function(error) {
  log.error(error);
  ws.close();
});
Socket.io Client Module

See the socket.io-client documentation for the full set of options.

The below example connects to our sample Socket.io echo service.

var socket = socketio('http://sample.testable.io:5811');
socket.on('connect', function(){
  log.info('connected');
  socket.emit('message', 'This is a test');
});
socket.on('event', function(data){
  log.info(data);
  socket.close();
});
socket.on('disconnect', function(){
  log.info('disconnected');
});
Engine.io Client Module

See the engine.io-client documentation for the full set of options.

The below example connects to our sample Engine.io echo service.

var socket = engineio('http://sample.testable.io:5812');
socket.on('open', function(){
  log.info('opened');
  socket.send('This is a test'); 
  socket.on('message', function(data){
    log.info(data);
    socket.close();
  });
  socket.on('close', function(){
    log.info('closed');
  });
});
Lodash Module

See the Lodash documentation for all the functions it supports.

var symbols = ['IBM', 'MSFT', 'AAPL'];
_.forEach(symbols, function(symbol) {
  http.get('http://sample.testable.io/stocks/' + symbol);
});
Math Module

Any function of the Math object can be used in a script.

var rand = Math.random();
if (rand > 0.5) {
  // do one thing here
} else {
  // do something else
}
moment Module

Any function of the momentjs can be used in a script.

log.info("Current timestamp: " + moment().valueOf());
setTimeout/clearTimeout

Use the setTimeout() function to add a delay to your script. The below example delays the entire script by 100ms.

setTimeout(function() {
  // your scenario code here

}, 100); // 100ms delay
Process Module

The following functions/properties in the Node.js process module are supported:

  1. process.uptime()
  2. process.hrtime()
  3. process.memoryUsage()
  4. process.env
  5. process.arch
  6. process.platform
log.info(util.inspect(process.memoryUsage()));
Util Module

All functions in the Node.js util module are supported.

var test = { a: b, c: d};
log.info(util.inspect(test));
URL Module

All functions in the Node.js url module are supported.

UUID Module

All functions in the NPM uuid module are supported.

Async Module

All functions in the NPM async module are supported.

jsonfile Module

All read functions in the NPM jsonfile module are supported.

var jsonfile = require('jsonfile');
var myJsonObj = jsonfile.readFileSync('myUploaded.json');
har-replay Module

An NPM module, har-replay, which supports reading and replaying the contents of a HTTP Archive (HAR). See the README for more details.

var harReplay = require('har-replay');
harReplay.load('myUploaded.har');

Execution Info

During execution the info object provides information on the current execution context. This object is unique per concurrent user and is accessible as a global variable in your script. It includes:

  1. Execution: Details of the execution including id, concurrent clients, duration/iterations, etc
  2. Agent: Unique identifier for the agent on which this test iteration is executing
  3. Region: Region (id, name, description) where the test iteration is executing
  4. Chunk: Within each region, the execution is broken into chunks. The details of the current chunk are provided here.
  5. Client: Within each chunk each concurrent user is assigned a unique ID. Remember that it is only unique within the chunk.
  6. Iteration: Within each concurrent client each iteration is assigned an increasing id.
  7. Unique ID: To get a unique natural ID corresponding to this iteration, use info.currentId(). This will return a string that combines all of the above values into a unique natural key.
  8. Output Directory: Directory to output files that you want to capture as part of the test results. This directory will be captured on a couple of test iterations per minute, not necessarily on every one.
  9. Context: A way to maintain state across iterations of a single concurrent user. Any property that is part of this object will get passed between one concurrent user's iterations of the script. Any properties assigned to this object must be serializable to JSON (i.e. no functions). See the maintaining state across iterations section for more details.

It is also accessible via require('testable-utils').info for better local testing compatibility.

An example of the info structure:

{ iteration: 98,
  client: 2,
  chunk: 
   { id: 47,
     executionType: 'Main',
     agent: '238b9add-e342-4e3f-af53-131ea9a866c7',
     createdAt: '2015-09-28T17:45:20.187Z',
     updatedAt: '2015-09-28T17:45:20.188Z',
     startedAt: '2015-09-28T17:45:20.512Z',
     iterations: 5,
     concurrentClients: 1 
   },
  agent: '238b9add-e342-4e3f-af53-131ea9a866c7',
  execution: 
   { id: 48,
     createdAt: '2015-09-28T17:45:10.611Z',
     updatedAt: '2015-09-28T17:45:17.120Z',
     startedAt: '2015-09-28T17:45:13.519Z',
     iterations: 5,
     concurrentClients: 1
   },
  region: 
   { id: 1,
     createdAt: '2015-08-11T22:03:34.761Z',
     updatedAt: '2015-08-11T22:03:34.761Z',
     name: 'aws-us-east-1',
     public: true,
     latitude: 39.0436,
     longitude: -77.4878,
     description: 'AWS N. Virginia',
     active: true 
   },
   outputDir: '/tmp/some/path/here',
   context: 
   { authToken: 'example-abcdef'
   }
 }

Maintaining State Across Iterations

As mentioned above, the execution info object provides a mechanism for passing state between each concurrent user's iterations of a script.

It is accessible in your script via info.context and by default is an empty object. Use this object to maintain session state like authentication details.

if (!info.context.myAuthToken) {
  // get the auth token on the first iteration of this concurrent user
  info.context.myAuthToken = 'abcdef';
}

// use the auth token
console.log('Token: ' + info.context.myAuthToken);