Running Suite Files

node-async-testing assumes you are going to have a one to one mapping between suites and files. So, to run a test suite, you actually tell it to run a file:

require('async_testing').run('test-suite.js');

The run function can take a file name or a directory name (it recursively searches directories for javascript files that start with test-) or an array of any combination of those two options.

In order for node-async-testing to run a file, the exports object of the file needs to be the test suite:

exports['first test'] = function(test) { ... };
exports['second test'] = function(test) { ... };
exports['third test'] = function(test) { ... };

// or

module.exports = {
  'first test': function(test) { ... },
  'second test': function(test) { ... },
  'third test': function(test) { ... }
};

To make a file easy to execute with the node command, we need to make our file run the suite when it is the script being ran. Some where in the file we put this code (I like to put it at the very top):

// if this module is the script being run, then run the tests:
if (module === require.main) {
  return require('async_testing').run(__filename);
}

Assuming we put that suite in a file called test-suite.js, we can now execute the it by running the following on the command line:

node test-suite.js

The run function can also be passed the process.ARGV array of command line arguments, so node-async-testing settings can be altered at run time:

if (module === require.main) {
  return require('async_testing').run(process.ARGV);
}

exports['first test'] = function(test) { ... };
exports['second test'] = function(test) { ... };
exports['third test'] = function(test) { ... };

Now, you can tell node-async-testing to run the tests in parallel:

node test-suite.js --parallel

Or to only run specific tests:

node test-suite.js --test-name "first test" --test-name "third test"

Use the help flag to see all the options:

node test-suite.js --help

node-async-testing can run multiple files at once this way, because additional files will get passed with process.ARGV:

node test-suite.js test-suite2.js

For example, you can run all the tests in a test directory by saying:

node test/*

With this arrangement, the exit code of the process will be the number of tests that didn't succeed.

node-async-testing comes with a "web" test runner. This runner launches a web server which can be used to run suites manually. Launch it with the --web flag:

node test/* --web

Once the server is started, in a browser you can pick and choose which suites to run, and run them as many times as you like. node-async-testing reloads the suites (and any code they use) from scratch each time they are run so you can leave the web server running and switch back and forth between editing tests or code and running the tests. Very handy!

To use the web runner you also need to install Socket.IO.

npm install socket.io

[The server is known to work in the lastest versions of Safari, Chrome and Firefox. Any help in getting it to work in Opera would be much appreciated. I don't have means of testing in IE, so I can't tell you how it performs there.]

Running suites using the API

If you want to organize your suites in a different manner (and say not have them organized by file), or don't like the included test runners, you are going to have to run your suites manually or write your own runner.

node-async-testing comes with the following functions for running suites and test runners:

runSuite()
runFile()
expandFiles()
run()
registerRunner()

runSuite(suiteObject, options)

The runSuite function is the heart of node-async-testing. It receives two arguments.

The first argument is the actual suite that you want to run. See Writing Tests for the details of writing a suite.

The second argument is an object with options for running the suite. This object can have the following properties:

name: This is the name of the test suite being run. This is optional, as it doesn't affect the running of the suite at all. If it is provided it is passed to the onSuiteStart event callback.
parallel: If this property is present and “truethy”, then the tests will be run in parallel. Otherwise, runSuite won't start another test until the previous one has completely finished.
testName: This should be either a String or an Array of Strings. If this property is present then only those tests whose names match it will be ran. Use this to only run specific tests.

In addition to those properties, the options in the events callbacks section are supported.

runSuite adds a listener to the process for uncaught exceptions, and as such, there should be no other code running while runSuite is doing its thing, otherwise the other code could interfere with the suite being ran.

runFile(suitePath, options)

The runFile function is similar to runSuite except instead of running the suite in the main process, it opens up a child process and runs it there. The benefits of this are that it is able to report on syntax errors, and also be run while other suites are running.

The first argument is a module path to the suite for Node's require() function.

The second argument is an object with options for running the suite. It is practically the same as the second argument for runSuite, except it has one for additional event callback. See the events callbacks section for the full list.

expandFiles(list, [filter], callback)

The expandFiles function takes a list of files and or directories and returns a list of just files. It recursively searches through any directories for files that begin with test-. It is useful for expanding a user supplied list of files and directories. It takes three properties:

A String or an Array of Strings, the list of files and directories to expand.
A String or an Array of Strings, a list of file names by which you want to filter the found files. This makes it so you can specify specific file names that you want to find. This is optional.
Callback, which will get called with the found files when expandFiles is done.

It returns an array of objects which have the following properties:

name: This is the file name that was passed in.
path: This is an absolute module path to the file that can be require()ed.

run()

The run functions is talked about at length in the Running Suite Files section. It handles outputing the results of suites for you, so you don't have to worry about it. node-async-testing comes with two built-in runners, one for consoles and one for web browsers.

run can take any number of arguments, which can be any one of:

A string, the name of a file to run. The exports object of the file should be a suite object.
An array of command line arguments, like process.ARGV.
An options object, for manipulating the options for the function directly. The options you can set here correspond directly to the ones you can set in via the command line.
The last argument can be a callback which will get called when all the specified suites have finished running. The callback will receive one argument, an array with an object for each suite, which has the name, status and suite result object for that suite.

If a callback is not supplied, node-async-testing assumes that you don't want to do anything after the suites have completed, so the run function creates its own callback which exits the process and sets the exit code to the number of suites that didn't complete successfully.

The order matters, latter settings override earlier ones. Here are some examples:

Run a file:

async_testing.run('myFile.js');

Change options:

async_testing.run({parallel: true}, 'myFile.js');

Use command line arguments:

async_testing.run(process.ARGV);

Use command line arguments with defaults:

async_testing.run({parallel: true}, process.ARGV);

Overwrite command line arguments:

async_testing.run(process.ARGV, {parallel: true});

registerRunner(modulePath, [default])

Use this function to add your own custom runners to node-async-testing. See lib/console-runner.js or lib/web-runner.js for examples of how to write a runner.

The first argument is the path to the runner which you are registering. The second variable is for whether or not you want this to be the default runner.

Description of event callbacks

The runSuite and runFile functions can be given event callbacks for outputing the results of the tests. Using these callbacks it is possible to write your own test runners and format the output however you'd like. These callbacks are not for manipulating tests, but purely for reporting.

Events

onTestStart(testName): Called when a test is started.
onTestDone(status, testResult): Called when a test finishes. See Test result below. status is one of the following:
failure or success.
onSuiteDone(status, suiteResult): Called when a suite finishes. See Suite result below. status is one of the following:
complete, error, exit or loadError.

Suite Result

A suite result is an object that looks like one of the following, depending on what the finish status of the suite was:

complete

occures when all tests finished running, and node-async-testing was able to accurately determine how each one finished.

{ tests: an Array of test results for each test that completed
, numFailures: the number of tests that failed
, numSuccesses: the number of tests that passed
}

error

occures when an uncaught error is thrown from a test and node-async-testing can't determine which test caused it. When this happens, node-async-testing stops running the suite and exits the process.

{ error: the error object that was thrown
, tests: an Array of the names of each test that could have caused the error
}

exit

occures when the process running the suite exits and there are still tests that haven't finished. This occurs when people forget to finish their tests or their tests don't work like they expected.

{ tests: an Array of the names of each test that didn't finish
}

loadError

this type of result is only produced from runFile, and occures when the child process can't load the suite.

{ stderr: what was written to <stderr> before the child process exited
}

Test Result

A test result is an object that looks like one of the following, depending on what the finish status of the test was:

success

the test completed successfully

{ name: test name
, numAssertions: number of assertions completed successfully
}

failure

the test failed in some way

{ name: test name
, failureType: 'assertion' or 'error' depending on how this failed
, failure: the error that caused this to fail
}

node-async-testing

Simple, intuitive testing for node.js