Storing & Executing Tests in Your Version Control

Execute tests stored outside of the Ghost Inspector system

Search our documentation:

Toggle Documentation Menu

Storing & Executing Tests in Your Version Control

Ghost Inspector now provides the ability to store tests offline in your own system and execute them on-demand. This can be useful when you want to keep your test in your own code base or version control system, instead of within the Ghost Inspector service.

One scenario where this can be beneficial is with code branches; on-demand tests will allow you to store your tests in your VCS and then modify them within your branch. Tests in your branch can be modified to suit the changes that you are making and then executed on-demand using Ghost Inspector in your CI process. This allows for immediate pass/fail feedback on the status of the branch.

Jump to...

On-demand Test Example

To create an on-demand test you can either download one of your tests from your account in Ghost Inspector (JSON) format, or you can create one manually.

On-demand tests only require three attributes, name, startUrl, and steps:

{
  "name": "On-demand Test",
  "startUrl": "https://www.ghostinspector.com/",
  "steps": [
    {
      "command": "assertTextPresent",
      "target": ".site-logo",
      "value": "Ghost Inspector"
    },
    {
      "command": "execute",
      "value": "5e34b17ea9096c7650f1df99",
      "notes": "Import Log In test"
    }
  ]
}

Now I can save this JSON to a file called myTest.json on my file system, or in my version control system like Git.


Executing an On-demand Test with Node.js

Our official Node.js client supports executing tests on-demand. In this next snippet, we will load up myTest.json, execute it using the Ghost Inspector API, and then poll for the status of the result:

// load the Ghost Inspector API client with our API key
const GhostInspector = require('ghost-inspector')('<my-api-key>')
// load our test
const myTest = require('./path/to/myTest.json')

// this function will let us execute the test using async/await
async function main () {
  const myOrganizationId = '<my-organization-id>'
  // we pass the `wait: true` option to wait for test completion
  const result = await GhostInspector.executeTestOnDemand(myOrganizationId, myTest, { wait: true })
  console.log(`Test status: ${result.passing}`)
}

// execute the test!
main()


Running the Test

Assuming we save the above code sample to a file called on-demand-test.js, we can execute it using Node.js:

$ node ./on-demand-test.js
Test status: passing


Test Attribute Reference

Here is a list of the variables that are available to a test (and steps) for your on-demand tests:

Attribute Description Options
autoRetry Automatically retry the test run if it fails when this is set to true. false (default), true
browser Specifies the browser to use during execution. chrome (default), chrome-<version>, firefox, firefox-<version>
disableVisuals Disable the screenshot and video features for this test run when true. false (default), true
failOnJavaScriptError Fail text execution when a JavaScript error is detected when true. false (default), true
finalDelay The length (in ms) the test will wait before taking the final screenshot. Any numeric value, defaults to 5000 (5 seconds).
globalStepDelay The length (in ms) the test will wait before taking the final screenshot. Any numeric value, defaults to 250.
maxAjaxDelay The max time (in ms) to wait for an XHR request to finish. Any numberic value, defaults to 10000 (10 seconds).
maxWaitDelay The max time (in ms) to wait for an element to be available. Any numberic value, defaults to 15000 (15 seconds).
name
(Required)
The name of the test. Any string value.
region Specifies the global region go execute the test in. us-east-1 (default), us-west-1, ca-central-1, eu-central-1, eu-west-1, eu-west-2, eu-west-3, eu-north-1, me-south-1, ap-east-1, ap-northeast-1, ap-northeast-2, ap-southeast-1, ap-southeast-2, ap-south-1, sa-east-1
screenshotCompareBaselineResult The ID of an existing test result to use for the screenshot comparison baseline. 24 character hex string.
screenshotCompareEnabled Perform a screenshot comparison using the result specified in screenshotCompareBaselineResult when true. false (default), true
startUrl
(Required)
The URL to begin the test from. Any string value.
steps
(Required)
The list of test steps to execute. Array of step objects, available properties are command,command, optional, target, value.
steps[N].command
(Required)
Specifies the type of action this step will take. Available options are assertElementNotPresent, assertElementNotVisible, assertElementPresent, assertElementVisible, assertEval, assertNotText, assertText, assertTextNotPresent, assertTextPresent, assign, click, dragAndDrop, eval, execute, exit, extract, extractEval, keypress, mouseOver, open, pause, refresh, screenshot, store.
steps[N].condition JavaScript script that will execute in the test to determine if the step should be run or not, step will be skipped when false. Any (sync) JavaScript, eg: return false;.
steps[N].optional Failing steps will be skipped when true. true or false (default).
steps[N].target CSS (or XPath) target to use for interactive test steps. Any valid CSS or XPath selector, eg: .submit-button.
steps[N].value Value to use with relative step action. String value will change according to step[N].command:
  • assertEval - JavaScript code
  • assertNotText - text value to check
  • assertText - text value to check
  • assertTextNotPresent - text value to check
  • assertTextPresent - text value to check
  • assign - variable value
  • eval - JavaScript code
  • execute - test ID to import
  • extractEval - JavaScript code
  • keypress - key code to use
  • open - URL to open
  • pause - time (in ms) to pause
  • store - string value to store
viewportSize The size to set the browser viewport for the test run Object, eg: {width: 1200, height: 800} (default)