Storing & Executing Tests in Your Version Control

Ghost Inspector now provides the ability to store tests offline in your own system and execute them on-demand. This can be useful when you want to keep your test in your own code base or version control system, instead of within the Ghost Inspector service.

One scenario where this can be beneficial is with code branches; on-demand tests will allow you to store your tests in your VCS and then modify them within your branch. Tests in your branch can be modified to suit the changes that you are making and then executed on-demand using Ghost Inspector in your CI process. This allows for immediate pass/fail feedback on the status of the branch.

Table of Contents

  • On-demand Test Example

  • Executing an On-demand Test with Node.js

  • Test Attribute Reference

On-demand Test Example

To create an on-demand test you can either download one of your tests from your account in Ghost Inspector (JSON) format, or you can create one manually.

On-demand tests only require three attributes, name, startUrl, and steps:

{
  "name": "On-demand Test",
  "startUrl": "https://www.ghostinspector.com/",
  "steps": [
    {
      "command": "assertTextPresent",
      "target": ".site-logo",
      "value": "Ghost Inspector"
    },
    {
      "command": "execute",
      "value": "5e34b17ea9096c7650f1df99",
      "notes": "Import Log In test"
    }
  ]
}

Now I can save this JSON to a file called myTest.json on my file system, or in my version control system like Git.

Executing an On-demand Test with Node.js

Our official Node.js client supports executing tests on-demand. In this next snippet, we will load up myTest.json, execute it using the Ghost Inspector API, and then poll for the status of the result:

// load the Ghost Inspector API client with our API key
const GhostInspector = require('ghost-inspector')('<my-api-key>')
// load our test
const myTest = require('./path/to/myTest.json')

// this function will let us execute the test using async/await
async function main() {
  const myOrganizationId = '<my-organization-id>'
  // we pass the `wait: true` option to wait for test completion
  const result = await GhostInspector.executeTestOnDemand(myOrganizationId, myTest, { wait: true })
  console.log(`Test status: ${result.passing}`)
}

// execute the test!
main()

Running the Test

Assuming we save the above code sample to a file called on-demand-test.js, we can execute it using Node.js:

$ node ./on-demand-test.js
Test status: passing

Test Attribute Reference

Here is a list of the variables that are available to a test (and steps) for your on-demand tests:

_id
Link the execution to an existing test in your organization.
Valid options: 24 character hex string
autoRetry
Automatically retry the test run if it fails when this is set to true.
Valid options: false (default), true
browser
Specifies the browser to use during execution.
Valid options: chrome (default), chrome-<version>, firefox, firefox-<version>
disableVisuals
Disable the screenshot and video features for this test run when true.
Valid options: false (default), true
failOnJavaScriptError
Fail text execution when a JavaScript error is detected when true.
Valid options: false (default), true
finalDelay
The length (in ms) the test will wait before taking the final screenshot.
Valid options: Any numeric value, defaults to 5000 (5 seconds)
globalStepDelay
The length (in ms) the test will wait before taking the final screenshot.
Valid options: Any numeric value, defaults to 250
maxAjaxDelay
The max time (in ms) to wait for an XHR request to finish.
Valid options: Any numberic value, defaults to 10000 (10 seconds)
maxWaitDelay
The max time (in ms) to wait for an element to be available.
Valid options: Any numberic value, defaults to 15000 (15 seconds)
name
(Required)
The name of the test.
Valid options: Any string value
region
Specifies the global region go execute the test in.
Valid options: us-east-1 (default), us-west-1, ca-central-1, eu-central-1, eu-west-1, eu-west-2, eu-west-3, eu-north-1, me-south-1, ap-east-1, ap-northeast-1, ap-northeast-2, ap-southeast-1, ap-southeast-2, ap-south-1, sa-east-1
screenshotCompareBaselineResult
The ID of an existing test result to use for the screenshot comparison baseline.
Valid options: 24 character hex string
screenshotCompareEnabled
Perform a screenshot comparison using the result specified in screenshotCompareBaselineResult when true.
Valid options: false (default), true
startUrl
(Required)
The URL to begin the test from.
Valid options: Any string value
steps
(Required)
The list of test steps to execute.
Valid options: Array of step objects, available properties are command,command, optional, target, value
steps[#].command
(Required)
Specifies the type of action this step will take.
Valid options: assertElementNotPresent, assertElementNotVisible, assertElementPresent, assertElementVisible, assertEval, assertNotText, assertText, assertTextNotPresent, assertTextPresent, assign, click, dragAndDrop, eval, execute, exit, extract, extractEval, keypress, mouseOver, open, pause, refresh, screenshot, store
steps[#].condition
JavaScript script that will execute in the browser session to determine if the step should be run (if true is returned) or skipped (if false is returned).
Valid options: Any valid JavaScript code, ex: return false;
steps[#].optional
Test will continue if this step fails when true.
Valid options: false (default), true
steps[#].target
CSS or XPath element target to use for interactive test steps.
Valid options: Any valid CSS or XPath selector, ex: .submit-button
steps[#].value
Value to use with relative step action.
Valid options: String value will change according to step[N].command:
  • assertEval - JavaScript code
  • assertNotText - text value to check
  • assertText - text value to check
  • assertTextNotPresent - text value to check
  • assertTextPresent - text value to check
  • assign - variable value
  • eval - JavaScript code
  • execute - test ID to import
  • extractEval - JavaScript code
  • keypress - key code to use
  • open - URL to open
  • pause - time (in ms) to pause
  • store - string value to store
viewportSize
The size to set the browser viewport for the test run
Valid options: Object with width and height fields, ex: { width: 1200, height: 800 } (default)