Continuous Integration Testing for WordPress

While continuous integration is a common practice for most development teams, the stateful nature of WordPress makes it difficult, but not impossible, to setup. For our open source WordPress plugin, we wanted to integrate our standard build and test process for every pull request using CircleCI. While it might be easier to setup a permanent staging environment, we wanted every build to be isolated for dependable WordPress testing. That meant being able to start from scratch without worrying about maintaining yet another environment. Thankfully, we were able to leverage Docker and the WordPress CLI to make this happen.

In the following tutorial I’ll show you how you can build the same setup using freely available tools. Of course we’ll also be using Ghost Inspector to run our automated UI tests, but you can use any tool once you have your WordPress environment running. Also, because we’re using mostly Docker commands, with a few tweaks you should be able to use this setup locally or on any CI provider (not just CircleCI). Let us know in the comments if you’re able to get this working on Travis CI or anywhere else!

Getting Started

To be able to use this exact config, you will need accounts with:

• ngrok
• CircleCI
• Ghost Inspector

CircleCI Config

You can jump straight to the full config file, but I’ll be breaking this down and explaining each step as I go. By the end, you’ll see how we use CircleCI to setup a Docker network that installs WordPress and triggers automated UI tests against that local instance using ngrok.

version: 2
executors:
  docker-executor:
    docker:
      - image: circleci/node

Our config starts by specifying the version of CircleCI we want to use. Then, under executors we specify that we want to use Docker. Here’s what CircleCI says about that:

The docker key defines Docker as the underlying technology to run your jobs using Docker Containers. Containers are an instance of the Docker Image you specify and the first image listed in your configuration is the primary container image in which all steps run.

Since we’ll be using this primary container to build our WordPress plugin, we need an image with Node.js installed. By using CircleCI’s convenience image we get Docker installed on the container as well, which we’ll use to setup more containers.

Build Code On Current Git Branch

jobs:
  test_plugin:
    executor: docker-executor
    steps:
      - checkout
      - run:
          name: build plugin
          command: cd frontend && npm install && npm run export

We only have one job defined and we’ve chosen to call it test_plugin. Every step in this tutorial will be under this single job. The first step checks out our code from GitHub. Next we build the plugin by navigating to the frontend directory, installing npm dependencies, and running our custom export script. This generates a zip file that we’ll be using later on.

Setup Docker Network

- setup_remote_docker
- run:
    name: Set up network
    command: |
      docker network create wp-network

In order to run Docker commands on CircleCI, we have to tell it to setup a remote, fully-isolated Docker environment. The first thing we do with this new Docker environment is create a network that we’re naming wp-network. We’ll use that name in future steps to communicate between containers. In the network we’ll have separate containers for:

• MySQL
• WordPress
• WordPress CLI
• Ghost Inspector

You could also achieve this same setup with a single container, either by installing everything you need on the running container, or using a custom image with everything baked in. We chose to go with multiple containers for many reasons:

1. It allows us to use existing images with a lot of support. There are simply more maintainers for the MySQL image and the WordPress image separately than there are for any image that has both.
2. It’s easier to test with different versions. While this setup uses MySQL 5.7 and the latest version of WordPress, we also have access to all past and future versions available on DockerHub as image tags that we can swap in with a single line change (or add as new steps to test multiple at once).
3. It’s easier to troubleshoot individual steps. With Docker, you are often working with a black box. Having each layer in it’s own container makes it easier to isolate and fix problems.

Setup MySQL Database

- run:
    name: Set up database
    command: |
      docker run -d \
        -e MYSQL_ROOT_PASSWORD=1234 \
        -e MYSQL_DATABASE=wordpress \
        --name db \
        --network wp-network \
        mysql:5.7

The first container we start is MySQL, since it takes the longest to get running. There is a possible race condition where you are trying to connect to a database that isn’t available yet. To prevent that, add a step using dockerize wait or a similar utility. However in our setup it’s not necessary.

When we start the container with docker run, we set some environment variables that the image will use when setting up the MySQL database:

• MYSQL_ROOT_PASSWORD is the password for the user root (which our WordPress config will use)
• MYSQL_DATABASE is the initial database that gets created

We also use the Docker name parameter to give the container a name we can reference later, and the network parameter to tell it to use the same network that we created in the previous step.

Install WordPress

- run:
    name: Setup WordPress
    command: |
      docker run -d \
        -e WORDPRESS_DB_HOST=db:3306 \
        -e WORDPRESS_DB_USER=root \
        -e WORDPRESS_DB_PASSWORD=1234 \
        -e WORDPRESS_DB_NAME=wordpress \
        -e WORDPRESS_CONFIG_EXTRA="define('WP_SITEURL', 'http://' . \$_SERVER['HTTP_HOST']); define('WP_HOME', 'http://' . \$_SERVER['HTTP_HOST']);" \
        --name wp-container \
        --network wp-network \
        wordpress

This starts the WordPress container using the official WordPress image. Just like MySQL, we set some environment variables that the container will use when setting up:

• WORDPRESS_DB_HOST uses the name we set for the MySQL container (db) and the default port for MySQL (3306)
• WORDPRESS_DB_USER is set to the root user
• WORDPRESS_DB_PASSWORD is set to the same value used for MYSQL_ROOT_PASSWORD on the database container
• WORDPRESS_DB_NAME is set to the same value used for MYSQL_DATABASE on the database container
• WORDPRESS_CONFIG_EXTRA is used to inline some PHP that will be added to wp-config.php — these definitions are required for ngrok

- run:
    name: Install WordPress
    command: |
      docker run -it --rm \
        --volumes-from wp-container \
        --network wp-network \
        wordpress:cli core install \
          --url=localhost \
          --title=test \
          --admin_user=admin \
          --admin_password=admin \
          --admin_email=foo@bar.com

Our third container uses WP-CLI to install WordPress without interacting with the UI. Again we tell Docker to use the same network. This image is just the CLI and does not contain WordPress, so we set volumes-from to the name of the WordPress container so it has access to those files. The rest of the parameters after core install are the values you would normally type in when setting up WordPress for the first time. Other than the url, you can use any values you want here.

- run:
    name: Install Relative URL Plugin
    command: |
      docker run -it --rm \
        --volumes-from wp-container \
        --user xfs \
        --network wp-network \
        wordpress:cli plugin install relative-url \
          --activate \

This uses the WP-CLI again to install the Relative URL plugin, which is another requirement to enable ngrok. In one command, this downloads the plugin from the official directory, installs, and activates it. We’re going to use the same command in our next step to install the Ghost Inspector plugin.

Install Plugin From Current Git Branch

- run:
    name: copy Ghost Inspector plugin
    command: docker cp ghost-inspector.zip wp-container:/var/www/html
- run:
    name: Install Ghost Inspector plugin
    command: |
      docker run -it --rm \
        --volumes-from wp-container \
        --user xfs \
        --network wp-network \
        wordpress:cli plugin install /var/www/html/ghost-inspector.zip \
          --activate

The first step copies the zip file that was built in step one, from the current container to the WordPress container. By default, WordPress is installed in /var/www/html so we put it there for consistency. Again we use WP-CLI’s command: plugin install, but instead of giving it a slug, we give the absolute path to the zip file we just copied into the container. This step is the magic that ensures our tests are being run against the code in the current branch.

Trigger UI Tests

- run:
          name: execute Ghost Inspector test(s)
          command: |
            docker run \
              -e APP_PORT=wp-container:80 \
              -e GI_API_KEY=$GI_API_KEY \
              -e GI_SUITE=$GI_SUITE \
              -e NGROK_TOKEN=$NGROK_TOKEN \
              -e GI_PARAM_wp_username=admin \
              -e GI_PARAM_wp_password=admin \
              --network wp-network \
              ghostinspector/test-runner-standalone
workflows:
  test:
    jobs:
      - test_plugin

For our final step, we use the Ghost Inspector standalone Docker image, which does all the work of connecting to ngrok and kicking off our UI tests. We just give it the right environment variables and the name of our Docker network and it handles the rest. For the environment variables we use:

• APP_PORT set to the name of our WordPress container (wp-container) and the port it’s running on (80)
• GI_API_KEY is our Ghost Inspector API key (save this and the next two to CircleCI environment variables for your project)
• GI_SUITE is our Ghost Inspector suite ID
• NGROK_TOKEN is our ngrok API token/key
• GI_PARAM_wp_username is the same admin_user we set when installing WordPress — wp_username is a variable used in our Ghost Inspector tests
• GI_PARAM_wp_password is the same admin_password we set when installing WordPress — wp_password is a variable used in our Ghost Inspector tests

The final part of our CircleCI config is the workflows definition, which for now, is just a single job. With this, CircleCI will install WordPress, our automated UI tests will be triggered and run with Ghost Inspector, which connects to the ngrok URL generated for this build. CircleCI will wait for Ghost Inspector to return a pass or fail for the tests. Here’s a video from a recent test using this exact config:

You can see the exact steps we used to create this test by reading our post on automated UI testing for WordPress.

Conclusion

Using this setup, you should be able to run any automated UI tests for WordPress. You could also install different versions of WordPress, or even different versions of MySQL and PHP by leveraging tagged Docker images.