Continuous Integration Testing for WordPress
28 May 2019
While continuous integration is a common practice for most development teams, the stateful nature of WordPress makes it difficult, but not impossible, to setup. For our open source WordPress plugin, we wanted to integrate our standard build and test process for every pull request using CircleCI. While it might be easier to setup a permanent staging environment, we wanted every build to be isolated for dependable testing. That meant being able to start from scratch without worrying about maintaining yet another environment. Thankfully, we were able to leverage Docker and the WordPress CLI to make this happen.
In the following tutorial I’ll show you how you can build the same setup using freely available tools. Of course we’ll also be using Ghost Inspector to run our automated UI tests, but you can use any tool once you have your WordPress environment running. Also, because we’re using mostly Docker commands, with a few tweaks you should be able to use this setup locally or on any CI provider (not just CircleCI). Let us know in the comments if you’re able to get this working on Travis CI or anywhere else!
To be able to use this exact config, you will need accounts with:
You can jump straight to the full config file, but I’ll be breaking this down and explaining each step as I go. By the end, you’ll see how we use CircleCI to setup a Docker network that installs WordPress and triggers automated UI tests against that local instance using ngrok.
version: 2 executors: docker-executor: docker: - image: circleci/node
Our config starts by specifying the version of CircleCI we want to use. Then, under
executors we specify that we want to use Docker. Here’s what CircleCI says about that:
The docker key defines Docker as the underlying technology to run your jobs using Docker Containers. Containers are an instance of the Docker Image you specify and the first image listed in your configuration is the primary container image in which all steps run.
Since we’ll be using this primary container to build our WordPress plugin, we need an image with Node.js installed. By using CircleCI’s convenience image we get Docker installed on the container as well, which we’ll use to setup more containers.
Build Code On Current Git Branch
jobs: test_plugin: executor: docker-executor steps: - checkout - run: name: build plugin command: cd frontend && npm install && npm run export
We only have one job defined and we’ve chosen to call it
test_plugin. Every step in this tutorial will be under this single job. The first step checks out our code from GitHub. Next we build the plugin by navigating to the
frontend directory, installing npm dependencies, and running our custom
export script. This generates a zip file that we’ll be using later on.
Setup Docker Network
- setup_remote_docker - run: name: Set up network command: | docker network create wp-network
In order to run Docker commands on CircleCI, we have to tell it to setup a remote, fully-isolated Docker environment. The first thing we do with this new Docker environment is create a network that we’re naming
wp-network. We’ll use that name in future steps to communicate between containers. In the network we’ll have separate containers for:
- WordPress CLI
- Ghost Inspector
You could also achieve this same setup with a single container, either by installing everything you need on the running container, or using a custom image with everything baked in. We chose to go with multiple containers for many reasons:
- It allows us to use existing images with a lot of support. There are simply more maintainers for the MySQL image and the WordPress image separately than there are for any image that has both.
- It’s easier to test with different versions. While this setup uses MySQL 5.7 and the latest version of WordPress, we also have access to all past and future versions available on DockerHub as image tags that we can swap in with a single line change (or add as new steps to test multiple at once).
- It’s easier to troubleshoot individual steps. With Docker, you are often working with a black box. Having each layer in it’s own container makes it easier to isolate and fix problems.
Setup MySQL Database
- run: name: Set up database command: | docker run -d \ -e MYSQL_ROOT_PASSWORD=1234 \ -e MYSQL_DATABASE=wordpress \ --name db \ --network wp-network \ mysql:5.7
The first container we start is MySQL, since it takes the longest to get running. There is a possible race condition where you are trying to connect to a database that isn’t available yet. To prevent that, add a step using dockerize wait or a similar utility. However in our setup it’s not necessary.
When we start the container with
docker run, we set some environment variables that the image will use when setting up the MySQL database:
MYSQL_ROOT_PASSWORDis the password for the user
root(which our WordPress config will use)
MYSQL_DATABASEis the initial database that gets created
We also use the Docker
name parameter to give the container a name we can reference later, and the
network parameter to tell it to use the same network that we created in the previous step.
- run: name: Setup WordPress command: | docker run -d \ -e WORDPRESS_DB_HOST=db:3306 \ -e WORDPRESS_DB_USER=root \ -e WORDPRESS_DB_PASSWORD=1234 \ -e WORDPRESS_DB_NAME=wordpress \ -e WORDPRESS_CONFIG_EXTRA="define('WP_SITEURL', 'http://' . \$_SERVER['HTTP_HOST']); define('WP_HOME', 'http://' . \$_SERVER['HTTP_HOST']);" \ --name wp-container \ --network wp-network \ wordpress
This starts the WordPress container using the official WordPress image. Just like MySQL, we set some environment variables that the container will use when setting up:
WORDPRESS_DB_HOSTuses the name we set for the MySQL container (
db) and the default port for MySQL (3306)
WORDPRESS_DB_USERis set to the
WORDPRESS_DB_PASSWORDis set to the same value used for
MYSQL_ROOT_PASSWORDon the database container
WORDPRESS_DB_NAMEis set to the same value used for
MYSQL_DATABASEon the database container
WORDPRESS_CONFIG_EXTRAis used to inline some PHP that will be added to
wp-config.php-- these definitions are required for ngrok
- run: name: Install WordPress command: | docker run -it --rm \ --volumes-from wp-container \ --network wp-network \ wordpress:cli core install \ --url=localhost \ --title=test \ --admin_user=admin \ --admin_password=admin \ --email@example.com
Our third container uses WP-CLI to install WordPress without interacting with the UI. Again we tell Docker to use the same network. This image is just the CLI and does not contain WordPress, so we set
volumes-from to the name of the WordPress container so it has access to those files. The rest of the parameters after
core install are the values you would normally type in when setting up WordPress for the first time. Other than the
url, you can use any values you want here.
- run: name: Install Relative URL Plugin command: | docker run -it --rm \ --volumes-from wp-container \ --user xfs \ --network wp-network \ wordpress:cli plugin install relative-url \ --activate \
This uses the WP-CLI again to install the Relative URL plugin, which is another requirement to enable ngrok. In one command, this downloads the plugin from the official directory, installs, and activates it. We’re going to use the same command in our next step to install the Ghost Inspector plugin.
Install Plugin From Current Git Branch
- run: name: copy Ghost Inspector plugin command: docker cp ghost-inspector.zip wp-container:/var/www/html - run: name: Install Ghost Inspector plugin command: | docker run -it --rm \ --volumes-from wp-container \ --user xfs \ --network wp-network \ wordpress:cli plugin install /var/www/html/ghost-inspector.zip \ --activate
The first step copies the zip file that was built in step one, from the current container to the WordPress container. By default, WordPress is installed in
/var/www/html so we put it there for consistency. Again we use WP-CLI’s command:
plugin install, but instead of giving it a slug, we give the absolute path to the zip file we just copied into the container. This step is the magic that ensures our tests are being run against the code in the current branch.
Trigger UI Tests
- run: name: execute Ghost Inspector test(s) command: | docker run \ -e APP_PORT=wp-container:80 \ -e GI_API_KEY=$GI_API_KEY \ -e GI_SUITE=$GI_SUITE \ -e NGROK_TOKEN=$NGROK_TOKEN \ -e GI_PARAM_wp_username=admin \ -e GI_PARAM_wp_password=admin \ --network wp-network \ ghostinspector/test-runner-standalone workflows: test: jobs: - test_plugin
For our final step, we use the Ghost Inspector standalone Docker image, which does all the work of connecting to ngrok and kicking off our UI tests. We just give it the right environment variables and the name of our Docker network and it handles the rest. For the environment variables we use:
APP_PORTset to the name of our WordPress container (
wp-container) and the port it’s running on (
GI_API_KEYis our Ghost Inspector API key (save this and the next two to CircleCI environment variables for your project)
GI_SUITEis our Ghost Inspector suite ID
NGROK_TOKENis our ngrok API token/key
GI_PARAM_wp_usernameis the same
admin_userwe set when installing WordPress --
wp_usernameis a variable used in our Ghost Inspector tests
GI_PARAM_wp_passwordis the same
admin_passwordwe set when installing WordPress --
wp_passwordis a variable used in our Ghost Inspector tests
The final part of our CircleCI config is the
workflows definition, which for now, is just a single job. With this, CircleCI will install WordPress, our automated UI tests will be triggered and run with Ghost Inspector, which connects to the ngrok URL generated for this build. CircleCI will wait for Ghost Inspector to return a pass or fail for the tests. Here’s a video from a recent test using this exact config:
You can see the exact steps we used to create this test by reading our post on automated UI testing for WordPress.
Using this setup, you should be able to run any automated UI tests for WordPress. You could also install different versions of WordPress, or even different versions of MySQL and PHP by leveraging tagged Docker images.