Using Convoy to Backup and Recover a Wordpress MySQL database with Docker

The latest release of Docker Engine now supports volume plugins, which allow users to extend Docker capabilities by adding solutions that can create and manage data volumes for containers that need to manage and operate on persistent datasets.This is especially important for databases, and addresses one of the key limitations in Docker. Recently at Rancher we released Convoy, an open-source Docker volume driver that makes it simple to snapshot, backup, restore Docker volumes across clouds. In this post I will put Convoy into action, by using Convoy to snapshot and backup a database state for a Wordpress application, and will use the backup to create a replica in another datacenter. I’ll also cover incremental and scheduled backups, so that you can begin regularly backing up any stateful data running in containers.

Installing Convoy

For this walkthrough you’ll need to make sure your hosts are running Docker 1.8 or later before installing Convoy. I’ll be using Digital Ocean for this walkthrough, and will begin by downloading the latest release of Convoy and then starting the Convoy daemon on two DigitalOcean machines running in different data centers:

$ wget https://github.com/rancher/convoy/releases/download/v0.2.1/convoy.tar.gz
$ tar xvf convoy.tar.gz
$ sudo cp convoy/convoy convoy/convoy-pdata_tools /usr/local/bin/
$ sudo mkdir -p /etc/docker/plugins/
$ sudo bash -c 'echo "unix:///var/run/convoy/convoy.sock" > /etc/docker/plugins/convoy.spec'

The last two commands will enable the Convoy plugin with Docker. Since we are going to use snapshots with the container’s volume I will start Convoy with the Linux Device Mapper storage backend which has support for snapshots.

Wordpress Application with Convoy Storage

For this setup, I am going to start a Wordpress application container and MySQL database container. The MySQL container will use Convoy to create a volume for the container’s data, the following is a compose file that contains instructions to create both containers:

 wordpress:

  image: wordpress

  links:

    - mysql:mysql

  ports:

    - 80:80

mysql:

  image: mysql

  environment:

    MYSQL_ROOT_PASSWORD: "somebigpassword"

  volume_driver: convoy

  volumes:

    - wp_vol:/var/lib/mysql

This means that we will create a convoy volume for the MySQL container, this volume will store the database of the created Wordpress, volume on the first node. Run the compose file using docker-compose:

$ docker-compose up

To make sure that the volume is created, use the convoy command line to list all the volumes and snapshots:

$ convoy list

{

"36746db2-6964-40ee-8a50-1fd9b59c0da1": {

"UUID": "36746db2-6964-40ee-8a50-1fd9b59c0da1",

"Name": "wp_vol",

"Driver": "devicemapper",

"Size": 107374182400,

"MountPoint": "/var/lib/convoy/devicemapper/mounts/36746db2-6964-40ee-8a50-1fd9b59c0da1",

"CreatedTime": "Thu Aug 27 16:14:42 -0400 2015",

"DriverInfo": {

"DevID": "1",

"Driver": "devicemapper",

"MountPoint": "/var/lib/convoy/devicemapper/mounts/36746db2-6964-40ee-8a50-1fd9b59c0da1",

"Size": "107374182400"

},

"Snapshots": {}

}

}

The volume is created and mounted to a directory inside /var/lib/convoy/devicemapper. Let’s create a sample post on the wordpress to test the application:

Snapshotting The Wordpress Volume

Snapshots are saved states of a block devices, which are mountable and writable without interfering with the original content, to take a snapshot of the volume, use the following command:

$ sudo convoy snapshot create wp_vol --name wp_snap
$ sudo convoy list
…...

"Snapshots": {

"9fe78648-0da4-42c8-a153-0a9b2f355044": {

"UUID": "9fe78648-0da4-42c8-a153-0a9b2f355044",

"Name": "wp_snap",

"CreatedTime": "Thu Aug 27 17:00:14 -0400 2015",

"DriverInfo": {

"DevID": "2",

"Driver": "devicemapper",

"Size": "107374182400",

"UUID": "9fe78648-0da4-42c8-a153-0a9b2f355044",

"VolumeUUID": "36746db2-6964-40ee-8a50-1fd9b59c0da1"

}

…...

Convoy Backup

Convoy supports incremental backup of snapshots to various object stores, including Amazon S3 or NFS/VFS. This means that Convoy will backup incremental changes between snapshots, as they are taken. First let’s create an S3 bucket to store our backups, I’ll use the awscli to create my S3 bucket:

# aws s3 mb s3://backup-convoy
 make_bucket: s3://backup-convoy/

Now use the following command to backup the snapshot to the backup-convoy s3 bucket:

# time convoy backup create wp_snap --dest s3://backup-convoy@us-west-1/
s3://backup-convoy@us-west-1/?backup=c993e80f-7bad-412d-bc63-55b7adfd3738\u0026volume=4cd384c5-5121-4c9b-a834-9150b34a81f7

real 2m36.789s
user 0m0.020s
sys 0m0.008s

The command will backup the snapshot to the backup-convoy S3 bucket and then will print the backup url. You can see it took 2 minutes and 36 seconds to backup the snapshot to the S3 destination. To take a look on the delta block backup, which only copies the incremental data between snapshots, let’s create another post on wordpress and check the time for the newly created backup this time:

$ sudo convoy snapshot create wp_vol --name wp_snap2
$ sudo time convoy backup create wp_snap2 --dest s3://backup-convoy@us-west-1/

s3://backup-convoy@us-west-1/?backup=e4b4f0e2-dd48-48ed-8720-6b2d5e889fff\u0026volume=4cd384c5-5121-4c9b-a834-9150b34a81f7
real 0m9.185s
user 0m0.006s
sys 0m0.013s

The process this time took only 9 seconds which is the time taken to only backup the data created by the new post.

Replicating The Docker Container to Another Datacenter

Using the S3 backend as a centralized place for Docker volumes, will give us the ability to replicate the state of the Docker container to another datacenter, on another machine. First install Convoy and Docker on the new machine. And then restore the backup of the latest snapshot created earlier:

$ sudo convoy create wp_repl_vol --backup s3://backup-convoy@us-west-1/?backup=e4b4f0e2-dd48-48ed-8720-6b2d5e889fff\u0026volume=4cd384c5-5121-4c9b-a834-9150b34a81f7

Now this will restore the backup of the snapshots as a new volume called wp_repl_vol:

# convoy list

{

"c18ef169-52b7-47fa-9065-eeb8de55e551": {

"UUID": "c18ef169-52b7-47fa-9065-eeb8de55e551",

"Name": "wp_repl_vol",

"Driver": "devicemapper",

"Size": 107374182400,

"MountPoint": "",

"CreatedTime": "Thu Aug 27 19:04:43 -0400 2015",

"DriverInfo": {

"DevID": "1",

"Driver": "devicemapper",

"MountPoint": "",

"Size": "107374182400"

},

"Snapshots": {}

}

}

Editing the docker-compose.yml to mount this new volume to the data directory of mysql will make the container point to the restored wordpress database:

volume_driver: convoy
volumes:
    - wp_repl_vol:/var/lib/mysql

Now when I run docker-compose up it will create a replica of the wordpress site: Using a simple cronjob I can run a script to create regular snapshot of the targeted volume and back it up to my S3 object store:

#!/bin/bash


S3_BUCKET=backup-convoy

S3_REGION=us-west-1

VOL_NAME=wp_vol

DATE=`date +%y%m%d%H%M`

SNAPSHOT_NAME=wp_snap-$DATE

BACKUP_URL_FILE=/var/log/backupurl.log


/usr/local/bin/convoy snapshot create $VOL_NAME --name $SNAPSHOT_NAME

/usr/local/bin/convoy backup create $SNAPSHOT_NAME --dest s3://$S3_BUCKET@$S3_REGION/ > $BACKUP_URL_FILE

Conclusion

Data management is an essential part of any Docker environment, Docker Volumes and Convoy makes it easy to manage persistent data by adding a set of operations to manage Docker volumes, and soon Convoy will be incorporated into Rancher orchestration suite which will make it simple to manage and recover these backups directly from the Rancher UI, CLI and API. To learn more about Convoy, view a recording of our recent Convoy meetup, or request a demonstration from one of our engineers.