Sometimes when running an application; you want to persist data. A common use-case for this is as a redundancy measure so that if the application goes down, no data currently being processed by the application is lost.
In this example I will persist data from a container running on an EC2 onto an EBS volume. I will make the following assumptions of the reader:
- You want to automatically configure and start the application using a user-data script or equivalent; and therefore you will run the commands as the root user.
- You are using an EC2 AMI which has docker pre-installed; if you are not, you can find instructions to install docker on an EC2 here.
- You have an EBS volume attached to your EC2 instance as /dev/sda1.
I will use the official Logstash container available here.
Note: I will not prefix any of the commands here with “sudo”; this is because commands run in user-data scripts are run as root and therefore “sudo” is not required.
- Let’s use bash for our user-data script.
- Then create a file system on the external drive.
mkfs --type ext4 /dev/sda1
- Mount the external drive onto a mount-point in your Linux directory tree. This specific mount point will later be mounted onto the docker-container.
mount /dev/sda1 /Logstash
- This is a crucial and easily missed step. Change the owner of the directory that you want to mount onto the docker-container to a user other than the root user; on EC2’s the ec2-user is convenient.
chown ec2-user /apps/Logstash
- Enable and start Docker.
# Enable docker.
systemctl enable docker
systemctl start docker
- Run the docker image and mount the mount-point of the external drive onto the directory within the container, that contains the data to be persisted.
docker run –-mount type=bind,source=/apps/logstash/data,target=/usr/share/logstash docker.elastic.co/logstash/logstash:7.9.2If a mounted drive is owned by root; the application running on the docker container will be unable to write data to the mounted drive which can cause the following:
- Data is not persisted on external drive.
- Application failure to start successfully.