Tutorials

Pre-baking couchbase docker images for faster unit tests


In this post we’ll create pre-baked couchbase docker images that we can use for running tests locally or as part of our CI/CD pipeline.

Background

Let’s say that you’re developing an application that uses couchbase for its persistence layer. For your application to work, it expects the couchbase cluster to be configured in a very specific way. It has to have the required node types (query, index, data), the required buckets, indexes, views, etc…
If we could somehow have a ready-to-use disposable docker image of the exact couchbase cluster that our application needs, that would be very useful. We could use it for our unit tests, for our CI/CD and for running our application locally without having to configure the database. We could spin up a new couchbase cluster before the tests starts and dispose of it when they finish. The question is how can we build a docker image that already contains a ready-to-use couchbase cluster?

Initializing couchbase at runtime

This seems to be the most widely used approach for bootstrapping couchbase docker images. You can see the original post here. The idea is to create a setup script that will be run when the container starts instead of the default command that comes with the couchbase image. This setup script will first start the couchbase service in the background and, once the service is running, it will proceed to configure the cluster. Very straightforward.

DockerFile

FROM couchbase

COPY setup.sh /opt/couchbase/

CMD /opt/couchbase/setup.sh

setup.sh

#!/bin/bash

# Start couchbase in the bg
/entrypoint.sh couchbase-server &

check_db() {
  curl --silent http://127.0.0.1:8091/pools > /dev/null
  echo $?
}

until [[ $(check_db) = 0 ]]; do
  sleep 1
done

# setup cluster
couchbase-cli cluster-init --cluster-username=admin --cluster-password=admin123 --cluster-port=8091 --cluster-ramsize=500 --cluster-index-ramsize=256 --services=data,index,query

#setup buckets
for i in `seq 1 5`;
do
    couchbase-cli bucket-create -c localhost --bucket=bucket$i --bucket-type=couchbase --bucket-ramsize=100 -u admin -p admin123
	sleep 5
	for f in `seq 1 5`;
	do
		cbq -e localhost -u admin -p admin123 --script="CREATE INDEX index${i}_${f} on \`bucket$i\`(field$f);"
	done
done

wait

The setup file launches the couchbase service, defines a new cluster and creates five buckets with five indexes each. This approach is satisfactory and once the container is launched, the script will run and eventually our couchbase cluster will be ready.

The only problem I have with this approach is that it can take quite a while from the moment the container starts until the couchbase cluster is ready. For instance, the setup script above takes about 2:30 minutes to finish. This might not seem like a long time at first, but if you recall, the main reason we’re doing this is because we want to spin up a new container every time we run our test suite. In that context, anything that increases the total running time of our tests by more than a few seconds, is too long. In the next section we’ll see how we can make the container start faster.

Initializing couchbase at image build time

If couchbase was already initialized by the time the container starts, then we wouldn’t need to wait so long every time we spin up a new container. What we’ll do is initialize couchbase when the image is built, rather than when the container starts. Obviously, this means that the build time will increase, but since the image will rarely be updated while our tests will be run continuously, locally, and by the CI/CD pipeline, this a trade-off that is worth making.

The main change we need to the Dockerfile file is to run the setup.sh at build time – that is, to change RUN /opt/couchbase/setup.sh to CMD/opt/couchbase/setup.sh

DockerFile

FROM couchbase-no-volume

COPY setup.sh /opt/couchbase/

RUN /opt/couchbase/setup.sh

You’ll also notice that this new DockerFile is using a different base image. This is because the official couchbase image won’t work for this. The reason it won’t work is that its DockerFile defines the couchbase data directory as volume, which will make Docker reset the contents of the directory whenever the container is first started. That means that the changes we’ve made during build time, would be overridden when the container starts. The solution is very simple: create a custom couchbase base image that doesn’t define the data directory as a volume. You can just copy the corresponding DockerFile (for example version 5.0 is here), remove the VOLUME definition and build your custom volume-less image. The only caveat here is that all changes will be lost when the container is removed, but since we’re using these containers as disposable containers for our tests, that won’t be a problem.

And, in case you’re wondering how long it takes for the couchbase cluster to be fully usable once the container has been launched, that would be 5 seconds now :).

Programming Patterns
Efficient WebVR Development Using the Adapter Pattern in Three.js
Tutorials
Better Testing of Microservices Using Consumer-Driven Contracts in Node.js
Tutorials
Mesh Manipulation Using Mean Values Coordinates in Three.js
There are currently no comments.