2017. május 10., szerda

Variable subtitution with older docker-compose


If you are working in a really big project you are using Agile methodologies I am 99% sure.
You know all the "definitions" why it is good, how it speeds up the development, etc...

But there is one thing what I have faced many times. The big projects can react the customer needs in a short time frame (being agile you know), but sometimes it is not flexible enough to keep up with developer needs and infrastructure changes.

We are living in a time where we release our new features in 2 weeks or couple of hours.

But in the background, where security is in the first place, it is hard to change anything, because it requires audits, approvals and money.

All in all we are still using Docker 1.9.1 and docker-compose 1.3.3. Yeah, I know it's shitty, but this is what we have and what we have to improve.

The problem

We were building a test environment which can be started with one command and with some parameters to help everyone spin up our app in local env.

Of course we targeted to use vagrant/VirtualBox/docker.

But we would like to specify which version of the app or dependecies we want to use.

The Solution

It's in evidence we need some variables and use it in the compose file
But there is one little thing. Docker-compose got this feature in 1.5.0+(release notes)

How can we "workaround" this?

We have infinite number of possible solution.
Just some hints:
  • Use templating engines like: Jinja2, Freemarker, etc.
  • Use string replace
  • Or what we have choose use small script to do it
Our choice was envsubst which can be found in the gettext package.

To install it:

Let's try it with a small example. We would like to start a python container for backward compatibility testing (maybe not the best example, but it's good enough)

Here is our docker-compose file:
Now run docker-compose up with 1.3.3 version
Yes, as we expected it's failed because "python_version" variable has not been resolved.

Let's try it with a newer version of compose (1.9.0)

Is it better now? Of course it is. But we cannot update the production server :/
So let's use envsubst.

The usage is really simple: pipe the input to the script and it will print to the output.
It's really simple.
After that we can use our generated docker-compose file to start up our services.

Here is an example how we use it:

2017. május 9., kedd

Who am I?

Who am I? This is a really good question.
Most of the people answers his/her name.
Let's start with that.
My name is Laszlo Tarcsanyi.
I am a proud father of a clever, almost 2 years old son and a proud husband of my beautiful wife. We have a cocker spaniel called Szotyi.

From the other hand I am a Senior Software Test Automaton Engineer. At least this is my current job title. I am a Linux, docker, python, shell script fan. Really! I like writing scripts for almost everything. This is why I would like to start this blog. To store my thoughts somewhere and let you read it, eat it, or throw it away.

My plan is to create a simple how-tos or know-hows (it depends on which company you are working for :) ). I would like to use Debian, CentOs and ArchLinux for installation instruction if needed. I will not post evenly, just where I feel that this is what should be shared or can be interesting for others or I will need it later.

Oh, one more thing. I am not long-winded. And I think I need to practice my english. I know. Many people said that.

Variable subtitution with older docker-compose

Background If you are working in a really big project you are using Agile methodologies I am 99% sure. You know all the "definitions...