How to reproduce command `docker run` via Docker Remote API with Node.js

This articles assumes a basic understanding of docker and its ecosystem.

TL;DR

I enjoy very much the concept of containers and became fascinated by the possibilities they enable. For an experiment I was doing, I needed a way to spin up some containers, send one time stdin data to them and receive its response via stdout, and I managed to implement this programatically via the Docker Remote API using Node.js.

Initially, I had some trouble because this part in the Remote API is a little bit tricky but after some time I finally achieved success and that’s why I wish to contribute with my findings on this topic.

Later, in the implementation section I will cover the execution details of one-off commands running inside Docker containers.

Background

Docker is one of the existent container technologies, and it has become one of the most popular tech stacks along with a great ecosystem of tools built around it.

Docker has a daemon server that keeps running on your machine and it exposes a Remote API for managing your containers. The most popular way to interact with containers is by using the Docker CLI tool.

If you ever used docker and its CLI, it’s for sure you’ve run into this command:

docker run <options> <image>.

The interesting part is that the Docker CLI uses the Docker Remote API under the hood (be sure to check this awesome post about the unix socket the daemon listens to). Additionally, the run command has not a direct corresponding API endpoint to run containers, and it is composed by three main steps through the API that create, attach and start your containers.

How `docker run` CLI command is composed under the hood

As I stated before, the Docker CLI abstracts the feature of "running a container". That way we can have a better and easy experience while not having to deal with each of the required steps separately.

For starters, let’s explain how it works:

Normally, you would use the command by informing a docker image along with some options to manage the container execution, by sending some data to the container, mounting volumes, enabling tty feature for interactive mode, and so on. Here, I will focus on the task of running a container, passing data into the stdin of the container, and obtaining its response via the stdout.

When you call docker run it will begin its internal execution. First, it will call the create endpoint and a container is then created and allocated with a unique ID and this container remains stopped. Then, it will call the attach endpoint. It means that it will connect to the stdin/stdout/stderr of the container, allowing the container to read data and generate an output based on the received data. After the attachment is complete, it will call the start endpoint, which starts the container. After the container execution, the response will be shown and the container will remain stopped, unless the option --rm was passed — in this case the container will be destroyed right after the end of its execution.

As this feature requires three steps to complete it’s ok to expect a little overhead during its execution, as the whole process is a little bit intensive. This overhead is covered by some experiments on the following post.

Implementation

Description

The goal of this implementation is to create a container, send some data through stdin and get the response from the stdout. I am using this great library to communicate with the Docker API called dockerode, which abstracts the direct access to the Docker API and provides some utilities like streaming stdin/stdout through the tcp connection.

We will use the following command as base for our implementation:

Command: echo "hello world" | docker run -i alpine tr [a-z] [A-Z]

Explanation: Here we pipe the string hello world via stdin to the docker cli run command. It will forward the input string to to the Docker container via the Docker API. On the run command we also specify some arguments: the alpine image which is a tiny image, the tr command that will (in this case) capitalize all the letters from the input string and the -i flag that informs the container will receive data via stdin.

Result: HELLO WORLD

Code

Execution

Wrapping up

Docker containers enable lots of cool possibilities like the example of this article which is really simple, but we could create an application that would act upon receiving data and perform complex operations with it like code compilation and runtime execution. We could spawn containers on demand to deal with some tasks, and if we think about it, it is what serverless actually is. On serverless world, containers can be scaled on-demand. However, the process of spawning containers should be managed by container orchestrators like kubernetes or docker swarm. There are lots if interesting open-source projects like these, and each one has some particularities. The project I enjoyed most was OpenFaas, as it integrates with the orchestrator quite easily. I am also keeping an eye on Knative, as it is on its way to be a good foundation for future serverless projects.