Running Docker containers in production normally doesn’t need mounting local files or dirs. But when using Docker for development, it makes sense to mount the source code to avoid baking the Docker image on every change. That looks straight forward except for some cases.

Some files or directories are generated by package managers or compilers during the compile time. They should be generated as part of the Docker image build and kept consistent during the runtime. On the other hand, mounting other files that need to be changed constantly during development is necessary.

This post uses an example app to experience different ways of using Docker volumes and learn about their pros and cons.

Let’s start with a sample Node application that uses Yarn for package management.

Here is the Dockerfile:

FROM node:12

# Create app directory
WORKDIR /usr/src/app

# Install app dependencies
COPY package.json ./
RUN yarn install

# Bundle app source
COPY . .

EXPOSE 8080
CMD [ "npm", "start" ]

A minimal package.json:

{
  "name": "sample",
  "main": "server.js",
  "scripts": {
    "start": "nodemon server.js"
  },
  "license": "ISC",
  "dependencies": {
    "express": "^4.17.1",
    "nodemon": "^2.0.4"
  }
}

express is used as the webserver library and nodemon is used to auto-reload the server on code changes.

And a simple webserver as server.js:

const http = require('http');

const hostname = '127.0.0.1';
const port = 3000;

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.end('OK');
});

server.listen(port, hostname, () => {
  console.log(`Server running at http://${hostname}:${port}/`);
});

Docker Compose is the easiest way to use Docker for development. To prevent building the image on every change we can simply mount the current host directory . to the container working dir /usr/src/app. Like this docker-compose.yml file:

services:
  web:
    build: .
    ports:
      - "3000:3000"
    volumes:
      - .:/usr/src/app

The following command should build the image and run the application:

docker-compose up --build

But it returns the following error:

web_1  | sh: 1: nodemon: not found

Which means nodemon is not installed by Yarn even though we made sure we installed all dependencies by putting RUN yarn install in the Dockerfile.

To see what is going on inside the Docker container we can bash into the container by:

docker-compose run web bash

It spins up a new container but rather than running npm start as specified in the Dockerfile, it runs a bash prompt.

Now we can see the app dir content inside the container:

root@760cd71c0919:/usr/src/app# ls
Dockerfile  docker-compose.yml  package.json  server.js

node_modules is missing. It should’ve been generated at build time by RUN yarn install in the Dockerfile. It’s missing because we mount the current host directory to /usr/src/app inside the container and there’s no node_modules on the host directory.

      [CONTAINER]                              [HOST]

.   -----------------------------------> .
├── docker-compose.yml                   ├── docker-compose.yml
├── Dockerfile                           ├── Dockerfile
├── package.json                         ├── package.json
└── server.js                            └── server.js

Basically the whole content of /usr/src/app is replaced by current directory of the host machine. Removing volumes from the docker-compose.yml can prove it:

 services:
   web:
     build: .
     ports:
       - "3000:3000"
-    volumes:
-      - .:/usr/src/app

Now, all files are back:

$ docker-compose run web bash
root@58620eea72ef:/usr/src/app# ls
Dockerfile  docker-compose.yml  node_modules  package.json  server.js  yarn.lock

And this time, running the container will work without any issue:

$ docker-compose up
Starting node-sample_web_1 ... done
Attaching to node-sample_web_1
web_1  | 
web_1  | > sample@ start /usr/src/app
web_1  | > nodemon server.js
web_1  | 
web_1  | [nodemon] 2.0.4
web_1  | [nodemon] to restart at any time, enter `rs`
web_1  | [nodemon] watching path(s): *.*
web_1  | [nodemon] watching extensions: js,mjs,json
web_1  | [nodemon] starting `node server.js`
web_1  | Server running at http://0.0.0.0:3000/

Now it’s clear why mounting the whole application directory is not a good idea. We need a way to exclude node_modules from the volume mounting. Through it’s not supported out of the box by Docker, there are some workarounds.

Re-Mounting Excluded Path

By changing docker-compose.yml like this:

 services:
   web:
     build: .
     ports:
       - "3000:3000"
+    volumes:
+      - .:/usr/src/app
+      - /usr/src/app/node_modules

Docker will create a clean anonymous volume only for /usr/src/app/node_modules dir and populate it with the Docker image content at the same path:

      [CONTAINER]                              [HOST]

.  ------------------------------------> .
├── docker-compose.yml                   ├── docker-compose.yml
├── Dockerfile                           ├── Dockerfile
├── package.json                         ├── package.json
├── server.js                            └── server.js
└── node_modules  -----------+
    ├──@sindresorhus         |
    ├──@szmarczak            |
    ├──abbrev                |
    .                        +---------> [Anonymous Volume]
    .
    .
    └── xdg-based

As we can see the content of node_modules inside the container:

$ docker-compose exec web bash
root@ef9eccaf2e4e:/usr/src/app# ls node_modules
@sindresorhus	   cacheable-request	 debug		      fill-range	    imurmurhash		     json-buffer	nodemon		qs		     statuses		   update-notifier
...

We can also see the newly created volume by running docker volume ls:

DRIVER              VOLUME NAME
local               e8d983d966df0b5770763bfacf5b40c87f43ea16496268e971f7d0c38e2e45e9

By running docker volume ls you might see other volumes from your past Docker usage. You can run docker system prune --volumes to remove unused volumes and try the experiments in this post.

Docker propagates this volume only during the creation and will keep it around no matter if the corresponding image files are changed. This causes big trouble when we need to update node_modules content. To see it in action, add a new dependency to package.json:

 {
   "name": "sample",
   "main": "server.js",
   "scripts": {
     "start": "nodemon server.js"
   },
   "license": "ISC",
   "dependencies": {
     "express": "^4.17.1",
     "nodemon": "^2.0.4",
+    "axios": "^0.20.0"
   }
 }

And add a dependency in server.js like:

const axios = require('axios');

Building and running the image results in this error:

web_1  | Error: Cannot find module 'axios'

Because node_modules do not contain axios module inside the container:

root@8430da938308:/usr/src/app# ls node_modules/axios
ls: cannot access 'node_modules/axios': No such file or directory

This can be solved by running docker-compose down and then up again. It’s because Docker abandons the previously created volume and creates a new volume propagated with the new image node_modules content. Running docker volume ls shows that there’s a new volume added:

DRIVER              VOLUME NAME
local               e8d983d966df0b5770763bfacf5b40c87f43ea16496268e971f7d0c38e2e45e9
local               ecbfc6a56ed7c288b333584a8f5a646bb5289968249a22167a0cdbfa13cd9fd4

So, this approach has two issues:

  1. Inconsistency: The volume is initialized as a copy of the image content but there’s no guarantee it will be identical to the image content
  2. Dangling Volumes: Even though stopping the container will release the volume and creates a new one on the next run, it left a dangling volume every time.

Mounting Files Selectively

Another approach is to mount only files and directories that are needed. So rather than excluding a directory, we just add the ones we need to.

The following changes to the docker-compose.yml will do the job:

 services:
   web:
     build: .
     ports:
       - "3000:3000"
     volumes:
-      - .:/usr/src/app
-      - /usr/src/app/node_modules
+      - ./server.js:/usr/src/app/server.js

This time Docker won’t create any new volumes and any file other than server.js is guaranteed to be loaded from the image content.

      [CONTAINER]                              [HOST]

.                                        .
├── docker-compose.yml                   ├── docker-compose.yml
├── Dockerfile                           ├── Dockerfile
├── package.json                         ├── package.json
├── server.js   -----------------------> └── server.js
└── node_modules
    ├──@sindresorhus
    ├──@szmarczak
    ├──abbrev
    .
    .
    .
    └── xdg-based

In this case it was only server.js that needs to be mounted. In some cases there are a few directories and files but in any case, it has to be tracked by the developers.

Conclusion

This was an example to demonstrate how Docker volume mounting can be used during development. The same concept applies in different situations and for different stacks.

Here is a summary and comparison of methods demonstrated:

Method Pros Cons
Mounting the Whole Dir Easy Corrupts the docker content - Basically, not usable
Re-Mounting Excluded Paths Needs to specify only excluded dirs Inconsistent, Dangling Images
Mounting Selectively Consistent and clear, probably best method Needs tracking all newly added files and dirs by development team