Should a developer be able to create a docker artifact from a lerna monorepo in their development environment?

Question

I've recently started using lerna to manage a monorepo, and in development it works fine.

Lerna creates symlinks between my various packages, and so tools like 'tsc --watch' or nodemon work fine for detecting changes in the other packages.

But I've run into a problem with creating docker images in this environment.

Let's say we have a project with this structure:

root
  packages
     common → artifact is a private npm package, this depends on utilities, something-specific
     utilities → artifact is a public npm package
     something-specific -> artifact is a public npm package
     frontend → artifact is a docker image, depends on common
     backend → artifact is a docker image, depends on common and utilities

In this scenario, in development, everything is fine. I'm running some kind of live reload server and the symlinks work such that the dependencies are working.

Now let's say I want to create a docker image from backend.

I'll walk through some scenarios:

1. I ADD package.json in my Dockerfile, and then run npm install.

   Doesn't work, as the common and utilities packages are not published.

2. I run my build command in backend, ADD /build and /node_modules in the docker file.

   Doesn't work, as my built backend has require('common') and require('utilities') commands, these are in node_modules (symlinked), but Docker will just ignore these symlinked folders.

   Workaround: using cp --dereference to 'unsymlink' the node modules works. See this AskUbuntu question.

3. Step 1, but before I build my docker image, I publish the npm packages.

   This works ok, but for someone who is checking out the code base, and making a modification to common or utilities, it's not going to work, as they don't have privledges to publish the npm package.

4. I configure the build command of backend to not treat common or utilities as an external, and common to not treat something-specific as an external.

   I think first build something-specific, and then common, and then utilities, and then backend.

   This way, when the build is occuring, and using this technique with webpack, the bundle will include all of the code from something-specfic, common and utilities.

   But this is cumbersome to manage.

It seems like quite a simple problem I'm trying to solve here. The code that is currently working on my machine, I want to pull out and put into a docker container.

Remember the key thing we want to achieve here, is for someone to be able to check out the code base, modify any of the packages, and then build a docker image, all from their development environment.

Is there an obvious lerna technique that I'm missing here, or otherwise a devops frame of reference I can use to think about solving this problem?

Answer 1

We got a similar issue and here is what we did: Put the Dockerfile in the root of the monorepo (where the lerna.json locates).

The reason: You really treat the whole repo as a single source of truth, and you want any modification to the whole repo to be reflected in the docker image, so it makes less sense to have separate Dockerfiles for individual packages.

Dockerfile

FROM node:12.13.0

SHELL ["/bin/bash", "-c"]

RUN mkdir -p /app
WORKDIR /app

# Install app dependencies
COPY package.json /app/package.json
COPY yarn.lock /app/yarn.lock
COPY packages/frontend/package.json /app/packages/frontend/package.json
COPY packages/backend/package.json /app/packages/backend/package.json
COPY lerna.json /app/lerna.json
RUN ["/bin/bash", "-c", "yarn install"]

# Bundle app source
COPY . /app
RUN ["/bin/bash", "-c", "yarn bootstrap"]
RUN ["/bin/bash", "-c", "yarn build"]

EXPOSE 3000

CMD [ "yarn", "start" ]

package.json

{
  "private": true,
  "workspaces": [
    "packages/*"
  ],
  "scripts": {
    "bootstrap": "lerna clean --yes && lerna bootstrap",
    "build": "lerna run build --stream",
    "start": "cross-env NODE_ENV=production node dist/backend/main",
  },
  "devDependencies": {
    "lerna": "^3.19.0",
    "cross-env": "^6.0.3"
  },
}

Answer 2

Late to the party but my approach is using webpack in conjunction with webpack-node-externals and generate-package-json-webpack-plugin, see npmjs.com/package/generate-package-json-webpack-plugin.

With node externals, we can bundle all the dependencies from our other workspaces (libs) into the app (this makes a private npm registry obsolete). With the generate package json plugin, a new package json is created containing all dependencies except our workspace dependencies. With this package json next to the bundle, we can do npm or yarn install in the dockerfile.