At Contactually we’ve recently taken on a project which reuses large parts of our core frontend architecture in a browser extension. This presents some challenges in terms of porting things like authentication, redux store structure, and large complex container components along with all of their descendants and their associated behavior. Ultimately, we chose to use Lerna and migrate our frontend code to a monorepo architecture.
We love to innovate, but reinventing the wheel isn’t that. After some research on cross-repository component sharing we came across the monorepo architecture which is used by groups like Digital Ocean, Google, React, Babel, Meteor and Ember. Facebook has an awesome write-up on some of the gains realized by a monorepo and Babel put together a collection of resources on other discussions around its merit.
Using Lerna, we resolve our package dependencies by enabling Yarn installation of the contained packages to each other. As an example, let’s say we have the following package structure in our Lerna monorepo.
│ └── package.json
Now, doing this 👇 inpackages/browser-extension/package.json will leverage Lerna to resolve frontend to our internal package, where left-pad will be resolved via Yarn.
We can now make concerted efforts to move shared dependencies (things like shared redux configs, general business logic utils, auth and/or api configs) into separate packages where it makes sense.
Shared dependency installations
It also allows us to leverage dependency hoisting to reduce any duplicated dependencies. An easy example would be Jest. If each package has Jest as a devDependency, we can install that at the root level instead of the package level, and allow Lerna to leverage cascading dependency resolution to make those shared dependencies available to each package.
How we migrated to Lerna
After much research, along with a healthy amount of trial and error, here’s how we deployed it successfully while also maintaining years of git history.
Create a new repository
In order to preserve the git history of our main application, we had to create a new repository. That’s due to the way lerna imports projects which does not work when importing a package into itself. This is as simple as lerna init in an empty repository. Then, using lerna import we were able to import a local copy of the repository while also keeping our existing git history.
Lerna uses git blood magic to rewrite the file path directories of all imported commits and apply the new package/my_imported_package structure to each file change. Pretty cool, right? Now at this point you should note that your hidden files and directories have also been imported. You can remove the git directories within your imported package and take inventory of everything else to ensure you have what you need, where you need it.
If you use a Release Candidate approach to deployments you should ensure you are on the most up to date RC branch locally of the project you are importing. This is now temporarily the master branch of your monorepo but we can fix that.
Do git branch rc which will create a new release candidate branch while remaining on master. This gives us an RC with the commits we haven’t yet verified as production ready. Now reset hard back to your last true develop commit (probably your previous RC) and do git branch develop which creates a a proper develop branch while remaining on master. Now do a hard reset back to your true master commit which puts master back at the proper state.
You can now use git push –all -u to push all 3 local branches up and voilà, you’re back in business.
Ensure deployments to staging still work
Deployments are technically easy when using the new monorepo structure. If you’re on Heroku, there is a Heroku buildpack you can use to easily configure our actual deployments. It provides a way for Heroku to look to a user-specified subdirectory which is provided via the CLI instead of serving from the root directory.
Organizationally, this is slightly more complex. and takes some department coordination. We tested strategy above over a weekend to reduce the impact to other engineers. Then we made a concerted effort to merge and test all outstanding work we could possibly wrap up before we officially switched over in order to prevent our engineers from losing their work. Where possible, this should happen between sprints.
Now you’re ready to create a new package
In our case, we had an existing repository that we wanted to import to also include our existing application as a dependency. This meant we could use the import script, followed by lerna add frontend –scope=browser-extension to add the existing frontend package as a dependency to the new browser extension package.
At this point, we ran into somewhat expected issues around our usage of PostCSS, svg’s, and some other things that rely on webpack loaders. Once these were resolved we were able to successfully import packages directly into our new project. Success!
The final boss has a name, and it’s Travis
We felt great up until this point, which is where we ran into a real head scratcher. Locally all specs were passing but Travis was failing consistently during the install process.
13.67s$ lerna bootstrap --scope=browser-extension
lerna info version 2.9.0
lerna info scope browser-extension
lerna info Bootstrapping 1 packages
lerna info lifecycle preinstall
lerna info Installing external dependencies
lerna ERR! execute callback with error
lerna ERR! Error: Command failed: yarn install --mutex network:42424 --non-interactive
lerna ERR! warning package.json: No license field
lerna ERR! warning No license field
lerna ERR! error An unexpected error occurred: "https://registry.yarnpkg.com/chai/-/chai-3.5.0.tgz: ENOENT: no such file or directory, open '/home/travis/.cache/yarn/v1/npm-chai-3.5.0-4d02637b067fe958bdbfdd3a40ec56fef7373247/lib/chai/utils/overwriteProperty.js'".
After researching the issue we found an open issue on Lerna which referenced it. Ultimately we were able to leverage Travis’s new beta feature, Stages, and use a combination of mutex and concurrency flags on our yarn install to prevent concurrency issues on the Travis build. Here’s an example of our configs.
New directory structure for our build scripts
│ ├── frontend/
│ │ └── package.json
│ └── browser-extension/
│ └── package.json
New Travis config leveraging build stages
— npm i -g yarn@”1.5.1" — cache-min 999999999
— npm i -g lerna — cache-min 999999999
— stage: frontend
- stage: extension
where ./scripts/frontend is
lerna bootstrap — scope=frontend --mutex network;
yarn test:ci --silent --maxWorkers=4 --coverage;
and ./scripts/extension is
lerna bootstrap — scope=browser-extension --concurrency=1 --mutex network;
yarn test:ci --silent --maxWorkers=4 --coverage
Now we’re able to realize all of the benefits we set out to gain by using Lerna.
If you’re interested in learning more about Contactually Engineering or would like to hear more about how we’re leveraging React/Redux and Monorepos to build one of the fastest growing CRM’s on the market, feel free to check out our blog or read about our open positions!
Using Lerna to share React/Redux functionality across web applications was originally published in Contactually Product and Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.