Migrating 2,000+ Specs from Karma to Jest

Migrating 2,000+ Specs from Karma to Jest

This post was originally published on Medium

Migrating a massive Karma test suite to Jest was no easy task, but with careful planning, dedicated teamwork, and a little help from computers (who would’ve thought?), we were able to make the switch in a surprisingly short amount of time.

New Year, New Suite

Our journey into React has been one which most React developers can relate to. Coming from a Monolithic Rails app, we didn’t initially feel that simple unit specs on components alone would be sufficient for catching UI regressions. Instead, we tried to recreate an environment similar to a Rails e2e testing suite with Mocha, Enzyme, and Chai.

Out with the old…

The explanation of the inner workings of the system which we attempted to put in place could easily fill another blog post, but to summarize:

  • We were testing entire page-level components, as well as sections of that page-level component, as well each component used in those sections. This required a ton of mocking of our Redux store, which made changes (to either the store or the test) very cumbersome.
  • All of our tests were bundled into a test bundle with the karma-webpack plugin. This took a while to process, and limited us from using features like “single test runs” (we had to run the entire suite).
  • We created a mockStore utility, which allowed us to populate our Redux store with any Records which we would normally get from our API in-app.

After a while, the mockStore utility became much too time consuming to maintain. We also found ourselves faced with cross-test contamination, and increasing Webpack bundle times. Over time, many of the workarounds we had implemented to keep our system working with Karma were coming back to bite us.

Eventually, we reached a breaking point, at which the time taken to resolve issues in the test environment began to consume the majority of the time taken to write a spec. We decided it was time for a change in our testing philosophy.

…And in with the new

At the start of 2018, we embarked on an ambitious path to employ a new testing philosophy. Instead of trying to roll our own sort-of-e2e-but-not-really with Karma, we would separate our concerns across two different test environments — each specifically focused on one aspect of testing:

  • Jest: Would be used for unit testing of components, as well as unit testing of sagas, selectors, utility functions, et cetera. We would not mock our API client in this environment.
  • Cypress: Would be used for true end-to-end testing. This environment would feature a fully-functional mock of our API, dynamically generated from our API codebase. (If you haven’t heard of cypress yet, you should check it out. It’s everything you wish Selenium/Capybara/Webdriver was, and more).

We’ll focus on Jest in this article, but certainly stay tuned to the Contactually Engineering Blog for a future article on Cypress.

Enter Jest

We were done working with our good-intentioned, yet fragile, Karma test suite and ready to embrace all that Jest and the Jest community had to offer.

Automating the Process

Facebook offers a codemod to assist with the switch from other popular testing frameworks to Jest. A codemod is always a great nice-to-have, but unfortunately they don’t provide a one-size-fits-all migration solution. Since the author of a codemod can’t possibly know all of the idiosyncrasies of every codebase other developers will run it against, there are always missing details and/or errors.

In our case, we found it much more efficient to supplement what the codemod was missing with a few good ole’ regex replaces.

At first, we took a shot at converting the whole suite in one fell swoop, with a combination of the jest-codemods script and ~50+ global replaces. Unfortunately, after running these automated migrations, we were still left with a significant number of non-trivial issues to resolve. This migration would not be trivial, and would need to be done incrementally, with careful review at each code merge.

Setting the Ground Work

Since this migration would happen gradually over time, we needed to plan our process to ensure we made a smooth transition:

  • We would run the Karma (legacy) and Jest (new) test suites side-by-side. This would allow us to convert our suite over time.
  • In order to separate files between the two test suites, we would leave Karma test filenames with their original *.spec.js suffixes. When we converted the test to Jest, we would change the suffix to *.test.js.
  • We would add another container to our travis builds to run the (much faster) Jest suite.
  • We would automate as much of the process as possible with a code migration script.
  • We would document the migration process in depth and divide the migration work amongst the developers on the team.

The Automated Script

We created a tool which does the following, given a directory:

Can you tell this console script was written by a FE dev (because it looks so nice)?
  1. Renames our karma *.spec.js files within the given directory to *.test.js.
  2. Runs 4 of the jest-codemod scripts (mocha, chai-assert, chai-should, assert).
  3. Runs a massive list of global replaces against the directory.
  4. Made incremental commits at each step.

This script allowed us to chip away at the migration, directory by directory. After the script ran, running jest would reveal how many tests need to be manually fixed by a developer. By our second or third chunk, our script was setting us up with about 70–90% of tests passing per chunk.

Since the script itself is much too long to put in the body of this article, here is a link to the gist. Again, there isn’t a one-size-fits-all solution, but if you’re making the same migration we made, this script will go pretty far. If you’re migrating from a different library, our migration script can at the very least offer some inspiration.

A Slack Integration

Automated posting in our #frontend slack room after the Jest groundwork.

After finishing the initial groundwork and dividing up the work, we threw together a quick script to monitor and report on our progress each time we merged a Jest migration PR.

Delegation & Teamwork

Our Karma and Jest test suites could run alongside one another, and our automated script allowed us to blast through a large portion of the work bit-by-bit. It was time to divide and conquer. We spent a few hours counting specs and identifying possible “problem” directories and splitting the work into (roughly) pointed tickets.

We then drafted up a list of migration instructions in a GitHub issue on our project repo. Over the course of this migration, this GitHub issue thread was used to share findings and decisions between developers on the team as we continued to tackle unique sections of code.

I took a first pass at two of the biggest “problem” directories, which allowed us to break through the majority of blockers which affected all tests. These blockers included issues such as:

  • Creating manual mocks for a few external packages that either didn’t play nice with the nodejs/jsdom environment, or just weren’t necessary in our test environment.
  • Stubbing globally mocked functions, like window.open.
  • Tightening up the migration script and GitHub issue instructions.

After this, members of the team chipped away at the migration little by little. Thanks to the incredible focus and hard work of a few team members, the team completed the rest of the migration in under 3 weeks (which was way shorter than we had estimated).

While we chipped away at smaller chunks of the codebase, we also took time to carefully review each code merge. In doing this, we were able to fortify and improve a bunch of our old specs — on top of converting them to jest. In other words, we took this migration as an opportunity to audit and improve all of our specs.

The End

At the time of writing, we can proudly say that Contactually has been 100% migrated to Jest. This write up is our way of giving back to the awesome and supportive JS community, and we hope it is helpful to those who are faced with the same problem we were.

We’re always innovating at Contactually, stay tuned to our Engineering Blog to be in the loop!

Migrating 2,000+ Specs from Karma to Jest was originally published in Contactually Product and Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

The following two tabs change content below.

Austin Pivarnik

Latest posts by Austin Pivarnik (see all)

Submit a Comment

Your email address will not be published. Required fields are marked *