CircleCI Tooling Wishlist

CircleCI is a great CI/CI solution for companies who do not want to build an internal expertise on running these systems. For small-to-medium startups (up to ~50 engineers), these is almost always a small tradeoff. Despite some issues I have brought up in the past, I still believe that CircleCI can be effectively used as long as it is setup and administered appropriately for the security concerns of the company.

There are still a couple feature improvements that I believe CircleCI would benefit from introducing, and many of these should (in theory, I have 0 inside knowledge of how CircleCI works) be fairly easy to implement in a backwards compatible fashion.

Many of these features are geared towards a more “monorepo” application development methodology, as opposed to the multi-repo micro-service workflow that CircleCI currently supports more natively.

1. Trigger Jobs based on File Changes

Currently I can chose to run jobs based on different filters, based on tags and branches. For instance, I can run a deploy job only on a tag of a specific release format.

What I cannot currently do is run a specific job only when files under a specific directory have been changed. For example, if I have 5 different applications in my monorepo, I really only want to run tests for one of the applications when files under a given directory change. This because much more useful as the number of applications and jobs increase, and you have limited containers. It is inconvenient when a small change kicks of a slew of unrelated jobs.

Note that it is possible to get similar results currently, however the UX is not as great. You can currently add a check to the beginning of each job to do a git diff and check the file paths, and then exit early. This approach has at least two downsides:

• It gives the appearance that there was a successful test run, even though the tests didn’t actually run
• It takes the time for a container to be provisioned and execute set up, which adds up as the job count grows

For example:

workflows:
  version: 2
  test-build-and-deploy:
    jobs:
    - test:
        filters:
          filePaths:
            only: "/src/service-a/"  # All changes under `/src/service-a`

2. Cache-able Test Results

Furthering on #1, it would be amazing if tests could be set as “cachable”, so that the result could be cached with a dynamic key, and returned upon subsequent pushes.

For example, lets assume I create a PR that modifies Service A, which is under /src/service-a in my repository, and Service B, which is under /src/service-b in my repository. On the first CircleCI run for this PR, I would want all tests to run. Now lets assume I push a new commit that fixes a typo in Service B, which fixes linting and test errors. It would be amazing if I could specify that the linting/test test results for Service A are cacheable, and have the job return the same results without actually rerunning the Service A jobs.

It is important to note that CircleCI already uses a similar caching mechanism to cache steps in a job. This is typically used for caching long installs, by using the restore_cache and save_cache directives.

My recommendation would be to add a result_cache directive at the beginning of the job, which short-circuits the rest of the job in the event that a match is found. For example:

- result_cache:
    key: result-cache-{{ checksum "/src/service-d" }}

3. Result-Dependent Steps

There are some times that I want to only run a job step based on the output of a previous step. For example, I would love to automatically deploy Terraform change on merge, but am not willing do that without manual review of the plan. CircleCI has a manual review step, however in the event that there are no changes in the plan, I do not want the manual review to block before applying the (empty) plan.

My recommendation would be to add the ability to include conditional logic when using the approval type. This would allow your workflow to bypass (or enforce) the approval intelligently.