Disclaimer: I am currently building a product in this space at Preview. As such, I have biased opinions regarding the “correct” way of addressing Continuous Product Review needs. Preview is explicitly not included in this posting.

This article is meant to be a summary of existing tools in the space, and not a comparison between tools.

Over the last year, there as been an explosion of new tools and products in the Continuous Product Review space. One of the earliest companies which addressed this space was Runnable, raising it’s first round of funding back in 2012, before pivoting into this space around 2015 and being acquired by Mulesoft in August 2017.

I personally feel that Runnable was a bit ahead of it’s time. Since then, Y Combinator has accepted and invested in at least 5 different companies in this exact space over their past 5 batches (Dockup (W19), FeaturePeek (S19), ReleaseApp (W20), Reploy (S20), and LayerCI (S20)). This alone speaks to the attention the space is getting, and the desire for a scalable and maintainable product.

So, what is Continuous Product Review?

Continuous Product Review is the practice of using dynamic, ephemeral application environments in order to enable individuals and stakeholders outside of the Engineering organization to provide feedback during the development process.

This means that a PM, or a member of the sales or support teams, should not have to wait until a new feature is into a staging or production environment in order to view, explore, and give feedback about new features or functionality. An engineer should not have to run a given revision locally in order to show the current progress to a stakeholder.

Given this fairly broad definition, there is a fair amount of variety in terms of the workflow and scope that different offerings try to address, and how they integrate Preview environments into your team’s existing workflow.

Add-on Services

Public, regular use of Continuous Product Review tools first began years ago as additional features within application hosting systems. These add-ons are extremely easy to use, and tend to function very well, assuming that you are already using the given hosting system. The tight integration which exists in these systems often allows them to provide features which are impractical with a more generic system.

Heroku Review Apps

Heroku was one of the first companies to introduce the concept of Review environments, with their first iteration becoming generally available in 2016.

If you are already using Heroku for your application, this is the obvious choice, however is not a reasonable option for applications which are hosted elsewhere.

Netlify Deploy Previews

Netlify Deploy Previews were first announced around the same time as Heroku’s offering going GA, in mid-2016. They have a tight integration with your chosen source control systems, as well as the rest of Netlify.

Netlify deploy previews are limited to the feature set that Netlify supports, which is currently only statis web sites, and hosted functions. If you have corresponding back-end changes as part of your Pull Requets, Netlify can not handle Deploy Previews for those systems.

Vercel Preview URLs

Vercel, in a similar vein to Netlify, offers Preview URLs for every Git branch and PR for front-end sites and applications which are hosted on Vercel. Due to the tight coupling of Preview URLs to the rest of the Vercel platform, they are not an option unless you already using Vercel, and the obvious choice if you are.

Render Pull Request Previews

Render Pull Request Previews are very similar to Heroku Review Apps; they work extremely well for applications which are already configured to run on the Render platform.

Render Pull Request Previews, when enabled, will automatically being when a PR is created, and be deleted when the PR is merged or closed. I did not see a way to configure this behaviour, or provide maximum Pull Request Preview life-times. Previews are also billed at the same rate as the base (production) service (despite it being unlikely that the same resource levels are required). Based on these two things, I would be concerned that Pull Request Previews on Render could get very expensive.

Gitlab Review Apps

Gitlab Review Apps are the most customizable Add-on service I’ve discovered, but that goes hand-in-hand with the flexibility that Gitlab attempts to offer. As part of their “Auto DevOps”, Review Apps deploy into an environment, which are all managed within Gitlab as well. The tight integration with the rest of the Gitlab configuration makes this is a great choice if you are already using, and are familiar with managing, Gitlab.

The one thing I will say about Gitlab’s offering here is that it has a ton of different features and functionality that can be enabled/disabled, which can present a rather steep learning curve.

Continuous Product Review Productized Offerings

The majority of innovation in this space over the last year has been in the form of independent hosted products. These projects and products, for the most part, focus on being platform-independent. Some of them do extend outwards to include development environments, or production environment management. If this is the case, there will be a note.

Preview-only Offerings

• TugBoat
• Reploy
• LayerCI

As a note, this is the category that Preview is designed to fit within.

These are the three solutions I’ve found which attempt to solve for Preview environments in an isolated way; ie, with a workflow which does not presume to impact the local development environment, or production environments.

• FeaturePeek

I am mentioning FeaturePeek separately because it is different than the other tools in this categoy: it focuses entirely on front-end (Single Page Applications) development. They do not have any support for back-end functionality, which the other tools here do support. They have documentation of how to connect your ephemeral front-end environment to externally-hosted back-ends, however managing those dependency systems is entirely on the developer.

In my opinion, their true value, and what separates them from other Product Review offerings, lies not in the creation/management of ephemeral environments, but in the suite of review tools which are injected into their environments.

Development Environments

• Garden
• Loft.sh
• Blimp (formerly Kelda)
• Krucible

These tools all offer Preview environment functionality, but as is more of an add-on to a primary focus of Remote Development and/or a developer Kubernetes environment. As such, utilizing the Preview functionality would likely require a larger change in how your development team operates.

Production Environment Management/Platform-as-a-Service (PaaS)

• ReleaseApp
• DockUp

Similarly, there is a set of offerings which attempt to utilize Preview environments as a part of a larger environment management system. As part of this, they are able to provide more advanced rollout management (such as allowing environments to be promoted into production). The additional operational benefits with these tools must be balanced with the functional limitations that could be instilled to your production environments.

Related Tooling

There is a fair amount of related tooling in the space, that I would not go as far as to call a product offering. These include open source projects which you can run yourself, or underlying tools that address required functionality if you were to build an in-house Preview apps implementation.

• PullPreview: This is an source-available project, but requires a paid license. It runs entirely within Github Actions.
• Jenkins X: Jenkins X supports Preview environments, as well as environment promotion. It is functional, however like Jenkins, it often requires a little TLC.
• StageHand: StageHand is an open tool for easily creating and viewing review applications.

Tools which may help you build your own:

• Orca
• kiosk
• Rancher
• EKSEphemeral

Honourable Mentions

Rezden

Rezden was a Kubernetes based Platform-as-a-Service, which appears to have shut down in the last year. If anyone has more information about this product, I’d love to add the missing information.

Ephemeral.run

I have no information on this project at the moment, because it was just recently accounced. It will be placed into a category above, and this post updated, when more information is available.