The superpower of automatic deployments

Imagine a world where, as a developer, you’re liberated from the repeated tasks. Where your energy is channeled solely into translating a feature request in code, and the rest magically happens. No manual tests, no deployment protocols, no wait time.

This isn’t a futuristic vision—it’s how we work today. Welcome to the world of 'fire & forget' deployments. In this blog post, we'll unveil the superpowers that automatic deployments have brought to XFA and how they can transform your development process, too. Dive in and discover how we made instant updates our secret weapon.

The Philosophy

Software creation fundamentally revolves around three core steps: writing code, testing changes, and deploying to wherever it should run. However, each of these steps can introduce its own set of complexities, potentially increasing costs and slowing down the development process.

Every change we make to our systems at XFA requires rigorous checks and a mandatory code review. This is a common industry practice and considered essential when developing a sensitive application. Companies often choose to batch changes that should be tested and (maybe) delivered in the hopes of optimizing the time spent writing code, instead of losing time on redundant testing and deploying small iterations.

We do the exact opposite. The moment a single change is requested—even minor ones like publishing this blog post—, the full set of tests are executed automatically against a live environment, a code review is requested, and moments later the changes go live. There's no release staging or scheduling. Every change goes live minutes after it is merged, and this is true for our cloud services, desktop clients, and even our mobile apps. Changes will typically reach production in as little as 10 minutes since the code was written, effectively propagating and being tested automatically across 3 complete environments before reaching production.

It feels truly amazing to just 'fire & forget'.

Are We Optimizing Too Early?

An engineer’s biggest tendency is optimizing a part or process that should not even exist (yet) in the first place. XFA has been in development for two years now, and we deliberately enforced the practice of fully automated testing and deploying every change from the very beginning. Now, 5 apps, 15 services, and 26 AWS accounts later –a story for another blog post–, it's one we don't regret.

XFA is a security product that sits in the critical path of authentication, making its availability and proper operation one of our top priorities. Enforcing an automatic deployment process created a very early, shared responsibility for both keeping existing functionalities working and strictly managing the change process, documenting difficult details along the way, because it just can’t only ‘work on my machine’.

Especially in an early startup, the ability to constantly make minor adjustments to our product and safely get them to our users with zero to no effort really speeds up the amounts of feedback we can take in and process. We are often switching between several tasks and priorities, and lowering the (imagined) mental hurdle to get started on a change really helps to keep momentum. This is particularly the case with bug fixes and incidents, where the turnaround time is instant, and the impact is minimized.

Too Much Work?

Creating, and especially starting out immediately with, these setups is always a case of “building on the shoulders of giants”. Both in the tools that we use (e.g. AWS SAM, Serverless technologies, and GitHub Actions) and borrowing from the experience we’ve gathered from working at other places. This fresh start is a chance to skip ahead, and it would be a shame not to use the opportunity when building products that should last.

When moving away from projects that do not purely rely on the cloud and serverless infrastructure and instead require interaction from the devices they will run on, like desktop applications and native mobile apps, the process gets trickier. Requiring a physical self-hosted build machine to build and test every change at an acceptable speed (and at an acceptable cost), as well as many days of engineering to make the process consistent and automated.

While it's not definitively clear whether the time invested in engineering these complex projects has been offset by the time saved over the past two years, we are certain of a few things: more code has been written, more feedback has been addressed, and no time has been wasted troubleshooting why a developer's machine couldn't produce a working binary. Plus, the build process is automatically documented, enabling any engineer to jump in seamlessly or repeat what has been done before.

The Process

We separate each application or project—whether it's a microservice, desktop app, or library—into its own code repository. Each one then deploys to entirely distinct resources, like a dedicated AWS account. This highly modularized approach separates concerns as much as possible to keep things small, individually deployable, and adds extra layers of security by minimizing the permissions required and the defaults that can be abused.

The build and deployment process for almost every module of XFA somewhat looks like this:

• Changes are proposed through a pull request.

• A list of checks is run, including a full deployment of the module with the changes on a separate environment on which integration and end-to-end tests are run. The resources for these extra environments are automatically created with a pull request, and deleted upon merging.

• Once the pull request passes all tests and gets approved in a code review, it will merge into the main branch. It then deploys to both test and production environments, using the same process.

On top of that, merging a pull request into the main branch also triggers maintenance routines. These tasks generate pull requests for any outdated packages and update existing pull requests with the most recently approved modifications.

Conclusion

Hopefully, this insight has been helpful. This setup allows our small team to efficiently and reliably manage numerous services and apps. Plus, it's simple for new members to jump in and hit the ground running. We hope these insights benefit others as well, and we look forward to sharing more about how XFA is built in the future.