Every user of AWS CloudFormation, will agree that it is an impressive tool but can be very cumbersome to work with. Amazon has continually released improvements such as the introduction of
YAML templates and others to ease some of the pain of using CloudFormation but it still has it’s shortcomings.
These shortcomings gave birth to several tools all with the single goal of giving developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.
Before we talk about the various tooling around CloudFormation, let’s address the elephant in the room; Terraform.
In their own words, HashiCorp say;
Terraform enables you to safely and predictably create, change, and improve production infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
Terraform is indeed an impressive and easy to pick up tool that allows you to quickly provision infrastructure resources in the cloud. And I say cloud because Terraform supports other cloud providers such as OpenStack, GCP and Azure not just AWS.
It does however come with it’s own caveats. At the time of writing this blog post, Terraform has 1,814 Issues and 285 Pull Requests on GitHub. This isn’t to say it’s a bad tool as we not only use Terraform heavily here at OSO but most of HashiCorp suite of open source tools including Vagrant, Packer, Consul and Vault.
If however, your project is affected by one or more of the 1,814 issues or waiting on a pull request that resolves your issue to be accepted and merged in or a requirement not currently integrated by Terraform, then you’re left in a bit of a pinch.
Aside from being an open source tool, HashiCorp has a Pro and Enterprise offering of Terraform which means extra functionalities required to make or ease the use of Terraform in a large Ops team such as Rollback workflow, Environment locking and more are not available. This is evident as more and more people find workarounds for their requirements or issues.
Such workaround is Terragrunt which is a wrapper around Terraform that attempts to add some of the Pro and Enterprise features such as Environment locking missing in the open source version.
Below are a round up of the current tooling around AWS CloudFormation, that “promise” to make the lives of your Ops teams easier.
StackMaster touts itself as “The missing CloudFormation tool”. It’s written in Ruby and supports diffs, change sets, and GPG parameter encryption which is definitely a welcome addition to ensure your CloudFormation templates do not contain any company secrets.
The main drawback for me is Ruby and past experience with Ruby based tools like CFNDSL have not been great.
Qaz is the newest tool and that is evident in it’s us of the Go programming language. It emphasizes minimal abstraction from the underlying AWS CloudFormation Platform. It instead enhances customisability and re-usability of templates through dynamic template generation and logic.
The minimal abstraction however means that you still have to write CloudFormation templates even though
YAML support makes this a little easier but is still cumbersome in addition to working with Qaz specific files.
Sceptre is written in Python and also goes for the minimalist approach which means you’ll have write your own CloudFormation templates
YAML and simply provides a wrapper around creating, updating and deleting stacks.
It does however allow for isolation of environments and support for running arbitrary code as hooks before/after stack builds.
Stacker from the beginning felt really familiar, this was due to a few things;
- Written in Python
- Backed by Troposphere
Troposphere is Python library to create AWS CloudFormation descriptions. It also supports some OpenStack resources. Stacker is backed by Troposphere and employs a very simple CLI and it’s main selling point is the concept of “Blueprints”.
Stacker works on the premise that Stacks should be modular and independent making them reusable and dependency free. Blueprints are very similar to Terraform Modules but differ in the sense that Terraform Modules are reusable infrastructure resources such as a VPC or an RDS but a Blueprint in Stacker can be a complete modular infrastructure. A good example will be a Bastion Blueprint. This blueprint will not only create an EC2 Instance (the bastion host) but also all the required resources needed to support the Bastion host, including Security Groups, Launch Configuration and AutoScaling groups.
With the Bastion blueprint done, it essentially means your Ops team will never need to write a CloudFormation for the provisioning of Bastion hosts saving them time, resources and reducing code duplication.
In a follow up post, we’ll dive deeper into Stacker and explain how we manage the infrastructure of our clients on AWS. If you need help with Infrastructure as Code and other DevOps practices, or AWS at your company, feel free to reach out to us at OSO.
At OSO, our experts can maintain your DevOps platform and be responsible for day-to-day operational issues, allowing you to develop and ship your product without the need for internal DevOps hires.