In my last post I talked about how to manage cloud resources with traditional configuration management tools like Chef, Puppet, Ansible, and SaltStack. Today we're going to talk about another technique to manage cloud resources: using a cloud provider's resource management solution. Amazon Web Services has a resource management solution called CloudFormation. What is CloudFormation, and why would you use it over a configuration management tool that you already know?
CloudFormation is an API provided by AWS. You provide a template describing your infrastructure as input, then AWS creates the infrastructure (servers, load balancers, etc.) described in the template. The template you provide as input can be JSON or YAML and it is a declarative format, by which I mean you describe the resources you want created, not how to create them.
So far, it sounds similar to what we already know with configuration management:
- Declarative. ✅
- Uses cloud provider API's. ✅
What distinguishes a cloud resource manager like CloudFormation is that it takes care of the dependency management. Your template describes what resources you want created (servers, etc.), and AWS takes care of creating everything in the correct order. Another benefit is that the resources can be managed as a logical unit called a "stack". You can create and destroy stacks as often as you like. The CloudFormation API is free to use, and you pay usage costs for the resources it creates.
CloudFormation also manages state. If I modify a template and add a new resource, like adding a new EC2 instance to the stack, then CloudFormation knows to leave the existing resources alone, and just spin up the new EC2 instance.
The way I think of CloudFormation is that it's a Makefile for infrastructure.
Let's use an example to make this clearer. We'll use the standard, 3-tier web app example from before.
Here is a snippet of CloudFormation template that would create one web server for the stack.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "AWS CloudFormation Sample Template EC2InstanceWithSecurityGroupSample: Create an Amazon EC2 instance running the Amazon Linux AMI. This example creates an EC2 security group for the instance to give you SSH access. **WARNING** This template creates an Amazon EC2 instance. You will be billed for the AWS resources used if you create a stack from this template.",
"Parameters": {
"KeyName": {
"Description": "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"Type": "AWS::EC2::KeyPair::KeyName",
"ConstraintDescription": "must be the name of an existing EC2 KeyPair."
},
"InstanceType": {
"Description": "WebServer EC2 instance type",
"Type": "String",
"Default": "t2.small",
"AllowedValues": [
"t1.micro",
"t2.nano",
"t2.micro",
"t2.small",
"t2.medium",
"t2.large",
"m1.small",
"m1.medium",
"m1.large",
"m1.xlarge",
"m2.xlarge",
"m2.2xlarge",
"m2.4xlarge",
"m3.medium",
"m3.large",
"m3.xlarge",
"m3.2xlarge",
"m4.large",
"m4.xlarge",
"m4.2xlarge",
"m4.4xlarge",
"m4.10xlarge",
"c1.medium",
"c1.xlarge",
"c3.large",
"c3.xlarge",
"c3.2xlarge",
"c3.4xlarge",
"c3.8xlarge",
"c4.large",
"c4.xlarge",
"c4.2xlarge",
"c4.4xlarge",
"c4.8xlarge",
"g2.2xlarge",
"g2.8xlarge",
"r3.large",
"r3.xlarge",
"r3.2xlarge",
"r3.4xlarge",
"r3.8xlarge",
"i2.xlarge",
"i2.2xlarge",
"i2.4xlarge",
"i2.8xlarge",
"d2.xlarge",
"d2.2xlarge",
"d2.4xlarge",
"d2.8xlarge",
"hi1.4xlarge",
"hs1.8xlarge",
"cr1.8xlarge",
"cc2.8xlarge",
"cg1.4xlarge"
],
"ConstraintDescription": "must be a valid EC2 instance type."
},
"SSHLocation": {
"Description": "The IP address range that can be used to SSH to the EC2 instances",
"Type": "String",
"MinLength": "9",
"MaxLength": "18",
"Default": "0.0.0.0/0",
"AllowedPattern": "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})",
"ConstraintDescription": "must be a valid IP CIDR range of the form x.x.x.x/x."
}
},
"Resources": {
"EC2Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"InstanceType": "t2.medium",
"SecurityGroups": [
{
"Ref": "InstanceSecurityGroup"
}
],
"KeyName": {
"Ref": "KeyName"
},
"ImageId": "ami-a9210ebf"
}
},
"InstanceSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Enable SSH access via port 22",
"SecurityGroupIngress": [
{
"IpProtocol": "tcp",
"FromPort": "22",
"ToPort": "22",
"CidrIp": {
"Ref": "SSHLocation"
}
}
]
}
}
},
"Outputs": {
"InstanceId": {
"Description": "InstanceId of the newly created EC2 instance",
"Value": {
"Ref": "EC2Instance"
}
},
"AZ": {
"Description": "Availability Zone of the newly created EC2 instance",
"Value": {
"Fn::GetAtt": [
"EC2Instance",
"AvailabilityZone"
]
}
},
"PublicDNS": {
"Description": "Public DNSName of the newly created EC2 instance",
"Value": {
"Fn::GetAtt": [
"EC2Instance",
"PublicDnsName"
]
}
},
"PublicIP": {
"Description": "Public IP address of the newly created EC2 instance",
"Value": {
"Fn::GetAtt": [
"EC2Instance",
"PublicIp"
]
}
}
}
}
A CloudFormation template is made up of three sections: Parameters, Resources, and Outputs. Parameters are the inputs to the template. Resources are the concrete cloud resources to be created and managed (e.g. EC2 instances, ELB's, RDS instances). Outputs are defined so that you know concrete details of the newly created resources like EC2 instance IP addresses and DNS names, so you can connect to the newly created resources.
If I were to add a new EC2 instance to the template above, then CloudFormation would leave the existing server alone and create a new EC2 instance, which is exactly what we want in this example (we'd probably use an AutoScaling Group in real life).
One issue I raised with managing cloud resources with traditional configuration management tools is that it's easy to leak resources. Since CloudFormation manages the stack as an atomic unit, it's much harder to leak resources.
Issues With CloudFormation
Nothing's perfect, and CloudFormation is no different.
What's great about CloudFormation is that it manages dependencies between resources for you. The trade-off is that it's a bit of a black box. You submit your template to the API, and see if it works. If it doesn't work, then make some changes and submit again. Infrastructure is a complex beast full of state, so the feedback cycle for this isn't quick like testing pure functions in an application. The feedback cycles can be 5-20 minutes to see if your changes worked. The other thing that makes this feedback loop challenging is that the error messages returned by CloudFormation can be inscrutable. However, with patience and practice you learn what they mean, and how to adjust.
Occasionally, CloudFormation doesn't take things down cleanly and requires some manual cleanup. But, unlike traditional configuration management tools, CloudFormation will tell you which resources are left behind.
In the template above, the InstanceType parameter has an AllowedValues attribute that serves as a validator. When you supply a value for that parameter, CloudFormation will check to see if it's one of the allowed values. This validation prevents many common mistakes, like typos. This initial list was filled in for me by a CloudFormation template provided by Amazon. This list has to be updated manually over time, as Amazon changes its list of supported instance types. Since Amazon already knows what the list of valid EC2 instance types are, it would be nice if there were a built-in validator for this sort of thing.
Summary
I prefer to manage my cloud resources using a resource manager, like CloudFormation, over traditional configuration management tools. In exchange for learning this new way of doing things, I get:
- A declarative template describing my infrastructure that I can manage in source control.
- The template is parameterized so I can easily use it to create the infrastructure for multiple environments like Dev, QA, and Prod.
- I don't have to worry about creating the resources in the correct order, nor in what order they must be destroyed.
- I can manage the infrastructure described in the template as an atomic unit. So I can spin up and tear down new Dev and Test environments easily.
Resources
- Amazon's CloudFormation Resource Types Reference - This is the reference to keep open in a browser tab all day. Pay special attention to how resources are updated when properties change.
- Amazon CloudFormation Sample Templates - These examples will get you started, so you don't have to start from a blank page.
- Amazon Web Services in Action - This book helped me understand CloudFormation in a clear, cohesive way. I learned a lot from it, and recommend it if you're interested in learning more.
If you're not on AWS, then here's a list of equivalent tools for the other big cloud platforms: