Chrysolites Tech Academy

Infrastructure as Code: Pulumi, Terraform and the AWS CDK · Course book

Chapter 1: Describing Infrastructure in Code

Pairs with Lesson 1: Infrastructure as Code: Concepts and Labs with Pulumi

Every application runs on infrastructure: servers or containers to run the code, networks to connect them, databases and storage for data, and security rules deciding who may reach what. For a long time this was set up by hand, by clicking through a cloud provider's web console or typing commands on servers. Infrastructure as code, IaC, replaces that with files that describe the infrastructure precisely, which a tool then creates, updates or removes to match. This chapter explains why that matters, the three styles of writing IaC, how an IaC tool decides what to change, and works through three typical projects: cloud storage, a fleet of virtual machines and containers behind a load balancer.

Three ways to manage infrastructure

Infrastructure covers five broad areas: compute such as virtual machines and containers, networking such as private networks and load balancers, security such as firewall rules and permissions, data stores such as databases and file storage, and the deployment of the applications themselves. There are three ways to manage it.

Manually. Someone logs in to the console and creates each piece by hand. It is fine for a quick experiment but fails at scale. Nobody can be sure what was configured, recreating an environment means repeating every click perfectly, and two environments that should be identical slowly drift apart.

Ad-hoc scripts. Someone writes shell scripts that call the provider's command-line tools. This is repeatable, but scripts only know how to run their steps in order. Running a creation script twice may create everything twice; changing one setting may require writing a whole new script; and the script cannot tell you what currently exists.

Infrastructure as code. You describe the end result you want, and a tool works out the steps. It records what it has created, compares that with your description and makes only the changes needed. The files live in version control, so every change is reviewed, recorded and reversible, exactly like application code.

Codewhat should exist Statewhat was created IaC toolcompares the two Preview+1 create, ~1 update,0 delete Cloudonly the changes applied After applying, the state is updated to match the cloud.
Figure 1.1 — How an IaC tool works. It compares the code with its record of what exists, shows the planned changes, and applies only those.

The state deserves particular care. It is the tool's memory of which real resources belong to which parts of the code, and without it the tool cannot tell whether a resource needs creating or already exists. Teams therefore keep state in shared, backed-up storage rather than on one person's laptop, and avoid changing managed resources by hand in the console. A manual change creates drift, a difference between the code and reality, which the next run will either undo or trip over. Most tools can detect drift with a refresh, which reads the real resources and reports what differs.

This makes IaC idempotent: running it again when nothing has changed changes nothing. If the code says three virtual machines and three exist, the tool does nothing. Change the code to four and it creates exactly one more.

Three styles of writing IaC

IaC tools differ mainly in the language you write in.

StyleExample toolsStrengthsWeaknesses
Data templates (JSON or YAML)AWS CloudFormationSimple, declarative, supported directly by the providerVerbose; little support for loops, reuse or logic
Domain-specific languageTerraform's HCLDesigned for infrastructure; readable; works across many providersA new language to learn, with limited general programming features
General-purpose languagesPulumi, AWS CDKLoops, functions, classes, testing and editors you already knowMore freedom also means more ways to write confusing code

All three are declarative at heart: even when you write Python or TypeScript, the program's job is to build a description of the desired infrastructure, which the tool then reconciles with reality. The difference is how pleasant and powerful the description language is. The labs in this chapter use Pulumi with Python on AWS, though the same ideas apply to every tool and provider.

Pulumi organises work into projects, a folder of code, and stacks, separate instances of that project such as dev and prod. Each stack has its own configuration and its own state, so the same code can create a small development environment and a larger production one, differing only in settings such as machine size or the number of copies.

Lab 1: storage and a website

The first project creates an S3 bucket, AWS's storage for files. In Pulumi with Python this is a couple of lines: import the AWS package, create an s3.Bucket object and export its name. Running pulumi up shows a preview, one resource to create, and on confirmation creates it.

Turning the bucket into a website is a matter of editing the code: enable website hosting, upload an index.html file as a bucket object, and allow public reading. Running pulumi up again shows that the bucket will be updated, not recreated, and that one new object will be added. The preview is the crucial safety feature: you see exactly what will change before anything does.

Creating a second stack, prod, with pulumi stack init prod and running the same code produces a completely separate bucket and website, with no change to the code. Finally, pulumi destroy removes everything the stack created, which is the reliable clean-up that manual work never offers.

The preview is the review. Treat a plan or preview like a code review: read every planned create, update and especially delete before confirming. A line saying a database will be replaced, rather than updated, deserves a second look.

Lab 2: a fleet of virtual machines

The second project creates EC2 virtual machines. One machine needs a few things: a machine image to start from, an instance size, and a security group allowing web traffic in on port 80. Because the code is Python, creating many machines is as simple as a loop.

A cloud region contains several availability zones, physically separate data centres, so that a fire or power cut in one does not take down the others. The code can ask AWS for the region's zones and create one machine in each. With 3 zones, the loop creates 3 machines, and the application survives the loss of any single zone.

Each machine can be given a short start-up script, run on its first boot, that installs a web server and writes a page reporting which machine and zone it is. That makes it easy to see the fleet working once traffic is spread across it.

Users should not have to know which machine to visit, so an Application Load Balancer is placed in front. It receives every request on one public address and passes each to a healthy machine in turn. If the code is changed to create 2 machines per zone, the next pulumi up adds 3 more and registers them with the load balancer, a change that would take dozens of careful clicks by hand.

Lab 3: containers without servers to manage

The third project runs Docker containers. Amazon ECS, the Elastic Container Service, runs containers in a cluster, and its Fargate mode means you never see the servers at all: you state the container image, how much processor and memory each copy needs and how many copies to run, and AWS finds the capacity. The code creates a cluster, a load balancer and a service running an nginx container.

Scaling is a one-number change. Setting the desired count from 1 to 5 and running pulumi up starts 4 more containers behind the same load balancer. Tearing everything down afterwards with pulumi destroy matters in the cloud, because resources are billed for as long as they exist. A forgotten load balancer or fleet of machines left running over a month can cost far more than the experiment was worth, and IaC makes it trivial to remove everything a stack created in one step.

The value of IaC grows with every repetition. Suppose building an environment by hand takes 2 hours, while writing the code takes 4 hours once and each later run takes about 6 minutes, 0.1 hours. For n environments, manual work costs 2n hours and IaC costs 4 + 0.1n. The two are equal when 2n = 4 + 0.1n, at n of about 2.1, so from the third environment onwards the code has already paid for itself, before counting the fewer mistakes.

break-even near 2 environments manual: 2 hours eachIaC: 4 hours once, then 0.1 each 01020 0510environments built total hours spent (illustrative)
Figure 1.2 — An illustrative cost comparison. Writing infrastructure code costs more up front and much less every time afterwards.

Check your understanding

  • Name the five broad areas that infrastructure covers.
  • Why do ad-hoc scripts struggle when run a second time?
  • What does it mean that IaC is idempotent?
  • Give one strength and one weakness of writing IaC in a general-purpose language.
  • What is a Pulumi stack, and why might a project have two?
  • A region has 4 availability zones and the code creates 2 machines per zone. How many machines exist? (Answer: 8.)
  • With manual builds of 3 hours each, and IaC costing 6 hours to write and 0.2 hours per run, after how many environments does IaC break even? (Answer: 3n = 6 + 0.2n gives n of about 2.1, so from the third.)

Install Chrysolites Tech Academy

Android / Chrome / Edge: open the browser menu and choose Install app or Add to Home screen.

iPhone / iPad (Safari): tap the Share button, then Add to Home Screen.