Chrysolites Tech Academy

DevOps Foundations: Linux, Pipelines and Delivery · Course book

Chapter 1: The Ground DevOps Stands On

Pairs with Lesson 1: DevOps Prerequisites: Linux, Networking and YAML

DevOps tools such as Docker, Kubernetes and CI/CD pipelines are built on a handful of older, more basic skills. Almost all of them run on Linux, are configured through text files, and connect machines over networks. People who skip these foundations often find DevOps confusing, because every error message assumes them. This chapter covers the four prerequisites that matter most: working at the Linux command line, editing files and managing software and services, building a small practice lab of virtual machines, understanding just enough networking, and reading the two data formats, YAML and JSON, in which nearly every DevOps tool is configured.

The Linux command line

Most servers, and most of the tools in this course, run on Linux. Servers rarely have a graphical desktop, so you control them by typing commands into a shell, the program that reads commands and runs them. The shell prompt usually shows your user name, the machine's name and your current folder, ending in $ for an ordinary user or # for the all-powerful administrator account, called root.

A small set of commands covers most daily work.

CommandWhat it doesExample
pwdPrint the current folderpwd shows /home/ana
lsList the files in a folderls -l shows details such as size and permissions
cdChange foldercd /etc moves to the configuration folder
mkdirMake a foldermkdir -p projects/web creates both levels at once
cp, mv, rmCopy, move or remove filesmv notes.txt archive/
cat, lessShow the contents of a fileless /var/log/syslog lets you scroll through a long log
sudoRun one command as the administratorsudo systemctl restart nginx

Linux arranges everything in a single tree of folders starting at /, the root. A path beginning with / is absolute, such as /home/ana/notes.txt; a path without one is relative to the current folder. Two shortcuts help: . means the current folder and .. means the folder above it. Configuration files conventionally live in /etc, logs in /var/log and users' own files in /home.

/ etchomevarusrnginxanalog notes.txtsyslog programs
Figure 1.1 — Part of the Linux folder tree. Everything, including devices and settings, hangs somewhere beneath the single root folder.

Commands become far more powerful when combined. The pipe symbol, |, sends one command's output into another: grep error /var/log/syslog | wc -l counts the lines of a log that mention "error". The symbol > saves output to a file, replacing its contents, and >> adds to the end instead. Pressing Tab completes a file or command name, and the up arrow recalls earlier commands, two habits that save a great deal of typing.

Every file has permissions saying who may read, write or run it, for three groups: the file's owner, its group and everyone else. ls -l shows them as a string such as rwxr-xr-x. They are also written as three digits, one per group, where read counts 4, write 2 and run (execute) 1. So rwx is 4 + 2 + 1 = 7 and r-x is 4 + 1 = 5, which is why chmod 755 script.sh lets the owner do everything and everyone else read and run the file but not change it.

Editing files, installing software, running services

Because DevOps work is mostly about configuration, you will edit text files on servers constantly, often with only a terminal available. The editor vi, or its improved version vim, is installed almost everywhere. It confuses beginners because it has modes. It opens in command mode, where keys are commands, not text. Pressing i enters insert mode, where typing adds text; pressing Esc returns to command mode. There, typing :w saves, :q quits and :wq does both; :q! quits without saving. Five keystrokes are enough to survive at first, and the rest can be learned gradually.

Software is installed with a package manager, which downloads programs from trusted repositories, along with everything they depend on, and keeps them updated. Debian and Ubuntu use apt, so sudo apt install nginx installs a web server. Red Hat-family systems such as CentOS use yum or its successor dnf for the same job.

Programs that run continuously in the background, such as web servers and databases, are called services. On most modern distributions they are managed by systemd through the command systemctl: systemctl start, stop and restart control a service now, systemctl enable makes it start automatically at boot, and systemctl status shows whether it is running and its most recent log lines. When a service misbehaves, its status and its logs are the first places to look; journalctl -u followed by the service name shows its full log.

A practice lab of virtual machines

Learning DevOps means breaking things, and you want to break them somewhere safe. A virtual machine, VM, is a complete simulated computer running as a program on your own computer. A hypervisor such as VirtualBox shares out your real processor, memory and disk among one or more VMs, each with its own operating system. If you ruin a VM's configuration, you delete it and start again.

Creating VMs by hand, clicking through installation screens, is slow and hard to repeat. Vagrant automates it. A short text file, the Vagrantfile, describes the machines you want: which operating system image, how much memory, which network settings. The command vagrant up then builds them all, and vagrant destroy removes them. Because the description is a file, it can be shared and versioned, and two people running it get identical labs. That idea, describing infrastructure in files instead of setting it up by hand, is the seed of infrastructure as code, covered later in this course.

Rebuild, do not repair. When a practice machine gets into a muddle, destroying and recreating it from its file is usually faster than working out what went wrong. Real infrastructure is increasingly managed the same way.

Just enough networking

DevOps work constantly involves machines talking to each other, and most problems that look mysterious turn out to be networking. A few ideas go a long way.

Each machine's network interface has an IP address, such as 192.168.1.10, and belongs to a network described by an address and a size. The size is often written after a slash: 192.168.1.0/24 means the first 24 of the 32 bits identify the network and the last 8 identify machines on it. Eight bits give 2 to the power 8, or 256, addresses, of which 254 can be given to machines, because the first identifies the network itself and the last is reserved for broadcasting to everyone on it.

Machines on the same network talk directly, through a switch. To reach another network, a machine sends traffic to its gateway, a router that forwards it onwards. The routing table, shown by ip route, lists which gateway to use for which destinations, with a default route for everything else. The command ip addr shows a machine's own addresses.

VM: web192.168.1.10VM: database192.168.1.11 switch192.168.1.0/24 router (gateway)192.168.1.1 DNS: names to addresses internet
Figure 1.2 — A small lab network. Machines on the same network reach each other through the switch; everything else goes through the gateway.

Finally, DNS turns names into addresses. Linux reads the file /etc/hosts first, which is handy in a lab for giving machines friendly names, and then asks the DNS servers listed in /etc/resolv.conf. When "the server cannot be reached", the usual checks follow this order: does the name resolve to an address, does the address respond to ping, and is the service actually listening on the expected port?

Configuration formats: YAML and JSON

Docker Compose, Kubernetes, Ansible and CI pipelines are all configured with structured text files, almost always in YAML or JSON. Both describe the same three building blocks: key-value pairs, lists and dictionaries nested inside each other.

# YAML
server:
  name: web
  ports:
    - 80
    - 443

// JSON
{ "server": { "name": "web", "ports": [80, 443] } }

JSON uses braces for dictionaries, square brackets for lists and quotation marks around keys. YAML uses indentation instead of braces and a dash for each list item, which makes it easier to read and to write by hand. The price is that YAML's indentation is meaningful: two spaces in the wrong place change the structure, and tabs are not allowed at all. Many failed deployments come down to a misaligned line in a YAML file, so it is worth learning to read the structure at a glance. One more distinction matters: a dictionary is unordered, so the order of its keys does not matter, while a list is ordered, so the order of its items does.

Check your understanding

  • What is the difference between the paths /etc/hosts and etc/hosts?
  • What permissions does chmod 644 set? (Answer: 6 = 4 + 2, read and write for the owner; 4, read only, for the group and for everyone else.)
  • In vi, how do you save your changes and quit? (Answer: press Esc, then type :wq.)
  • Which command makes a service start automatically when the machine boots?
  • Why is describing a lab in a Vagrantfile better than building it by hand?
  • How many machine addresses does the network 10.0.5.0/24 provide? (Answer: 254.)
  • Rewrite the JSON { "app": { "replicas": 3 } } as YAML. (Answer: app: on one line, then replicas: 3 indented beneath it.)

Install Chrysolites Tech Academy

Android / Chrome / Edge: open the browser menu and choose Install app or Add to Home screen.

iPhone / iPad (Safari): tap the Share button, then Add to Home Screen.