Bash Scripting for DevOps

Sooner or later every DevOps task turns into a shell script: a deploy step, a backup job, a health check, some glue in a CI pipeline. Bash is everywhere because it is already on every Linux box, and a small script beats a whole program for wiring commands together. The catch is that Bash is full of sharp edges - a missing quote or an unhandled error can turn a helpful script into one that silently deletes the wrong thing.

This guide teaches the parts you actually use, and just as importantly the safety habits that keep scripts from betraying you. If you are new to the command line, read Linux for DevOps: The Fundamentals first - this builds directly on pipes, redirects, and exit codes.

The shebang and running a script

A script is a text file of commands. The first line, the shebang, tells the system which interpreter to run it with:

#!/usr/bin/env bash
echo "hello from a script"

#!/usr/bin/env bash finds bash on the PATH, which is more portable than hardcoding #!/bin/bash. Prefer bash over sh - on many systems sh is a stripped-down shell (dash) that lacks [[ ]], arrays, and other things you will want.

Save it as deploy.sh, then make it executable and run it:

chmod +x deploy.sh     # add the execute bit
./deploy.sh            # run it (the ./ is required - PATH doesn't include .)

You can also run a script without the execute bit by handing it to the interpreter directly: bash deploy.sh. That is handy for a quick test, but for anything you install, set the execute bit and give it a real shebang.

Variables and the quoting rules that matter most

You assign a variable with no spaces around the =, and read it back with $:

name="web-server"
count=3
echo "Deploying $name x$count"      # Deploying web-server x3

Now the single most important rule in Bash: always quote your variable expansions with double quotes. "$var" is safe; a bare $var is a trap. Here is why. When Bash sees an unquoted $var, it takes the value, splits it on whitespace, and then expands any glob characters (*, ?) it finds. Quoting turns that off and passes the value through as a single, literal string.

file="my report.txt"

rm $file        # BROKEN: runs `rm my report.txt` - tries to delete TWO files
rm "$file"      # correct: deletes the one file "my report.txt"

The unquoted version splits my report.txt into my and report.txt on the space, so rm gets two arguments. With real filenames, cron jobs, and CI variables that can contain spaces or be empty, this is how scripts quietly do the wrong thing. The habit to build: if it has a $, wrap it in "...". There is almost never a reason not to.

A related helper is ${var} braces, which disambiguate where the name ends:

env="prod"
echo "${env}_config.yml"     # prod_config.yml
echo "$env_config.yml"       # BROKEN: looks for a variable named env_config

You will also see ${var:-default}, which substitutes a default when the variable is unset or empty - useful for optional settings:

port="${PORT:-8080}"         # use $PORT if set, otherwise 8080

Command substitution

To capture the output of a command into a variable, use $( ):

today="$(date +%Y-%m-%d)"
branch="$(git rev-parse --abbrev-ref HEAD)"
echo "Backing up on $today from branch $branch"

Prefer $( ) over the older backtick form `...` - it nests cleanly and is far easier to read. And notice the quotes around the whole thing: "$(...)" keeps multi-word output intact, same rule as any other expansion.

Positional arguments

Values passed on the command line arrive as $1, $2, and so on. $0 is the script name itself. A few special variables round it out:

• $# - the number of arguments.
• $@ - all the arguments, as separate quoted words when written "$@".
• $* - all the arguments as one string (you almost always want "$@" instead).

#!/usr/bin/env bash
# usage: ./greet.sh <environment> <service>

echo "Script: $0"
echo "Got $# arguments"
echo "Environment: $1"
echo "Service: $2"

for arg in "$@"; do
  echo "arg -> $arg"
done

A common opening move is to check that you got the arguments you need and bail out with a usage message if not:

if [[ $# -lt 2 ]]; then
  echo "usage: $0 <environment> <service>" >&2
  exit 1
fi

Note >&2 sends the message to stderr, where errors belong, and exit 1 signals failure. Always write "$@" with quotes when passing arguments onward (for example to another command) so arguments containing spaces survive as single arguments.

Conditionals

Bash decides true or false based on exit codes: a command that exits 0 is "true," anything else is "false." That is the engine under every if.

if grep -q "ERROR" app.log; then
  echo "found errors"
else
  echo "clean"
fi

grep -q prints nothing and just sets its exit code, so the if reacts to whether a match was found. For comparing values and testing files, use [[ ]] (the modern Bash conditional). It is safer than the older [ ]/test because it does not word-split its operands and supports &&, ||, and pattern matching:

if [[ "$env" == "prod" ]]; then
  echo "production - be careful"
fi

if [[ -f "/etc/nginx/nginx.conf" ]]; then
  echo "config exists"
fi

if [[ "$count" -gt 5 && -d "/var/log" ]]; then
  echo "many items and the log dir is present"
fi

The operators you reach for most:

• Strings: ==, !=, -z "$x" (empty), -n "$x" (not empty).
• Numbers: -eq, -ne, -gt, -ge, -lt, -le.
• Files: -f (regular file), -d (directory), -e (exists), -r/-w/-x (readable/writable/executable).

You can also branch straight off a command's success without if, using && (then) and || (else):

mkdir -p /backups && echo "ready"       # echo only if mkdir succeeded
curl -sf "$url" || echo "endpoint down" # echo only if curl failed

The old single-bracket [ "$a" = "$b" ] still works and is what POSIX sh requires, but in Bash scripts prefer [[ ]] - it removes a whole class of quoting bugs.

Loops

A for loop iterates over a list. Quote the loop variable when you use it, and quote "$@" or a glob so items with spaces stay intact:

for service in nginx postgres redis; do
  systemctl is-active --quiet "$service" && echo "$service up" || echo "$service DOWN"
done

for conf in /etc/myapp/*.conf; do    # glob expands to real filenames
  echo "checking $conf"
done

A C-style for is handy for counting:

for i in {1..5}; do
  echo "attempt $i"
done

A while loop runs as long as its condition is true - good for retries and polling:

attempt=0
until curl -sf "http://localhost:8080/health"; do
  attempt=$((attempt + 1))
  [[ $attempt -ge 10 ]] && { echo "gave up after $attempt tries" >&2; exit 1; }
  echo "not ready, retrying in 2s..."
  sleep 2
done

Reading a file line by line

This is a spot people get wrong constantly. The correct, safe pattern is while read with IFS= and -r:

while IFS= read -r line; do
  echo "line: $line"
done < servers.txt

IFS= stops leading and trailing whitespace from being trimmed, and -r stops backslashes from being interpreted as escapes. Do not loop over a file with for line in $(cat file) - that splits on every space, not just newlines, so any line with a space in it breaks apart. while IFS= read -r reads exactly one line at a time, whitespace and all.

Functions and returning values

Functions keep scripts readable and let you reuse logic. Define one, then call it by name with arguments - inside, the arguments are $1, $2, and so on, just like the script's own:

log() {
  echo "[$(date +%H:%M:%S)] $*"
}

log "starting deploy"     # [14:03:12] starting deploy

Here is the part that trips people up: a Bash function's return value is an exit code (0-255), not data. return is for success/failure, and you test it with if or $?. To hand back actual data, either echo it and capture with command substitution, or set a variable:

# Pattern 1: echo the result, capture it
get_pod_count() {
  echo 3            # in reality: kubectl get pods --no-headers | wc -l
}
count="$(get_pod_count)"
echo "pods: $count"

# Pattern 2: return an exit code for pass/fail
is_healthy() {
  curl -sf "http://localhost:8080/health" >/dev/null
  # the function's exit code IS curl's exit code
}
if is_healthy; then
  echo "healthy"
fi

By default, variables set inside a function are global. Use local for anything that should not leak out - it prevents a function from stomping on a variable somewhere else in the script:

retry() {
  local url="$1"
  local tries="${2:-3}"
  # url and tries only exist inside retry
}

Exit codes and error handling

Every command sets $? to its exit code: 0 for success, non-zero for failure. Your scripts should do the same, so callers (a CI job, another script, a human) can tell whether things worked:

if ! ./run-migrations.sh; then
  echo "migration failed, aborting" >&2
  exit 1
fi

echo "all good"
exit 0

You can check $? directly, but do it immediately - the next command overwrites it:

./backup.sh
status=$?           # grab it right away
if [[ $status -ne 0 ]]; then
  echo "backup exited with $status" >&2
fi

For cleanup that must run no matter how the script ends, use a trap:

tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT     # runs on normal exit, error, or Ctrl+C

The safety header: set -euo pipefail

Put this near the top of every serious script:

#!/usr/bin/env bash
set -euo pipefail

By default Bash is dangerously forgiving - it barrels on after a failed command and treats undefined variables as empty. These three flags fix that:

• set -e (errexit) - exit immediately if any command fails (returns non-zero). No more plowing ahead after a broken step and doing damage on top of it.
• set -u (nounset) - treat use of an unset variable as an error instead of silently substituting an empty string. This alone prevents the classic rm -rf "$PREFIX/" disaster when PREFIX was never set, which would otherwise expand to rm -rf /.
• set -o pipefail - make a pipeline fail if any command in it fails, not just the last one. Without this, broken_command | tee log.txt reports success because tee succeeded, hiding the real failure.

set -e is not magic - a command in an if condition or one followed by || true is allowed to fail without aborting, which is exactly what you want. When you genuinely expect a command might fail and want to handle it yourself, write some_command || handle_it and the script continues.

set -euo pipefail

# expected-to-maybe-fail commands need explicit handling under set -e:
grep -q "flag" config || echo "flag not set, using default"

Pipes and combining commands

The real power of shell scripting is chaining small tools with pipes, where each command's stdout feeds the next command's stdin. This is where scripts earn their keep on logs and command output:

# Top 5 IPs hitting the server
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -5

# Count non-comment, non-blank lines in a config
grep -vE '^\s*(#|$)' app.conf | wc -l

# Extract failed systemd units and act on each
systemctl --failed --no-legend | awk '{print $2}' | while IFS= read -r unit; do
  echo "restarting $unit"
  systemctl restart "$unit"
done

With set -o pipefail in effect, if any stage of these pipelines fails, the whole pipeline reports failure - so your error handling actually catches it.

A real health-check script

Here is a small but complete script that pulls the pieces together: safety header, arguments with a default, functions, a loop over endpoints, exit codes, and cleanup. It checks a list of HTTP endpoints and exits non-zero if any are down - exactly the kind of thing you would drop into cron or a CI gate.

#!/usr/bin/env bash
set -euo pipefail

# usage: ./healthcheck.sh [timeout_seconds]
# reads one URL per line from endpoints.txt

timeout="${1:-5}"
endpoints_file="endpoints.txt"
failed=0

log() {
  echo "[$(date +%H:%M:%S)] $*"
}

check() {
  local url="$1"
  local code
  code="$(curl -o /dev/null -s -w '%{http_code}' --max-time "$timeout" "$url" || echo "000")"

  if [[ "$code" == "200" ]]; then
    log "OK   $url ($code)"
    return 0
  else
    log "FAIL $url ($code)"
    return 1
  fi
}

if [[ ! -f "$endpoints_file" ]]; then
  echo "missing $endpoints_file" >&2
  exit 1
fi

while IFS= read -r url; do
  [[ -z "$url" || "$url" == \#* ]] && continue   # skip blanks and comments
  check "$url" || failed=$((failed + 1))
done < "$endpoints_file"

if [[ "$failed" -gt 0 ]]; then
  log "$failed endpoint(s) unhealthy"
  exit 1
fi

log "all endpoints healthy"
exit 0

Every safe habit from this guide shows up here: set -euo pipefail up top, every expansion quoted, local inside functions, while IFS= read -r for the file, ${1:-5} for an optional argument, functions returning exit codes for pass/fail, || echo "000" to handle an expected failure under set -e, and a meaningful exit code at the end so a pipeline or cron job knows the result.

Common pitfalls

Unquoted variables (word splitting). The number-one Bash bug. cp $src $dest breaks the moment a path has a space or is empty. Quote every expansion: cp "$src" "$dest". When in doubt, add the quotes.

Unquoted globs and unexpected expansion. A bare $var doesn't just split on whitespace, it also expands * and ? against the filesystem. A variable that happens to hold * can match every file in the directory. Quoting stops it.

Parsing ls. Never do for f in $(ls) or scrape ls -l output - filenames can contain spaces, newlines, and glob characters, and ls formatting is not stable. Use a glob (for f in *.log) or find ... -print0 piped to while IFS= read -r -d '' f.

Reading files with for. for line in $(cat file) splits on every whitespace character, not on lines. Always use while IFS= read -r line; do ... done < file.

Forgetting set -euo pipefail. Without it, a failed command mid-script is ignored and the script keeps going, often doing more damage. Without -u, a typo'd variable name silently becomes an empty string. Add the header by reflex.

Spaces around = in assignments. name = "web" is not an assignment - Bash reads it as running a command called name. It must be name="web" with no spaces.

Expecting return to hand back data. A function returns an exit code (0-255), not a value. To return data, echo it and capture with "$(func)".

cd without checking. If cd /some/path fails and the next line is rm -rf *, you have just wiped the wrong directory. Under set -e a failed cd aborts the script, which is one more reason the safety header matters; or write cd /some/path || exit 1 explicitly.

Get quoting and the safety header into your fingers first - they prevent the scariest failures. Everything else here is the vocabulary you will reach for daily once those two habits are automatic.