Process Management and AutomationLesson 5.1

Bash background jobs and job control explained

background with &, jobs command, wait builtin, fg and bg, disown, nohup, job IDs vs PIDs, parallel execution pattern, collecting exit codes from background jobs

Running Commands in the Background

Append & to run a command asynchronously. The shell immediately continues to the next line. Use wait to synchronize.

#!/usr/bin/env bash
set -euo pipefail

# Run three tasks in parallel
process_region "us-east" &
pid1=$!
process_region "eu-west" &
pid2=$!
process_region "ap-south" &
pid3=$!

# Wait for all and capture exit codes
wait $pid1; status1=$?
wait $pid2; status2=$?
wait $pid3; status3=$?

if (( status1 + status2 + status3 > 0 )); then
  echo "One or more regions failed" >&2
  exit 1
fi
echo "All regions processed successfully"

Parallel with Controlled Concurrency

max_jobs=4
for file in /data/*.csv; do
  process_file "$file" &
  # Throttle: wait when we hit the limit
  while (( $(jobs -r | wc -l) >= max_jobs )); do
    sleep 0.5
  done
done
wait  # wait for all remaining jobs

Persistent Background Jobs

# Survives terminal close; stdout/err to file
nohup long_running_task.sh > task.log 2>&1 &
echo "PID: $!"

# Or disown a running job (after &)
some_task &
disown $!

Always capture $! immediately after & — it holds the PID of the most recently backgrounded command and gets overwritten with the next one.