linux command-line

Pipe

pipe shell command-line unix-philosophy bash
Plain English

A pipe connects two commands so the output of the first becomes the input of the second. It is written with the | character. Instead of saving results to a file and opening it with another program, you chain small tools together: list files, filter them, count them, all in one line. Pipes are the heart of how Linux command-line work gets done.

Technical Definition

A pipe connects the stdout of one process to the stdin of the next through a kernel buffer (the pipe() syscall). All commands in a pipeline run at the same time; the next one reads as the previous one writes.

Building blocks commonly piped together:

  • grep: filter lines
  • sort, uniq -c: order and count
  • cut, awk: pick columns
  • sed: edit text in the stream
  • head, tail: limit lines
  • wc -l: count lines
  • xargs: turn input lines into arguments for another command
  • tee: write to a file and pass the stream along

Details that matter:

  • Pipes carry stdout only. Add 2>&1 (or |& in Bash) to include stderr.
  • A pipeline’s exit code is the last command’s. set -o pipefail makes scripts fail if any stage fails.
  • sudo applies to one command, not the pipeline: cmd | sudo tee /etc/file is the way to write a root-owned file.

Named pipes (FIFOs), made with mkfifo, do the same between unrelated processes.

Source: pipe(7) Linux manual page

Pipelines you will actually use 

# Top 5 IPs hitting a web server
$ awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -5
   1842 203.0.113.7
    911 198.51.100.23

# Biggest directories in /var
$ sudo du -sh /var/* 2>/dev/null | sort -h | tail -5

# Failed SSH logins today, counted
$ journalctl -u ssh --since today | grep -c "Failed password"

# Save and view at the same time
$ dmesg | grep -i error | tee dmesg-errors.txt
In the Wild

Pipes are how a one-line answer replaces an hour of scrolling. Log analysis, finding what is filling a disk, counting failed logins, and filtering command output all come down to chaining grep, sort, uniq, and awk. Build a pipeline one stage at a time, checking output as you go. In scripts, add set -o pipefail so a failure in the middle is not hidden by a successful last command.

← Back to Dictionary