Module 1.7 — Redirection and standard streams

Every command you run has three quiet channels attached to it: one that carries its input, one that carries its normal results, and one that carries its complaints. Once you can see those channels, you can send each of them somewhere useful, such as a file, another command, or nowhere at all.

Sam is about to start keeping a proper log of the tidewatch readings, and that needs exactly this skill.

By the end you should be able to:

• name the three standard streams and say which one a message travels on;
• save output with > and >>, and feed a file to a command with <;
• capture errors with 2>, merge streams with 2>&1, and know where &> is and is not available;
• silence a stream with /dev/null, and explain why keeping logs and errors apart is usually wise.

Scroll Core ideas → worked examples → Quiz. Plan on roughly 60 minutes, with a terminal open in ~/cli-practice/tidewatch/.

Core ideas — where output goes, and how to steer it

Up to now, every command Sam has run has printed its answer straight onto the screen. That is a comfortable default, but it hides something important: the screen is just one possible destination. Behind the scenes, each command writes to named channels, and the shell decides where those channels lead. When you learn to steer them, a one-off command becomes something you can keep, reuse, and combine.

This module is about those channels, called standard streams, and about the small set of symbols (>, >>, <, 2>, 2>&1, &>) that redirect them. The symbols look cryptic at first, so we will build them one at a time and always say what they mean in plain words.

---

Three streams, three jobs

Whenever a command starts, the system hands it three open channels. Each has a number, a name, and a purpose.

| Number | Name | Purpose | Default destination | |--------|------|---------|---------------------| | 0 | stdin (standard input) | where the command reads from | your keyboard | | 1 | stdout (standard output) | where normal results are written | your terminal screen | | 2 | stderr (standard error) | where warnings and error messages are written | your terminal screen |

The numbers 0, 1, and 2 matter later, because they are how you tell the shell which stream you are redirecting. For now, hold onto the idea of one way in, two ways out.

Why two ways out? Imagine a command that lists a hundred files and also hits one folder it cannot open. The hundred file names are the answer. The complaint about the one locked folder is a side message. If both were mixed in a single stream, you could not save the answer without also saving the complaint. Keeping them separate means you can treat them differently, and that is the whole point of this module.

Here is the surprising part. On your screen, stdout and stderr look identical. Both appear as plain text in the same window. You cannot tell by eye which stream a line travelled on. You only find out when you redirect one of them and notice that a particular line either vanished from the screen or stayed behind.

---

Redirect stdout into a file — >

echo "gauge A calibrated" > notes/calibration.txt

The > symbol means "take what this command writes to stdout and put it in this file instead of on the screen." After you press Enter, the terminal prints nothing, because the text went into notes/calibration.txt. Check with:

cat notes/calibration.txt

gauge A calibrated

Two details deserve attention.

The file is created if it does not exist. You did not need touch first. The shell opens the file for you.

If the file already exists, > replaces its contents. It does not add to the end; it empties the file and starts over. This is the single most common way beginners lose their own notes. Suppose calibration.txt had ten lines and you run echo "new" > notes/calibration.txt: the ten lines are gone, and the file holds one. There is no undo and no trash can.

A subtle point: the shell empties the file before the command even runs. So even a command that fails, or that prints nothing, can leave the target file empty. Treat every > as a small act of overwriting and ask yourself, "is this file safe to replace?"

---

Append instead of replace — >>

echo "gauge B calibrated" >> notes/calibration.txt

The doubled symbol >> means "add to the end of the file." Existing lines stay where they are, and the new text lands after them. If the file does not exist yet, >> creates it, just as > does.

cat notes/calibration.txt

gauge A calibrated
gauge B calibrated

A useful habit: when you are building a log that grows over time, start it with > once (to begin fresh) and add to it with >> afterward. When you are unsure whether the file matters, choose >>; the worst outcome of an unneeded append is a few extra lines, while the worst outcome of an unneeded > is lost data.

---

Feed a file into a command — <

So far we have redirected output. The symbol < redirects input: it connects a file to the command's stdin, as if you had typed the file's contents on the keyboard.

wc -l < notes/calibration.txt

2

Here wc -l counts lines. With <, the command reads the file's text from stdin and prints only the number. Compare it with the version that names the file:

wc -l notes/calibration.txt

2 notes/calibration.txt

When wc is given a file name, it knows the name and prints it next to the count. When it reads from stdin, it has no idea where the text came from, so it prints just the count. Neither is better; they answer slightly different needs. A clean number is handy when you want to use the result somewhere else.

Most commands you have met accept a file name as an argument, so < is not always necessary. It matters most for commands that only read stdin, and for the pipelines you will build in the next module.

---

Redirect the error stream — 2>

By default, > handles stdout only. To capture stderr, you put its number in front:

ls notes/ missing_folder > listing.txt

Suppose missing_folder does not exist. The listing of notes/ goes into listing.txt, but the complaint (ls: missing_folder: No such file or directory) still appears on your screen. That surprise confirms that the complaint travelled on stderr, which > did not touch.

Now redirect stderr instead:

ls notes/ missing_folder 2> errors.txt

This time the normal listing of notes/ appears on the screen, and the complaint goes into errors.txt. The reading is: "stream 2, send it to this file."

You can redirect both streams in one command, to different files:

ls notes/ missing_folder > listing.txt 2> errors.txt

Now the screen stays quiet; the answer is in listing.txt and the complaint is in errors.txt. This is the pattern that makes logs trustworthy, and we will return to it below.

There is also 2>>, which appends stderr to a file rather than replacing it, just as >> does for stdout.

---

Send stderr to the same place as stdout — 2>&1

Sometimes you want one file that holds everything, in the order it happened. For that, you write:

ls notes/ missing_folder > everything.txt 2>&1

Read 2>&1 as: "send stream 2 to wherever stream 1 is currently going." The & before the 1 is the shell's way of saying "I mean the stream numbered 1, not a file named 1." Without the &, you would be redirecting errors into a file literally called 1.

Order matters. The shell processes redirections left to right. In > everything.txt 2>&1, stdout is first pointed at the file, and then stderr is pointed at "wherever stdout now goes", which is the file. Both end up there. If you reverse them:

ls notes/ missing_folder 2>&1 > everything.txt

then stderr is first pointed at "wherever stdout goes now" (the screen), and only afterward is stdout moved to the file. The result: the listing goes into the file, but the error still appears on your screen. Same symbols, different meaning, purely because of order. When in doubt, use the first form: file first, then 2>&1.

---

The shorthand — &>

Many shells accept a shorter spelling for "stdout and stderr both":

ls notes/ missing_folder &> everything.txt

This is convenient, but it comes with a portability note. &> is a bash feature (and zsh understands it too). It is not part of the most basic, widely portable shell standard. A script written for plain sh on a minimal system may not understand it, and may even interpret it in a surprising way (running the command in the background, then making an empty file). If you are typing at your own prompt in bash or zsh, &> is fine. If you are writing something other people or other machines will run, prefer the longer and more portable > file 2>&1.

---

Throw output away — /dev/null

Your system has a special "file" named /dev/null that accepts anything you write to it and keeps nothing. People call it the bit bucket. Redirecting a stream there silences it:

ls notes/ missing_folder 2> /dev/null

The listing appears and the complaint disappears. You can silence the normal output instead with > /dev/null, or both with > /dev/null 2>&1.

Use this carefully. Silencing errors means you will not see a problem even when one exists. It is reasonable when you expect a harmless complaint and do not want the noise (for example, checking whether a file exists and only caring about the answer), and unwise as a way to make a confusing command "look clean." If a command printed errors and you hid them, the errors did not go away; you just lost the evidence.

---

Why separating the streams matters

Picture Sam's nightly check on the gauges. It prints one line per reading, which is the log, and occasionally prints a warning when a gauge is silent. If both were mixed into one file, the person reading it later would have to hunt for warnings among hundreds of ordinary lines. If they are separated, readings.log holds the steady record and warnings.txt holds only what needs attention. An empty warnings.txt is itself good news.

Separation also protects pipelines (next module). When one command's output feeds another, you want only real results to flow, not a stray error message that the second command would treat as data.

---

Redirection and the file system: a mental picture

Think of each command as a small machine with three pipes attached. Redirection does not change the machine; it changes where the pipes lead. The command itself does not know whether stdout points at your screen, a file, or the bit bucket. That is why the same ls works in all those situations without being rewritten, and why redirection is such a general tool.

---

Common pitfalls

• Overwriting by accident with >. If the file matters, use >>, or look first with ls or cat.
• Forgetting that errors skip >. If errors still show on the screen after you redirected, they are on stderr. Add 2>.
• Writing 2>&1 before the file redirect. Put the file redirect first.
• Assuming &> works everywhere. It is bash and zsh friendly, not universal.
• Reading a file into itself. A command like sort data.txt > data.txt empties the file before sort can read it. Always write to a different file.
• Hiding errors with /dev/null and then wondering why nothing works.

---

Common confusions

• "Is stderr only for fatal errors?" No. It is for any diagnostic or warning-style message, even when the command carries on and succeeds.
• "Why did my file end up empty?" Either the command printed nothing to stdout, or its output went to stderr, or you used > on a file the command was also reading.
• "Do > and 1> differ?" No. > is short for 1>.
• "What does < do to the file?" It only reads. It does not change the file.

---

Self-study checklist

• Name the three standard streams and give their numbers.
• Explain the difference between > and >> in one sentence each, and say which is risky.
• Run wc -l < file and wc -l file and explain why the outputs differ.
• Use 2> to capture an error from a command you know will fail, and confirm it with cat.
• Explain why > out.txt 2>&1 and 2>&1 > out.txt behave differently.
• Say where &> works and where you would prefer the longer form.
• Give one good reason and one bad reason to send output to /dev/null.
• Describe, in your own words, why a log and its errors are often kept in separate files.

Worked examples — keeping a tidy gauge log

Sam wants the nightly gauge check to leave a record behind. The check is a tiny script that reports each gauge's water level and complains when a gauge goes quiet. Each scene gives a situation, the commands, an interpretation, and a takeaway. Your spacing may differ slightly; focus on which stream went where.

First, a small setup so you can follow along:

cd ~/cli-practice
mkdir -p tidewatch/notes tidewatch/scripts tidewatch/logs
printf 'Oct 12: calm morning\nOct 13: surge at gauge A\nOct 14: repaired cable\n' > tidewatch/notes/field_log.txt
printf '#!/bin/sh\necho "gauge A: 160 cm"\necho "gauge B: 142 cm"\necho "warning: gauge C is silent" >&2\necho "gauge D: 97 cm"\n' > tidewatch/scripts/check_gauges.sh
chmod 755 tidewatch/scripts/check_gauges.sh
cd tidewatch

The script prints three readings on stdout and one warning on stderr (the >&2 inside the script sends that one line to stream 2). Treat it as a command that uses both streams.

---

Scene 1 — Everything on the screen, then half of it in a file

Scene. Sam runs the check once with no redirection, then again saving only the normal output.

./scripts/check_gauges.sh
./scripts/check_gauges.sh > logs/readings.log
cat logs/readings.log

gauge A: 160 cm
gauge B: 142 cm
warning: gauge C is silent
gauge D: 97 cm
warning: gauge C is silent
gauge A: 160 cm
gauge B: 142 cm
gauge D: 97 cm

The first four lines come from the first run. The fifth line is all the second run printed on screen. The last three lines are what cat shows from the file.

Interpretation. In the first run, readings and warning are visually mixed. In the second, > captured only stdout, so the three readings went into the file. The warning travelled on stderr, which > does not touch, so it still reached the screen. That one stray line is the visible proof that it lives on a different stream.

Takeaway. If a message still shows up after you redirect with >, it is on stderr. That is the design, not a mistake.

---

Scene 2 — Two streams, two files

Scene. Sam wants a clean separation: readings in one file, warnings in another, nothing on screen. Then the next night's run should be added, not overwrite.

./scripts/check_gauges.sh > logs/readings.log 2> logs/warnings.txt
wc -l logs/readings.log logs/warnings.txt
cat logs/warnings.txt
./scripts/check_gauges.sh >> logs/readings.log 2>> logs/warnings.txt
wc -l logs/readings.log logs/warnings.txt

       3 logs/readings.log
       1 logs/warnings.txt
       4 total
warning: gauge C is silent
       6 logs/readings.log
       2 logs/warnings.txt
       8 total

Interpretation. The screen stayed quiet during both runs. The first run split four lines by stream: three into the log, one into the warnings file. The second run, using >> and 2>>, added three more readings and one more warning. Had Sam used > and 2> again, both files would have been reset to their first sizes and last night's record would be lost.

Takeaway. Use > to start a record, and >> / 2>> to keep growing it. An empty warnings file is itself good news.

---

Scene 3 — One combined file, and the order trap

Scene. Sam also wants a single timeline with everything in the order it happened.

./scripts/check_gauges.sh > logs/all.txt 2>&1
cat logs/all.txt

gauge A: 160 cm
gauge B: 142 cm
warning: gauge C is silent
gauge D: 97 cm

Curious, Sam tries the reversed order:

./scripts/check_gauges.sh 2>&1 > logs/reversed.txt
cat logs/reversed.txt

warning: gauge C is silent
gauge A: 160 cm
gauge B: 142 cm
gauge D: 97 cm

The first line was printed on screen by the script run; the other three are the file's content.

Interpretation. In the first form, stdout was pointed at the file and then stderr was pointed at "wherever stdout goes", so the warning sits between the readings. In the reversed form, stderr was copied to the screen first, and only afterward was stdout moved to the file. The shell reads redirections left to right and uses the situation as it stands at that moment.

Takeaway. To merge both streams into a file, write > file 2>&1, in that order. In bash or zsh, &> file is a shorter spelling, but prefer the longer form in anything you share.

---

Scene 4 — Quiet runs with /dev/null

Scene. Gauge C is known to be offline for repair, and Sam only wants today's readings.

./scripts/check_gauges.sh 2> /dev/null
./scripts/check_gauges.sh > /dev/null

gauge A: 160 cm
gauge B: 142 cm
gauge D: 97 cm
warning: gauge C is silent

Interpretation. The first command threw the warning into the bit bucket, leaving three readings. The second threw the readings away, leaving only the warning, which is a handy way to ask "did anything go wrong?" Hiding the warning in the first command is reasonable because Sam knows why it appears; the cost is that a second silent gauge would also go unnoticed.

Takeaway. /dev/null means "I do not need this stream right now." Hide streams deliberately, and leave errors visible when you are unsure.

---

Scene 5 — Input from a file, and a self-overwrite trap

Scene. Sam counts the lines in the field log in two ways, then nearly makes a classic mistake.

wc -l < notes/field_log.txt
wc -l notes/field_log.txt

       3
       3 notes/field_log.txt

Interpretation. With <, the command read from stdin and printed only a number. With a file name, it printed the name too. Same count, different decoration.

Sam then wants a sorted log and starts to type this line, but stops:

sort notes/field_log.txt > notes/field_log.txt     # do NOT run this

The shell empties the target file before sort starts reading, so the log would be left empty. Sam writes to a new file instead:

sort notes/field_log.txt > notes/field_log_sorted.txt
cat notes/field_log_sorted.txt

Oct 12: calm morning
Oct 13: surge at gauge A
Oct 14: repaired cable

Takeaway. Never redirect a command's output into the file it is reading. Write to a new file, check it, and replace the original only if you still want to.