Mastering the Linux sed Command: Syntax, Examples, and Practical Text Processing
The sed (Stream Editor) command in Linux is a versatile tool for manipulating text. It allows users to carry out tasks such as finding, replacing, inserting, and removing text in files. This tutorial delivers a detailed guide to understanding the sed command through hands-on examples, syntax breakdowns, and more advanced usage scenarios.
What Is the sed Command?
The sed command is a stream editor that handles text one line at a time. Because of this, you can alter file content without opening the file in a text editor. It is commonly used in shell scripts and system administration for automating text-processing operations.
Key Features of sed
- Pattern matching and replacement
- In-place file editing
- Text filtering and manipulation
- Support for regular expressions
- Multiline operations
Basic Syntax of the sed Command
The basic structure of the sed command includes three primary parts: command options, a script that defines the editing rules, and the file that will be processed.
This format makes it possible to control how the command behaves, describe the required text changes, and apply them to the selected file.
Command Options: These determine how the command operates. For instance, the -i option performs in-place editing, which means the original file is overwritten.
Script: The script contains the editing instructions. It may be wrapped in single quotes (‘) or double quotes (“). A script can include one or several editing commands, separated by semicolons (;).
Input File: This is the file that will be processed. It may be a single file or multiple files separated by spaces. If no file is provided, sed reads from standard input.
The general syntax of the sed command is shown below:
sed [options] 'script' file
In this syntax, sed is the command itself, [options] represents the command options, ‘script’ includes the editing instructions, and file refers to the file being processed.
sed [options] 'script' file
The following examples will make it easier to understand.
sed 's/hello/world/' sample.txt
This command replaces the first instance of hello with world on every line of sample.txt.
Commonly Used Options in sed
| Option | Description | Example |
|---|---|---|
| -i | Edits the file directly | sed -i 's/old/new/' file.txt |
| -n | Disables automatic output | sed -n '/pattern/p' file.txt |
| -e | Runs multiple commands | sed -e 's/old/new/' -e '/pattern/d' file.txt |
| -f | Loads commands from a file | sed -f script.sed file.txt |
| -r | Uses extended regular expressions | sed -r 's/old/new/' file.txt |
| -E | Uses extended regular expressions, similar to -r |
sed -E 's/old/new/' file.txt |
| -z | Treats lines as separated by the NUL character | sed -z 's/old/new/' file.txt |
| -l | Sets the line length for the l command |
sed -l 100 'l' file.txt |
| -b | Uses binary mode and does not remove CR characters | sed -b 's/old/new/' file.txt |
Most Common Use Cases of sed
Below are some of the most practical ways to use the sed command.
To begin, create a sample file named file1.txt and add the following text to it so you can follow along more easily.
cat > file1.txt
Then paste in the following content:
Linux is a family of free and open-source operating systems based on the Linux kernel.
Operating systems based on Linux are known as Linux distributions or distros.
Examples include Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Search and Replace
In the command below, s indicates the substitution action, while the / characters act as delimiters. /Linux/ is the search pattern, and Unix is the replacement text.
Note: By default, sed only replaces the first match of the pattern on each line. It does not change the second or third occurrence on the same line.
sed 's/Linux/Unix/' file1.txt
This command substitutes the first occurrence of Linux with Unix in every line.
Output:
Unix is a family of free and open-source operating systems based on the Linux kernel.
Operating systems based on Unix are known as Linux distributions or distros.
Examples include Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Unix, and many others.
Replace Globally in Each Line
The /g substitution flag tells sed to replace every occurrence of the matched string within each line.
sed 's/Linux/Unix/g' file1.txt
This command changes all occurrences of Linux to Unix on each line.
Output:
Unix is a family of free and open-source operating systems based on the Unix kernel.
Operating systems based on Unix are known as Unix distributions or distros.
Examples include Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Unix, and many others.
In-Place Editing
The -i option allows direct editing of the file. In simple terms, it overwrites the original file.
sed -i 's/Linux/Unix/' file1.txt
This command updates file1.txt directly by replacing Linux with Unix. Without -i, the change appears only in the command output and does not alter the actual file. To keep the modification permanently, you must use the -i option.
Delete Specific Lines
sed '2d' file1.txt
This command removes the second line from file1.txt.
Output:
Unix is a family of free and open-source operating systems based on the Linux kernel.
Examples include Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Unix, and many others.
Print Specific Lines
The -n option prevents automatic printing of the pattern space, and p is the print command.
sed -n '1,2p' file1.txt
This command displays lines 1 through 2 from file1.txt.
Output:
Unix is a family of free and open-source operating systems based on the Unix kernel.
Operating systems based on Unix are known as Unix distributions or distros.
Delete Lines Matching a Pattern
The /pattern/ expression matches lines that contain the pattern, and the d command removes those matched lines.
sed '/kernel/d' file1.txt
This command deletes every line that contains the word kernel.
Output:
Operating systems based on Unix are known as Unix distributions or distros.
Examples include Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Unix, and many others.
Substitute with a Backup File
The following command changes every occurrence of Unix back to Linux and creates a backup file called file1.txt.bak containing the earlier file content before replacement. The -i.bak option performs in-place editing while also generating a backup copy.
sed -i.bak 's/Unix/Linux/g' file1.txt
Output:
Linux is a family of free and open-source operating systems based on the Linux kernel.
Operating systems based on Linux are known as Linux distributions or distros.
Examples include Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Below is the content of the backup file file1.txt.bak.
more file1.txt.bak
Output
Unix is a family of free and open-source operating systems based on the Unix kernel.
Operating systems based on Unix are known as Unix distributions or distros.
Examples include Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Unix, and many others.
Replace Tabs with Spaces
The following command replaces each tab character with four spaces. The \t sequence matches tab characters, and the /g flag applies the replacement globally across the line.
sed 's/\t/ /g' file1.txt
Output:
Linux is a family of free and open-source operating systems based on the Linux kernel.
Operating systems based on Linux are known as Linux distributions or distros.
Examples include Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Remove Empty Lines
The following command removes every blank line from file1.txt. The /^$/ pattern identifies empty lines, and the d command deletes those matched lines.
sed '/^$/d' file1.txt
You can open file1.txt with the vi text editor and insert a few blank lines to test this command.
Print Lines Matching a Pattern
The following command shows only the lines that contain Ubuntu.
sed -n '/Ubuntu/p' file1.txt
The -n option turns off automatic printing. /Ubuntu/ matches lines containing the pattern, and p prints the matched lines.
Output:
Example includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Advanced Use Cases of sed
This section covers several advanced and more complex ways to use the sed command.
Insert Text Before a Line
The following command inserts This is inserted text. before the second line in file1.txt.
sed -i '2i\This is inserted text.' file1.txt
The -i option enables in-place editing, while the 2i\ instruction inserts text before the second line.
Note: Without -i, the inserted text appears only in the command output and does not change the file itself. To save the modification permanently, you must use the -i option with sed.
Output:
Linux is a family of free and open-source operating systems based on the Linux kernel.
This is inserted text.
Operating systems based on Linux are known as Linux distributions or distros.
Example includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Replacing the nth Occurrence of a Pattern in a Line
You can use flags such as /1 or /2 to replace the first or second occurrence of a pattern within a line. The command below replaces the second appearance of the word Linux with Unix in a line.
sed 's/Linux/Unix/2' file1.txt
Output:
Linux is a family of free and open-source operating systems based on the Unix kernel.
This is inserted text.
Operating systems based on Linux are known as Unix distributions or distros.
Examples includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Append String After a Line
The command below adds This is appended text. after the third line in file1.txt. The -i option ensures the modification is saved, and 3a\ appends the text after the specified third line.
sed -i '3a\This is appended text.' file1.txt
Output:
Linux is a family of free and open-source operating systems based on the Linux kernel.
This is inserted text.
Operating systems based on Linux are known as Linux distributions or distros.
This is appended text.
Example includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Replace String at the Beginning of a Line
The ^<pattern> expression is used to match a specific pattern at the beginning of a line. The command below replaces Linux with Unix only when Linux appears at the start of a line.
sed 's/^Linux/Unix/' file1.txt
Output:
Unix is a family of free and open-source operating systems based on the Linux kernel.
This is inserted text.
Operating systems based on Linux are known as Linux distributions or distros.
This is appended text.
Example includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Replace String at the End of a Line
The command below replaces distros. with distributions only when it appears at the end of a line. The <pattern>$ expression is used to match a specific pattern at the end of a line.
sed 's/distros.$/distributions/' file1.txt
Output:
Linux is a family of free and open-source operating systems based on the Linux kernel.
This is inserted text.
Operating systems based on Linux are known as Linux distributions or distributions.
This is appended text.
Example includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Case-Insensitive Replacement
The following command replaces linux with Unix without considering letter case. The I flag makes the pattern match case-insensitive.
sed 's/linux/Unix/I' file1.txt
Extract Lines Between Patterns
The command below prints all lines from inserted to appended, including both matching lines.
sed -n '/inserted/,/appended/p' file1.txt
,: Range operator used to match lines between two patterns.
p: Prints the matched lines.
And the -n option suppresses automatic line printing.
Output:
This is inserted text.
Operating systems based on Linux are known as Linux distributions or distros.
This is appended text.
Process Multiple Files
The following command replaces Linux with Unix in both file1.txt and file2.txt and overwrites the original files.
sed -i 's/Linux/Unix/' file1.txt file2.txt
Format and Number Non-Empty Lines
The command below adds line numbers to all non-empty lines in file1.txt.
sed '/./=' file1.txt | sed 'N;s/\n/ /'
/./=: Matches non-empty lines and prints their numbers.
N: Appends the next line to the pattern space.
s/\n/ /: Replaces the newline character with a space.
Output:
1 Linux is a family of free and open-source operating systems based on the Linux kernel.
2 This is inserted text.
3 Operating systems based on Linux are known as Linux distributions or distros.
4 This is appended text.
5 Example includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Replacing String on a Specific Line Number
You can limit the sed command so that a replacement happens only on a specific line number. The command below replaces the string distros with distributions only on the third line.
sed '3 s/distros/distributions/' file1.txt
Output:
Linux is a family of free and open-source operating systems based on the Linux kernel.
This is inserted text.
Operating systems based on Linux are known as Linux distributions or distributions.
This is appended text.
Example includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Replacing String on a Range of Lines
You can also provide a range of line numbers to sed when replacing text. The command below replaces only the first occurrences of Linux with Unix between lines 1 and 3.
sed '1,3 s/Linux/Unix/' file1.txt
Output:
Unix is a family of free and open-source operating systems based on the Linux kernel.
This is inserted text.
Operating systems based on Unix are known as Linux distributions or distros.
This is appended text.
Example includes Debian, Ubuntu, Fedora, CentOS, Gentoo, Arch Linux, and many others.
Performance Considerations for Large Files
Working with large files in sed can become resource-intensive, especially when many operations are involved or when datasets are very large. The following suggestions can help improve performance and keep sed usage efficient.
- Use -n to reduce unnecessary output – The -n option disables automatic printing of every line and ensures that only the required output is displayed. This helps reduce overhead when handling large files.Example:
sed -n '/pattern/p' largefile.txt
- Simplify Scripts – Reduce the number of operations in a single workflow. For example, instead of running several sed commands one after another, combine them into one script to reduce repeated file reads.Example:
sed -e 's/foo/bar/' -e '/pattern/d' largefile.txt
- Stream Input with Pipes – When handling data from other commands or input streams, use pipes to avoid creating intermediate files and to reduce disk I/O.Example:
cat largefile.txt | sed 's/foo/bar/' > output.txt
- Avoid In-Place Editing on Large Files – Instead of modifying large files directly, write the output to a separate file and replace the original only after confirming that the result is correct.Example:
sed 's/old/new/' largefile.txt > temp.txt && mv temp.txt largefile.txt
- Benchmark Alternatives – For extremely large files, tools such as awk, perl, or grep may deliver better performance for certain workloads.Example:
awk '{gsub(/old/, "new"); print}' largefile.txt > output.txt
Integration with Shell Scripts
The sed command is often used in shell scripts to automate repetitive text-editing tasks. Here is an example:
#!/bin/bash
# Replace all occurrences of "foo" with "bar" in input.txt and save the result
sed 's/foo/bar/g' input.txt > output.txt
This script reads input.txt and writes the modified result to output.txt.
sed vs Other Alternatives
Although sed is an efficient and lightweight solution for basic text processing, newer alternatives such as awk and perl provide additional capabilities that may be more suitable for certain tasks. Below is an overview of the key differences and the best situations for each tool.
When to Use sed
- Fast and simple text substitutions or deletions.
- Line-based transformations in files.
- Tasks that require minimal scripting overhead.
When to Use awk
- Working with structured data such as CSV or TSV files.
- Performing arithmetic operations together with text processing.
- Generating formatted reports from input data.
Example:
awk -F, '{print $1, $3}' data.csv
This command extracts and prints the first and third fields from a CSV file.
When to Use perl
- Complex text transformations involving advanced regular expressions.
- Combining text processing with logic or conditional rules.
- Writing short but powerful scripts.
Example:
perl -pe 's/(error)/WARNING: $1/' logfile.txt
This command adds a WARNING: prefix to lines that contain the word error.
Conclusion
Learning the sed command improves your ability to process and manipulate text efficiently in Linux. Its powerful capabilities and smooth integration into shell scripts make it an important tool for text-based editing and automation tasks.
