What are sed and awk, and when would you use each? Can you provide practical examples?

Question

Accepted Answer

Both sed and awk are powerful text processing tools, but they excel at different tasks.

Sed, the stream editor, is best for simple text substitutions and line-based operations. Think of it as find-and-replace on steroids.

# Replace text
sed 's/old/new/g' file.txt

# Delete lines matching a pattern
sed '/DEBUG/d' logfile.txt

# Replace in place with backup
sed -i.bak 's/localhost/server.example.com/g' config.txt

# Delete first 10 lines
sed '1,10d' file.txt

# Print only lines 5 to 10
sed -n '5,10p' file.txt

Awk is a full programming language for text processing. It excels at working with columnar data and complex logic.

# Print specific columns
awk '{print $1, $3}' file.txt

# Sum a column
awk '{sum += $2} END {print sum}' numbers.txt

# Filter by condition
awk '$3 > 100 {print $1}' data.txt

# Process CSV with custom separator
awk -F',' '{print $2}' data.csv

# Print with formatting
awk '{printf "%-20s %10d
", $1, $2}' file.txt

When to use which? Use sed for simple substitutions and when you need to modify files in place. Use awk when dealing with columns, doing arithmetic, or needing programming logic.

A real-world example combining both:

# Parse Apache access logs: show top 10 IPs
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -10

# Replace version in config files
sed -i 's/VERSION=.*/VERSION=2.0.0/' config.env

For complex tasks, awk is usually more readable than chained sed commands.