Linux sed and awk Guide

Complete beginner-friendly guide to sed and awk on Linux, covering Arch Linux, CachyOS, and other distributions including text processing, pattern matching, and data manipulation.

---

Table of Contents

1. Understanding sed and awk
2. sed Basics
3. sed Advanced
4. awk Basics
5. awk Advanced
6. Practical Examples
7. Troubleshooting

---

Understanding sed and awk

What is sed?

sed (stream editor) edits text streams.

Uses:

• Find and replace: Replace text in files
• Delete lines: Remove specific lines
• Insert text: Add text to files
• Batch editing: Edit multiple files

What is awk?

awk is a pattern scanning and processing language.

Uses:

• Data extraction: Extract fields from text
• Text processing: Process structured data
• Report generation: Generate formatted reports
• Data analysis: Analyze text data

---

sed Basics

Install sed

sed is pre-installed:

# Check version
sed --version

# Usually included in base system

Basic Substitution

Replace text:

# Replace first occurrence
sed 's/old/new/' file.txt

# Replace all occurrences
sed 's/old/new/g' file.txt

# Case insensitive
sed 's/old/new/gi' file.txt

In-Place Editing

Edit file directly:

# Edit file in place
sed -i 's/old/new/g' file.txt

# Backup original
sed -i.bak 's/old/new/g' file.txt

---

sed Advanced

Delete Lines

Remove lines:

# Delete line 5
sed '5d' file.txt

# Delete lines 2-5
sed '2,5d' file.txt

# Delete lines matching pattern
sed '/pattern/d' file.txt

Insert/Append

Add text:

# Insert before line 3
sed '3i\New line' file.txt

# Append after line 3
sed '3a\New line' file.txt

---

awk Basics

Install awk

awk is pre-installed:

# Check version
awk --version

# Usually included in base system

Print Fields

Extract columns:

# Print first field
awk '{print $1}' file.txt

# Print multiple fields
awk '{print $1, $3}' file.txt

# Print all fields
awk '{print $0}' file.txt

Field Separator

Custom separator:

# Use comma as separator
awk -F',' '{print $1}' file.txt

# Use colon
awk -F':' '{print $1}' /etc/passwd

---

awk Advanced

Pattern Matching

Match patterns:

# Print lines matching pattern
awk '/pattern/ {print}' file.txt

# Print lines where field matches
awk '$1 == "value" {print}' file.txt

Calculations

Perform calculations:

# Sum column
awk '{sum += $1} END {print sum}' file.txt

# Average
awk '{sum += $1; count++} END {print sum/count}' file.txt

---

Practical Examples

sed Examples

Common tasks:

# Remove blank lines
sed '/^$/d' file.txt

# Remove leading spaces
sed 's/^[ \t]*//' file.txt

# Remove trailing spaces
sed 's/[ \t]*$//' file.txt

awk Examples

Common tasks:

# Print lines longer than 80 characters
awk 'length > 80' file.txt

# Count lines
awk 'END {print NR}' file.txt

# Print unique values in column
awk '{print $1}' file.txt | sort -u

---

Troubleshooting

sed Not Working

Check syntax:

# Test without modifying
sed 's/old/new/g' file.txt

# Check for errors
sed -n 's/old/new/p' file.txt

awk Errors

Debug awk:

# Check field numbers
awk '{print NF, $0}' file.txt

# Check field separator
awk -F',' '{print NF}' file.txt

---

Summary

This guide covered sed and awk installation, basic usage, and advanced features for Arch Linux, CachyOS, and other distributions.

---

Next Steps

• Grep Command Guide - Text searching
• Text Editors - Text editors
• sed Documentation: man sed
• awk Documentation: man awk

---

This guide covers Arch Linux, CachyOS, and other Linux distributions. For distribution-specific details, refer to your distribution's documentation.