Skip to content

uniq Command Cheat Sheet

uniq reports or omits repeated lines. Important: uniq only detects adjacent duplicate lines. You must sort the input first.

Synopsis

uniq [OPTION]... [INPUT [OUTPUT]]

Basic Usage

Remove Duplicates

sort file.txt | uniq

Count Occurrences (-c)

Prefix lines by the number of occurrences.

sort access.log | uniq -c
# Output:
#   20 192.168.1.1
#    5 192.168.1.50

Sort by Count

Common pattern to find "top" items.

sort access.log | uniq -c | sort -rn
Sorts by frequency (descending).

Filtering

Only Print Duplicates (-d)

Show only lines that appear more than once.

sort file.txt | uniq -d

Only Print Unique Lines (-u)

Show only lines that appear exactly once.

sort file.txt | uniq -u

Advanced Options

Skip Fields (-f)

Ignore the first N fields when comparing.

# Ignore timestamp (field 1), compare rest
uniq -f 1 log.txt

Skip Characters (-s)

Ignore the first N characters.

uniq -s 10 log.txt

Ignore Case (-i)

sort file.txt | uniq -i
Treats "Apple" and "apple" as duplicates.

Common Use Cases

Find Most Frequent Command

history | awk '{print $2}' | sort | uniq -c | sort -rn | head -n 10

Clean Up Lists

sort unsorted_list.txt | uniq > unique_list.txt