Remove Duplicate Lines
Remove duplicate lines from text, optionally ignoring case and whitespace, or list only the lines that repeat.
0 characters · 1 line
- Lines
- 0
- Removed
- 0
- Result lines
- 0
- Repeated lines
- 0
Options only change how lines are compared; the result keeps each line as written.
How to use
- Paste a list into “Your text”, one item per line, or click “Example” to try it out.
- The result appears instantly on the right (below on phones), along with the number of lines, how many were removed and how many lines repeated.
- Turn on “Ignore case” or “Ignore leading and trailing spaces” if needed, and choose whether the first or last occurrence is kept.
- To find out what repeats, set “Show” to “Only repeated lines”; “Only lines that appear once” keeps just the items that are not repeated.
- Click “Copy” or “Download” to grab the result.
FAQ
Is my data uploaded?
No. Lines are compared in your browser and never leave your device, even for lists with hundreds of thousands of lines.
Does “Ignore leading and trailing spaces” remove spaces from the result?
No. The options only decide which lines count as duplicates; every line in the result is kept exactly as written. To clean up spaces as well, run the text through the Whitespace Cleaner first.
What is the difference between keeping the first and the last occurrence?
The same lines remain, but in different places: with “First occurrence” a repeated line stays where it first appears, with “Last occurrence” where it last appears. When ignoring case, the result uses that occurrence’s spelling, such as Apple or APPLE.
How are empty lines handled?
“Treat like other lines” keeps only the first empty line. “Keep all” leaves every empty line in place, which preserves paragraph breaks. “Remove all” deletes them. Lines with only spaces or tabs count as empty.
Why are two lines that look the same not treated as duplicates?
Usually because of invisible trailing spaces or tabs, or full-width versus half-width characters (Ａ and A). Try “Ignore leading and trailing spaces”, or remove non-breaking spaces and zero-width characters with the Whitespace Cleaner.
Are line breaks changed?
The result uses the first kind of line break found in the input (Windows CRLF or macOS/Linux LF), and a line break after the last line is kept.
Related tools
Sort Lines
Sort lines alphabetically, naturally (file2 before file10), by number or length, reverse them or shuffle them randomly.Whitespace Cleaner
Trim trailing spaces, collapse blank lines and repeated spaces, remove zero-width and non-breaking spaces, convert tabs and line endings.Find & Replace
Find and replace text in bulk with case and whole-word options or regular expressions with $1 groups, with matches highlighted.Text Diff
Compare two texts side by side or inline, with changed words highlighted, ignore whitespace or case, and copy a unified patch.