EasyWordCounter

Remove Duplicate Lines

Paste a list to take out every repeated line, keeping the order of what is left. See exactly how many duplicates were removed.

When a line repeats, keep the

Without duplicates

Your list without duplicates appears here.

Runs in your browser. Your text is never uploaded.

How it works

The tool reads your text one line at a time. The first time it sees a line, it keeps it; any later line that matches is removed. The options decide what counts as a match:

  • Match case: off by default, so “Apple” and “apple” are duplicates.
  • Trim spaces: on by default, so a line with a stray space at the end still matches. Trimmed lines are also tidied in the result.
  • Remove empty lines: off by default, which leaves blank lines exactly where they were.
  • Keep the first or last copy: decides where a repeated line ends up.

Examples

You paste:

anna@example.com
ben@example.com
Anna@example.com
cara@example.com
ben@example.com

You get:

anna@example.com
ben@example.com
cara@example.com

Two duplicates removed: the capitalised “Anna” and the second “ben”, which had a trailing space. Common uses are cleaning a mailing list before an import, merging keyword lists, and de-duplicating tags or a list of URLs.

Frequently asked questions

Does removing duplicates change the order of my list?
No. Lines stay in the order you pasted them, and only the repeats are taken out. To sort the list as well, use alphabetize a list, which can also remove duplicates.
What is the difference between keeping the first and last copy?
It decides where a repeated line ends up. Keeping the first copy leaves it where it first appeared. Keeping the last moves it to where it last appeared, which is useful for logs and lists where later entries are more up to date.
Are “Apple” and “apple” duplicates?
By default, yes: case is ignored, and the first spelling is kept. Turn on “Match case” to treat them as different lines.
How many lines can it handle?
Hundreds of thousands. Each line is checked against the lines already seen in a single pass, so even a long export from a spreadsheet takes a moment at most. It all happens in your browser, so email lists and customer data are never uploaded.

Related tools

Alphabetize a List

Sort, reverse or shuffle lines

Text Cleaner

Strip the mess out of pasted text

Compare Text

See what changed between two versions

Remove Line Breaks

Join broken lines back into paragraphs

Related reading