Paste any list and duplicates disappear instantly — first occurrence kept, original order preserved, with a live count of removed lines and one-click copy. Free, no signup, and your list never leaves your browser.
How to use it
- Paste your lines into the top box — one item per line.
- Adjust the comparison options if needed (case, whitespace, blank lines).
- Copy the deduplicated result; the counter shows exactly how many lines were removed.
Where duplicate lines pile up
Merged email lists that share subscribers. Keyword lists combined from several research exports. Log files where the same error repeats thousands of times. URL lists from multiple crawls. CSV columns pasted from different sheets. Inventory or SKU lists maintained by two people. Whenever two sources are concatenated, duplicates follow — and spotting them by eye stops working after about twenty lines.
Comparison options that match real lists
- Case-sensitive comparison (on by default): “Apple” and “apple” stay separate. Turn it off for name or email lists, where case differences are almost never meaningful — the first spelling encountered is kept.
- Ignore leading/trailing spaces (on by default): pasted lists routinely carry invisible trailing spaces, so “item “ and “item” are treated as the same line. Turn it off when whitespace is significant, such as indented config or code.
- Keep blank lines (on by default): blank lines usually mark section breaks, so they are never removed as duplicates of each other. Untick it to collapse runs of blank lines while deduplicating.
Unlike sort-based dedupers, nothing is reordered — the output is your list, minus the repeats. That matters when the list order encodes priority, chronology, or ranking.
Details worth knowing
Comparison is exact (after your chosen normalization), not fuzzy — “John Smith” and “John Smith” with a doubled inner space are different lines, since only leading and trailing whitespace is ignored. Windows and Unix line endings both work. The removal counter reads “Removed X duplicate lines of Y”, a quick audit that the tool agreed with your expectations.
Cleaning a list often takes more than one pass: fix broken wrapping first with the remove line breaks tool, normalize capitalization with the case converter, and check the final size with the word counter.
Private by design
There is no server behind this page — deduplication is JavaScript running on your device. That is why a 100,000-line log dedupes instantly, why it works offline once loaded, and why pasting customer emails or internal logs here is safe: nothing is uploaded, logged, or stored.