We use cookies

We use cookies and similar technologies to enhance your browsing experience, analyze site traffic, and personalize content and ads. By clicking "Accept", you consent to our use of cookies. Learn more in our Privacy Policy.

Cosmovex← All tools
Free · in your browser · no signup

Remove Duplicate Lines & Sort Text

Clean up lists of text fast — remove duplicate lines, sort A→Z or Z→A, reverse, trim, and strip empty lines, all in your browser.

0
Lines

Remove Duplicate Lines & Sort Text is a small utility for cleaning up line-based text. Paste in a list, log dump, set of email addresses, or any block where each line is its own record, and it will strip out duplicate lines, sort A-Z or Z-A, reverse the order, trim leading and trailing whitespace, and drop empty lines. You pick which operations to apply and see the result immediately.

It's built for the everyday text wrangling that doesn't justify writing a script: deduping a keyword list, alphabetizing a config block, or normalizing pasted data before it goes somewhere else. Everything runs in your browser, so the text you paste never leaves your machine and there's no upload, account, or size limit beyond what your browser can hold in memory. That makes it safe to use on internal lists or anything you'd rather not send to a server.

What each operation does

The tool treats your input as a list of lines split on newlines, then applies the operations you've enabled:

  • Remove duplicates keeps the first occurrence of each line and discards later repeats.
  • Sort A-Z / Z-A orders lines lexicographically (by character code), not by meaning.
  • Reverse flips the current line order top-to-bottom.
  • Trim removes leading and trailing spaces and tabs from each line.
  • Remove empty lines drops lines that are blank (or blank after trimming).

Order matters. A line that's " apple" with a leading space won't be seen as a duplicate of "apple" unless you trim first. The tool applies trimming before deduplication so whitespace differences don't hide real duplicates. Enable only what you need and the rest of your text passes through untouched.

A worked example

Say you've collected tags from a few spreadsheets and the paste looks like this:

  react
vue
React
vue

angular
  react

There are inconsistent spaces, a blank line, a casing difference, and two clear duplicates of vue and react. Turn on Trim, Remove empty lines, Remove duplicates, and Sort A-Z, and you get:

React
angular
react
vue

Note that React and react both survive because deduplication is case-sensitive. Sorting also placed React before angular because uppercase letters sort ahead of lowercase ones in character-code order. If you want a single canonical tag, lowercase the input first (or in your editor) before deduping.

Common use cases

This tool earns its keep on quick, throwaway cleanups:

  • Keyword and tag lists — dedupe and alphabetize SEO keywords, hashtags, or labels pulled from multiple sources.
  • Email and ID lists — collapse a messy paste of addresses or user IDs into a unique, sorted set before importing.
  • Log triage — sort or dedupe repeated log lines to see distinct error messages instead of thousands of identical rows.
  • Config and allowlists — alphabetize hostnames, IPs, or package names so diffs stay clean and entries are easy to find.
  • Writing and notes — strip empty lines from a rough outline or reverse a list to flip chronological order.

Because it's client-side, it's fine to use on lists that contain internal data, customer addresses, or anything you wouldn't paste into a third-party server.

Tips and gotchas

A few behaviors that trip people up:

  • Deduplication is exact and case-sensitive. Foo, foo, and FOO are three different lines. Normalize case beforehand if you want them merged.
  • Sorting is lexicographic, not natural. item10 sorts before item2 because it compares character by character. For human-friendly numeric ordering, zero-pad your numbers (item02, item10).
  • Trailing whitespace is invisible but real. A line ending in a space looks identical on screen but won't match its trimmed twin. Run Trim first when deduping pasted data.
  • Trailing newline. A final empty line at the end of your paste counts as a blank line; Remove empty lines will clear it.
  • Order of your operations is fixed, so you don't have to think about whether to trim before or after dedupe — the tool already trims first.

Why duplicates and whitespace hide so easily

Two lines are "the same" only if every character matches, and several invisible characters can break that match. The usual culprits are leading or trailing spaces, tab characters, and the difference between line-ending styles.

Windows tools often write lines ending in carriage-return plus line-feed (\r\n), while Unix tools use just line-feed (\n). If a stray \r rides along at the end of a line, that line won't equal its clean counterpart even though they print identically. This tool splits on common newline styles and trims surrounding whitespace before comparing, which neutralizes the most frequent reasons a "duplicate" slips through. If you ever see two seemingly identical lines that won't collapse, the cause is almost always a hidden character. Enabling Trim resolves the space-and-tab case; copying the text through a plain-text editor first can strip exotic invisibles like non-breaking spaces.

Tips

  • Run Trim before Remove duplicates so whitespace differences don't keep real duplicates apart.
  • Lowercase your list in your editor first if you want case-insensitive deduplication.
  • Zero-pad numbers (item02, item10) to get natural-looking numeric order from lexicographic sort.
  • Use Sort A-Z plus Remove duplicates together to produce a clean, ordered unique set in one pass.
  • Reverse after sorting A-Z is a quick way to get Z-A if you've already sorted ascending.
  • Safe for internal or sensitive lists since the text is processed in your browser and never uploaded.

How to use Remove Duplicate Lines & Sort Text

  1. 1Paste your lines of text into the box.
  2. 2Tap an operation: remove duplicates, sort, reverse, trim, or remove empty lines.
  3. 3Chain operations — each one transforms the text in place.
  4. 4Copy the cleaned result — nothing is uploaded.

Frequently asked questions

Is duplicate removal case-sensitive?

Yes. Apple and apple are treated as different lines and both are kept. Convert the text to a single case before deduping if you want them merged.

Which copy of a duplicate line is kept?

The first occurrence is kept and any later repeats are removed, so the relative order of the lines you keep is preserved.

Why is item10 sorting before item2?

Sorting is lexicographic (character by character), not numeric. '1' comes before '2', so item10 lands before item2. Zero-pad the numbers to fix the order.

Does it change the order of lines I'm not deduplicating?

Only if you enable a sort or reverse. With just dedupe, trim, and empty-line removal, the surviving lines stay in their original order.

Is there a limit on how much text I can paste?

There's no fixed cap. The practical limit is your browser's available memory, which comfortably handles lists of hundreds of thousands of lines.

Does my text get uploaded anywhere?

No. Everything runs locally in your browser. The text you paste is never sent to a server, so it's safe for internal or private data.

Two lines look identical but won't dedupe. Why?

They differ in a character you can't see, usually trailing whitespace, a tab, or a stray carriage return. Enable Trim, which strips surrounding whitespace before comparing.

Can I sort and remove duplicates at the same time?

Yes. Enable both and you'll get a deduplicated, sorted result in one step. Trimming is applied first so whitespace doesn't interfere.

← All toolsRead our guides →