Daily Hacks

Hack #15 -> Uniq Command

Uniq command is mostly used in combination with sort command, as

uniq removes duplicates only from a sorted file. i.e In order for uniq to

work, all the duplicate entries should be in the adjacent lines. The

following are some common examples.

  1. When you have an employee file with duplicate entries, you can do

the following to remove duplicates.

$ sort namesd.txt | uniq

$ sort –u namesd.txt

  1. If you want to know how many lines are duplicates, do the following.

The first field in the following examples indicates how many duplicates

where found for that particular line. So, in this example the lines

beginning with Alex and Emma were found twice in the namesd.txt file.

$ sort namesd.txt | uniq –c

2 Alex Jason:200:Sales

2 Emma Thomas:100:Marketing

1 Madison Randy:300:Product Development

1 Nisha Singh:500:Sales

1 Sanjay Gupta:400:Support

  1. The following displays only the entries that are duplicates.

$ sort namesd.txt | uniq –cd

2 Alex Jason:200:Sales

2 Emma Thomas:100:Marketing

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s