Daily Hacks

Hack #21 -> Sed Basics – Find and Replace Using RegEx

This hack explains how to use sed substitute command “s”.

The `s’ command is probably the most important in `sed’ and has a lot

of different options.

The `s’ command attempts to match the pattern space against the

supplied REGEXP; if the match is successful, then that portion of the

pattern space which was matched is replaced with REPLACEMENT.

Syntax:

 

#sed ‘ADDRESSs/REGEXP/REPLACEMENT/FLAGS’ filename

#sed ‘PATTERNs/REGEXP/REPLACEMENT/FLAGS’ filename

 

  • s is substitute command
  • / is a delimiter
  • REGEXP is regular expression to match
  • REPLACEMENT is a value to replace

FLAGS can be any of the following :

  • g Replace all the instance of REGEXP with REPLACEMENT
  • n Could be any number,replace nth instance of the REGEXP with

REPLACEMENT.

  • p If substitution was made, then prints the new pattern space.
  • i match REGEXP in a case-insensitive manner.
  • w file If substitution was made, write out the result to the given

file.

  • We can use different delimiters ( one of @ % ; : ) instead of /

Let us first create thegeekstuff.txt file that will be used in all the

examples mentioned below.

 

$ cat thegeekstuff.txt

 

# Instruction Guides

  1. Linux Sysadmin, Linux Scripting etc.
  2. Databases – Oracle, mySQL etc.
  3. Security (Firewall, Network, Online Security etc)
  4. Storage in Linux
  5. Productivity (Too many technologies to explore, not

much time available)

# Additional FAQS

  1. Windows- Sysadmin, reboot etc.

 

Substitute Word “Linux” to “Linux-Unix” Using sed s//

In the example below, in the output line “1. Linux-Unix Sysadmin, Linux

Scripting etc” only first Linux is replaced by Linux-Unix. If no flags are

specified the first match of line is replaced.

 

$ sed ‘s/Linux/Linux-Unix/’ thegeekstuff.txt

 

# Instruction Guides

  1. Linux-Unix Sysadmin, Linux Scripting etc.
  2. Databases – Oracle, mySQL etc.
  3. Security (Firewall, Network, Online Security etc)
  4. Storage in Linux-Unix
  5. Productivity (Too many technologies to explore, not

much time available)

# Additional FAQS

  1. Windows- Sysadmin, reboot etc.

 

Substitute all Appearances of a Word Using sed s//g

The below sed command replaces all occurrences of Linux to Linux-Unix

using global substitution flag “g”.

 

$ sed ‘s/Linux/Linux-Unix/g’ thegeekstuff.txt

 

# Instruction Guides

  1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
  2. Databases – Oracle, mySQL etc.
  3. Security (Firewall, Network, Online Security etc)
  4. Storage in Linux-Unix
  5. Productivity (Too many technologies to explore, not

much time available)

# Additional FAQS

  1. Windows- Sysadmin, reboot etc.

 

Substitute Only 2nd Occurrence of a Word Using sed s//2

In the example below, in the output line “1. Linux Sysadmin, Linux-Unix

Scripting etc.” only 2nd occurrence of Linux is replaced by Linux-Unix.

 

$ sed ‘s/Linux/Linux-Unix/2’ thegeekstuff.txt

 

# Instruction Guides

  1. Linux Sysadmin, Linux-Unix Scripting etc.
  2. Databases – Oracle, mySQL etc.
  3. Security (Firewall, Network, Online Security etc)
  4. Storage in Linux
  5. Productivity (Too many technologies to explore, not

much time available)

# Additional FAQS

  1. Windows- Sysadmin, reboot etc.
  2. Write Changes to a File and Print the Changes Using sed

s//gpw

The example below has substitution with three flags. It substitutes all

the occurrence of Linux to Linux-Unix and prints the substituted output

as well as written the same to the given the file.

 

$ sed -n ‘s/Linux/Linux-Unix/gpw output’ thegeekstuff.txt

 

  1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
  2. Storage in Linux-Unix

 

$ cat output

 

  1. Linux-Unix Sysadmin, Linux-Unix Scripting etc.
  2. Storage in Linux-Unix
  3. Substitute Only When the Line Matches with the Pattern

Using sed

In this example, if the line matches with the pattern “-”, then it replaces

all the characters from “-” with the empty.

 

$ sed ‘/\-/s/\-.*//g’ thegeekstuff.txt

 

# Instruction Guides

  1. Linux Sysadmin, Linux Scripting etc.
  2. Databases
  3. Security (Firewall, Network, Online Security etc)
  4. Storage in Linux
  5. Productivity (Too many technologies to explore, not

much time available)

# Additional FAQS

  1. Windows

 

Delete Last X Number of Characters From Each Line Using

sed

 

This sed example deletes last 3 characters from each line.

$ sed ‘s/…$//’ thegeekstuff.txt

 

# Instruction Gui

  1. Linux Sysadmin, Linux Scripting e
  2. Databases – Oracle, mySQL e
  3. Security (Firewall, Network, Online Security e
  4. Storage in Li
  5. Productivity (Too many technologies to explore, not

much time availab

# Additional F

  1. Windows- Sysadmin, reboot e

 

Eliminate Comments Using sed

Delete all the comment lines from a file as shown below using sed

command.

$ sed -e ‘s/#.*//’ thegeekstuff.txt

 

  1. Linux Sysadmin, Linux Scripting etc.
  2. Databases – Oracle, mySQL etc.
  3. Security (Firewall, Network, Online Security etc)
  4. Storage in Linux
  5. Productivity (Too many technologies to explore, not

much time available)

  1. Windows- Sysadmin, reboot etc.
  2. Eliminate Comments and Empty Lines Using sed

 

In the following example, there are two commands separated by ‘;’

First command replaces the lines starting with the # to the blank lines

Second command deletes the empty lines.

 

$ sed -e ‘s/#.*//;/^$/d’ thegeekstuff.txt

 

  1. Linux Sysadmin, Linux Scripting etc.
  2. Databases – Oracle, mySQL etc.
  3. Security (Firewall, Network, Online Security etc)
  4. Storage in Linux
  5. Productivity (Too many technologies to explore, not

much time available)

  1. Windows- Sysadmin, reboot etc.
  2. Convert DOS newlines (CR/LF) to Unix format Using sed

 

Eliminate HTML Tags from file Using sed

 

In this example, the regular expression given in the sed command

matches the html tags and replaces with the empty.

$ sed -e ‘s/<[^>]*>//g’

 

This <b> is </b> an <i>example</i>.

This is an example

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s