Monday, July 26, 2021

10 Examples of SED Command in UNIX and Linux

SED command in UNIX stands for stream editor and it can perform lots of functions on file like searching, find and replace, insertion or deletion. Though the most common use of SED command in UNIX is for a substitution or for find and replace. By using SED you can edit files even without opening it, which is a much quicker way to find and replace something in the file, than first opening that file in VI Editor and then changing it. In this SED command tutorial, we will see some practical examples of SED commands in UNIX based systems e.g. Linux. I must say having a good grip on the find, grep, sort, vi editors and SED can take you next level of UNIX and Linux working experience.

These are very powerful UNIX command and helps with a lot of different tasks in the server. By the way, we will use the following text file for our SED common example. As I have said before, the best way to learn any UNIX command is to use them in your day to day tasks, and a good example is a good start. 

This file contains details of some popular android and iPhone smartphones, e.g. Model, company, price, etc, separated by a colon. You can also use any CSV file for this example.

$ cat list-of-smartphones-2011.txt
Model:Company:Price:Camera:3G
IPhone4:Apple:1000$:Yes:Yes
Galaxy:Samsung:900$:Yes:Yes
Optimus:LG:800$:Yes:Yes
Sensation:HTC:400$:Yes:Yes
IPhone4S:Apple:1100$:Yes:Yes
N9:Nokia:400$:Yes:Yes

Few things to keep in mind about  SED in UNIX

1) SED is a powerful text stream editor. Can do insertion, deletion, Search and replace(substitution).

2) SED command in UNIX supports regular expression which allows it perform complex pattern matching.


SED command to Remove a Line from File in UNIX:

SED command in UNIX to search string in fileUsing SED command to delete lines or remove lines is one of the popular use of SED command in UNIX. most of the time you either delete a first-line or last line in unix. "d" flag is used to delete lines in SED. Following the example of SED command in unix will show how to delete first and the last line in UNIX based system e.g. Linux:

sed command to delete first line


$ sed '1d' list-of-smartphones-2011.txt

sed command to delete last line


$ sed '$d' list-of-smartphones-2011.txt

sed command to delete from line 1,4


$ sed '1,4d' list-of-smartphones-2011.txt
Sensation:HTC:400$:Yes:Yes
IPhone4S:Apple:1100$:Yes:Yes
N9:Nokia:400$:Yes:Yes

quote around "d" is not mandatory but its good practice and increase readability.

SED Command to Remove blank lines

One of the common usage of SED command is to delete empty lines or remove blank lines from files, without opening them. Unix sysadmin often use SED command in Linux to clean up files by removing empty lines. Following example of UNIX SED command will show you how to remove blank lines using SED in unix:

sed '/^$/d' list-of-smartphones-2011.txt

here text between /--/ is a regular expression for matching matching empty lines i.e. “^$”. Since “^” denote start of line and “$” denote end of line, “^$” means empty lines.

I have inserted 3 blank line in our example file and now number of lines are 10

$ cat list-of-smartphones-2011.txt | wc -l
10

$ sed '/^$/d' list-of-smartphones-2011.txt | wc -l
7

with this same technique we can remove spaces, or any lines which is matching to a string by using sed command in unix.

SED Command to Remove Matching String in UNIX

In this SED command example, we will learn how to remove lines which are matching to particular text.

$ sed '/Apple/d' list-of-smartphones-2011.txt
Model:Company:Price:Camera:3G
Galaxy:Samsung:900$:Yes:Yes
Optimus:LG:800$:Yes:Yes
Sensation:HTC:400$:Yes:Yes
N9:Nokia:400$:Yes:Yes

Above sed command in unix has already removed all lines which matches string "Apple" in it. You can see that, there is no line which contains “Apple”, because they are already removed.

UNIX SED Command to Find and Replace text from File

One of my favorite use of SED command in UNIX is, to find and replace strings or patterns without opening the file. Though you can also do find and replace in VI, sed is much faster for such operation, especially if you are dealing with large files which can take relatively long time to open in Vi editor. Linux SED command supports regular expression which allows you to perform sophisticated find and replace in UNIX. "s" flag is used for substitution in sed and it works mostly like in VI.

SED command to find and replace:


sed 's/Apple/Microsoft/' list-of-smartphones-2011.txt

In below example of unix SED command we will find "Apple" in line and replace it with "Microsoft" string.

$ sed 's/Apple/Microsoft/' list-of-smartphones-2011.txt
Model:Company:Price:Camera:3G
IPhone4:Microsoft:1000$:Yes:Yes Apple
Galaxy:Samsung:900$:Yes:Yes
Optimus:LG:800$:Yes:Yes
Sensation:HTC:400$:Yes:Yes
IPhone4S:Microsoft:1100$:Yes:Yes
N9:Nokia:400$:Yes:Yes

An important thing is to note here is that it will only replace first occurrence of matching string on each line. Suppose if you have two matching string in one line , second will not be deleted. You can see this in second line of above SED command example, Apple is not replaced by Microsoft, to remove all occurrence of matching string use "g" at the end.

$ sed 's/Apple/Microsoft/g' list-of-smartphones-2011.txt
Model:Company:Price:Camera:3G
IPhone4:Microsoft:1000$:Yes:Yes Microsoft
Galaxy:Samsung:900$:Yes:Yes
Optimus:LG:800$:Yes:Yes
Sensation:HTC:400$:Yes:Yes
IPhone4S:Microsoft:1100$:Yes:Yes
N9:Nokia:400$:Yes:Yes


That’s all on How to use SED command in UNIX based operating systems like Linux or Solaris. We have seen some of the most common examples of SED command, e.g. doing find and replace in file without opening it.


Related UNIX Command Tutorials
  • 10 examples of find command in UNIX (examples)
  • 10 examples of grep command in UNIX (examples)
  • 10 examples of date command in Linux (examples)
  • How to get IP address from hostname and vice-versa in Linux (command)
  • 10 examples of xargs command in Linux (examples)
  • 10 examples of tar command in UNIX (examples)
  • 10 examples of Vim in UNIX (examples)
  • How to create, update and delete soft link in UNIX (command)
  • 5 examples of sort command in Linux (examples)
  • 5 examples of kill command in Linux (examples)
  • 10 examples of chmod command in UNIX (examples)
  • 10 tips to work fast in UNIX? (tips)

Thanks for reading this article so far. If you like this article then please share with your friends and colleagues. If you have any questions or feedback then please drop a note.

5 comments:

  1. What is the best way to find and replace from large files in UNIX? For example, you need to find all occurrences of word "INFO" and has to replace with "DEBUG", how will you do it and which is the fastest way to do that? This was the complete question asked on one UNIX interviews. My answer was by using SED command, e.g.

    sed s/INFO/DEBUG/g application.log

    But I want to know from experts, whether that was right answer or not, with explanation?

    ReplyDelete
  2. sed 's/INFO/DEBUG/g' application.log

    ReplyDelete
  3. Insert a row as a first row into a large files in UNIX.

    ReplyDelete
  4. Hello @Phutapong, yes you can use sed without regular expression.

    ReplyDelete