SED command in UNIX stands for
stream editor and it can perform lots of functions on file like searching,
find and replace, insertion or deletion. Though the most common use of SED command
in UNIX is for a substitution or for find and replace. By using SED you can edit
files even without opening it, which is a much quicker way to find and replace
something in the file, than first opening that file in VI Editor and then changing it.
In this SED command tutorial, we will see some practical examples of SED commands
in UNIX based systems e.g. Linux. I must say having a good grip on the find, grep, sort, vi editors and SED can
take you next level of UNIX and Linux working experience.
These are very powerful UNIX command and helps with a lot of different tasks in the server. By the way, we will use the following text file for our SED common example. As I have said before, the best way to learn any UNIX command is to use them in your day to day tasks, and a good example is a good start.
This file contains details of some popular android and iPhone smartphones, e.g. Model, company, price, etc, separated by a colon. You can also use any CSV file for this example.
These are very powerful UNIX command and helps with a lot of different tasks in the server. By the way, we will use the following text file for our SED common example. As I have said before, the best way to learn any UNIX command is to use them in your day to day tasks, and a good example is a good start.
This file contains details of some popular android and iPhone smartphones, e.g. Model, company, price, etc, separated by a colon. You can also use any CSV file for this example.
$ cat list-of-smartphones-2011.txt
Model:Company:Price:Camera:3G
IPhone4:Apple:1000$:Yes:Yes
Galaxy:Samsung:900$:Yes:Yes
Optimus:LG:800$:Yes:Yes
Sensation:HTC:400$:Yes:Yes
IPhone4S:Apple:1100$:Yes:Yes
N9:Nokia:400$:Yes:Yes
Few things to keep in mind about
SED in UNIX
1) SED is a powerful text stream editor. Can do insertion, deletion,
Search and replace(substitution).
2) SED command in UNIX supports regular expression which allows it
perform complex pattern matching.
SED command to Remove a Line from File in UNIX:
Using SED command to delete lines or remove lines is one of the popular use
of SED command in UNIX. most of the time you either delete a first-line or last line in unix. "d" flag is
used to delete lines in SED. Following the example of SED command in unix will show
how to delete first and the last line in UNIX based system e.g. Linux:
sed command to delete first line
$ sed '1d' list-of-smartphones-2011.txt
sed command to delete last line
$ sed '$d' list-of-smartphones-2011.txt
sed command to delete from line 1,4
$ sed '1,4d' list-of-smartphones-2011.txt
Sensation:HTC:400$:Yes:Yes
IPhone4S:Apple:1100$:Yes:Yes
N9:Nokia:400$:Yes:Yes
quote around "d" is not mandatory but its good practice and
increase readability.
SED Command to Remove blank lines
One of the common usage of SED command is to delete empty lines or
remove blank lines from files, without opening them. Unix sysadmin often use SED
command in Linux to clean up files by removing empty lines. Following example
of UNIX SED command will show you how to remove blank lines using SED in unix:
sed '/^$/d' list-of-smartphones-2011.txt
here text between /--/ is a regular expression for matching matching
empty lines i.e. “^$”. Since “^” denote
start of line and “$” denote end of line, “^$” means
empty lines.
I have inserted 3 blank line in our example file and now number of lines
are 10
$ cat list-of-smartphones-2011.txt | wc -l
10
$ sed '/^$/d' list-of-smartphones-2011.txt | wc -l
7
with this same technique we can remove spaces, or any lines which is matching to a string by using sed command in unix.
SED Command to Remove Matching String in UNIX
In this SED command example, we will learn how to remove lines which are
matching to particular text.
$ sed '/Apple/d' list-of-smartphones-2011.txt
Model:Company:Price:Camera:3G
Galaxy:Samsung:900$:Yes:Yes
Optimus:LG:800$:Yes:Yes
Sensation:HTC:400$:Yes:Yes
N9:Nokia:400$:Yes:Yes
Above sed command in unix has already removed all lines which matches string "Apple" in it. You can see that, there is no line which contains “Apple”, because they are already removed.
UNIX SED Command to Find and Replace text from File
One of my favorite use of SED command in UNIX is, to find and replace
strings or patterns without opening the file. Though you can also do find and
replace in VI, sed is much faster for such operation, especially if you are
dealing with large files which can take relatively long time to open in Vi
editor. Linux SED command supports regular expression which
allows you to perform sophisticated find and replace in UNIX. "s" flag
is used for substitution in sed and it works mostly like in VI.
SED command to find and replace:
sed 's/Apple/Microsoft/'
list-of-smartphones-2011.txt
In below example of unix SED command we will find "Apple" in line
and replace it with "Microsoft" string.
$ sed 's/Apple/Microsoft/' list-of-smartphones-2011.txt
Model:Company:Price:Camera:3G
IPhone4:Microsoft:1000$:Yes:Yes Apple
Galaxy:Samsung:900$:Yes:Yes
Optimus:LG:800$:Yes:Yes
Sensation:HTC:400$:Yes:Yes
IPhone4S:Microsoft:1100$:Yes:Yes
N9:Nokia:400$:Yes:Yes
An important thing is to note here is that it will only replace first occurrence of matching string on each line. Suppose if you have two matching string in one line , second will not be deleted. You can see this in second line of above SED command example, Apple is not replaced by Microsoft, to remove all occurrence of matching string use "g" at the end.
$ sed 's/Apple/Microsoft/g' list-of-smartphones-2011.txt
Model:Company:Price:Camera:3G
IPhone4:Microsoft:1000$:Yes:Yes
Microsoft
Galaxy:Samsung:900$:Yes:Yes
Optimus:LG:800$:Yes:Yes
Sensation:HTC:400$:Yes:Yes
IPhone4S:Microsoft:1100$:Yes:Yes
N9:Nokia:400$:Yes:Yes
That’s all on How to use SED command in UNIX based operating systems like Linux or Solaris. We have seen some of the most common examples of SED command, e.g. doing find and replace in file without opening it.
Related UNIX Command Tutorials
- 10 examples of find command in UNIX (examples)
- 10 examples of grep command in UNIX (examples)
- 10 examples of date command in Linux (examples)
- How to get IP address from hostname and vice-versa in Linux (command)
- 10 examples of xargs command in Linux (examples)
- 10 examples of tar command in UNIX (examples)
- 10 examples of Vim in UNIX (examples)
- How to create, update and delete soft link in UNIX (command)
- 5 examples of sort command in Linux (examples)
- 5 examples of kill command in Linux (examples)
- 10 examples of chmod command in UNIX (examples)
- 10 tips to work fast in UNIX? (tips)
Thanks for reading this article so far. If you like this article then please share with your friends and colleagues. If you have any questions or feedback then please drop a note.
What is the best way to find and replace from large files in UNIX? For example, you need to find all occurrences of word "INFO" and has to replace with "DEBUG", how will you do it and which is the fastest way to do that? This was the complete question asked on one UNIX interviews. My answer was by using SED command, e.g.
ReplyDeletesed s/INFO/DEBUG/g application.log
But I want to know from experts, whether that was right answer or not, with explanation?
sed 's/INFO/DEBUG/g' application.log
ReplyDeleteInsert a row as a first row into a large files in UNIX.
ReplyDeletecan i use sed without regex ?
ReplyDeleteHello @Phutapong, yes you can use sed without regular expression.
ReplyDelete