The utility Sed, stream editor, can be found on almost all unix-based systems. Sed takes a steam of text and, as the name suggests, edits it according to some instructions you have given it. It is a very flexible tool and can be very useful when using the commandline or in a shell script.

As a classic Hello World example do

sed 's/.*/Hello World/g'

Then press enter a few times and see sed replace the lines with Hello World.

This article contains some (fairly artificial) examples in order to demonstrate the features of sed. There are far more features than are in the scope of this article; I suggest checking out the man page if you are curious for more.

Uses

There is nothing that sed can do that is unique. Other tools such as grep awk and perl can be used for similar things to name just a few. Sed provides a style of working that some people like the most.

A simple use for sed is the situation when you are running some software that produces a lot of output but you only care about certain messages and do not want to miss them amidst the output you don't care about, maybe these messages appear in blocks starting with a line WARNING: and ending with an empty line. In this case you can do

./some_software | sed -n '/WARNING:/, /^$/p'

The -n option tells sed not to print the output of ./some_software unless explicitly told to do so by the print instruction, written p, this comes at the end of our sed command. Before it there are two regular expressions in forward slashes separated by a comma. This tells sed that you want to print in the range of lines between any that match those two regular expressions.

Another useful command that can go in place of the print command is the delete line command which is written d. Now when your boss is watching you can show him that it runs without any warnings by removing all the lines containing the string WARNING with

./some_software | sed '/WARNING/d'

Notice that the -n option is not there because we want sed to print all the text that is being output by ./some_software unless it matches WARNING, in which case delete that line from the stream.

Yet another usecase is taking the output of one command and changing parts of it; This is called a substitution and is done with the s/. For example, maybe you are having problems with a piece of software. You want to ask for help on an online forum but need to be careful not to post personal information which is included in the command output. Maybe it gives away that your name is Joe Bloggs so you do

./some_software | sed 's/Joe Blogs/<Name Redacted>/g'

For a more involved example, imagine you want to get the IP address of the wlan0 interface on your machine and put it in a variable in a shell script. ifconfig might ouput something that looks like:

wlan0     Link encap:Ethernet  HWaddr 08:11:96:05:6b:6c  
          inet addr:192.168.1.122  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a11:96ff:fe05:6b6c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:67062 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18075 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:21572026 (20.5 MiB)  TX bytes:3448315 (3.2 MiB)

We want to extract the address 192.168.1.122. By eyeballing it we can see that we want the line after it says wlan0, then we want to match the regex inet addr:[^ ]* and we want to print that. You could use the following line

wlan0ip=$(ifconfig | sed -n '/wlan0/{n; s/.*inet addr:\([^ ]*\).*/\1/p; q}')

Notice that in this command after the /wlan0/ to match there is a curley brace. This is a block like in other programming languages because all the commands between the braces will be executed when there is a matching line. Commands are separated by semicolons. The n command simply tells sed to look at the next line. The next bit is a little fiddly. We only want to output the IP address, not the whole line so instead of doing a normal print p I used the s/.*(<some regex>).*/\1/p trick. This is a fairy standard trick which replaces the entire line with just the match before printing it in order to only print what you want. Finally the q tells sed to quit because you have found what you want and do not need to process the remainder of the output.

Limitations

Sed is bad when you need a line that comes before a matching line. Sed never goes backwards so by the time it finds the matching line it has already thrown away.