TIL - xargs tricks!

Hello everyone!  Today is the first day of  #vDM30in30 (For details, check out http://discoposse.com/vdm30in30/).  I was inspired to participate by rnelson0 in the #voxpupuli channel on freenode.

I have been using xargs for a while, but I learned something new!  Today I Learned (TIL) about the --replace-string or -I argument.

What IS xargs?  Straight from the xargs man page

 xargs reads items from the standard input, delimited by blanks (which can be protected with double or single quotes or a back‐slash) or newlines, and executes the command (default is /bin/echo) one or more times with any initial-arguments followed by items read from standard input.   

What can xargs do for us?  If you have the result of a command that you wish to pass in as arguments to another command, xargs is a more natural way to flow from one command to the other using a pipe operator ( | ) instead of embedding the results in a for loop.


A good set of xargs examples can be found here.  The feature that I didn't know about was the -I, or --replace-str argument. From the man page...

       -I replace-str
              Replace occurrences of replace-str in the initial-arguments with names read from standard input.  Also, unquoted blanks do not terminate input items; instead  the  separator is the newline character.  Implies -x and -L 1.

What that allows you to do is to use the value from stdin in an arbitrary (and possibly repeated) position in your command.

If you wanted to create a backup file of all files in a directory except for a few that you wish to exclude, you can do something like the following..

$ ls

a  b  c  dime  penny  quarter

$ mkdir archive

$ ls | grep -v -e dime -e penny -e archive # This will print the list of files, excluding dime, penny, and the archive directory

a

b

c

quarter

$ ls | grep -v -e dime -e penny -e archive | xargs -I{} cp {} archive/{}.bak

$ ls archive/

a.bak  b.bak  c.bak  quarter.bak

You specify the symbol for the -I argument, {} in this case, which will be replaced by the results from the previous command.

Where else have I used xargs lately?  If you want to find all of the files of a particular type that contain a particular string, and edit them one by one:

$ find ./ -name '*.pp' | grep -v params | xargs grep  params -l | xargs -n 1 vi

What that will do is find all files that match the pattern of *.pp, remove any files that contain params in the name (grep -v is an inverse match), xargs will pass all of the listed files into the grep command,  -l prints the name of any matching files, then the final xargs will send each file, one at a time, into vi for editing.


Before someone learns about xargs, they may end up embedding a bash for loop into their commands, something like

$ for i in $(ls | grep -v archive); do cp $i archive/$i.bak; done

With a short example like that, it may not be to hard to follow, but as your examples get much longer, it can get hard to tell which parts of the command belong where.  It is usually much easier to read left to right.

From left to right using xargs..

$ ls | grep -v archive | xargs -I{} cp {} archive/{}.bak