Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
A Unix-like operating system is one that behaves in a manner similar to a Unix system, while not necessarily conforming to or being certified to any version of the Single UNIX Specification....
operating systems used to build and execute command lines from standard input
Standard streams
In Unix and Unix-like operating systems , as well as certain programming language interfaces, the standard streams are preconnected input and output channels between a computer program and its environment when it begins execution...
The Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
before version 2.6.23, arbitrarily long lists of parameters could not be passed to a command, so xargs breaks the list of arguments into sublists small enough to be acceptable.
For example, commands like:
rm /path/*
or
rm `find /path -type f`
will fail with an error message of "Argument list too long" if there are too many files in /path.
However the version below (functionally equivalent to rm `find /path -type f`) will not fail:
find /path -type f -print0 | xargs -0 rm
In Unix-like and some other operating systems, find is a command-line utility that searches through one or more directory trees of a file system, locates files based on some user-specified criteria and applies a user-specified action on each matched file...
feeds the input of xargs with a long list of file names. xargs then splits this list into sublists and calls rm once for every sublist.
The previous example is more efficient than this functionally equivalent version which calls rm once for every single file:
find /path -type f -exec rm '{}' \;
Note however that with modern versions of find, the following variant does the same thing as the xargs version:
find /path -type f -exec rm '{}' +
xargs often covers the same functionality as the backquote (`) feature of many shells
Unix shell
A Unix shell is a command-line interpreter or shell that provides a traditional user interface for the Unix operating system and for Unix-like systems...
, but is more flexible and often also safer, especially if there are blanks or special characters in the input. It is a good companion for commands that output long lists of files like find
Find
In Unix-like and some other operating systems, find is a command-line utility that searches through one or more directory trees of a file system, locates files based on some user-specified criteria and applies a user-specified action on each matched file...
locate is a Unix utility first created in 1983 used to find files on filesystems. It searches through a prebuilt database of files generated by updatedb or a daemon and compressed using incremental encoding...
grep is a command-line text-search utility originally written for Unix. The name comes from the ed command g/re/p...
, but only if you use -0, since xargs without -0 deals badly with file names containing ', " and space. GNU Parallel
Parallel (software)
GNU parallel is a command-line driven utility for Linux or other Unix-like operating systems which allows the user to execute shell scripts in parallel. GNU parallel is a free software, written in Perl...
In Unix-like and some other operating systems, find is a command-line utility that searches through one or more directory trees of a file system, locates files based on some user-specified criteria and applies a user-specified action on each matched file...
locate is a Unix utility first created in 1983 used to find files on filesystems. It searches through a prebuilt database of files generated by updatedb or a daemon and compressed using incremental encoding...
grep is a command-line text-search utility originally written for Unix. The name comes from the ed command g/re/p...
if file names may contain ', " and space (newline still requires -0).
Examples
find . -name "*.foo" | xargs grep bar
The above is equivalent to:
grep bar `find . -name "*.foo"`
Note that the above command uses backticks (`), not single quotes ('). It searches all files in the current directory
Directory (file systems)
In computing, a folder, directory, catalog, or drawer, is a virtual container originally derived from an earlier Object-oriented programming concept by the same name within a digital file system, in which groups of computer files and other folders can be kept and organized.A typical file system may...
and its subdirectories which end in .foo for occurrences of the string
String (computer science)
In formal languages, which are used in mathematical logic and theoretical computer science, a string is a finite sequence of symbols that are chosen from a set or alphabet....
bar. These commands will not work as expected if there are whitespace characters, including newline
Newline
In computing, a newline, also known as a line break or end-of-line marker, is a special character or sequence of characters signifying the end of a line of text. The name comes from the fact that the next character after the newline will appear on a new line—that is, on the next line below the...
s, in the filenames. In order to avoid this limitation one may use:
find . -name "*.foo" -print0 | xargs -0 grep bar
The above command uses GNU specific extensions to find and xargs to separate filenames using the null character
Null character
The null character , abbreviated NUL, is a control character with the value zero.It is present in many character sets, including ISO/IEC 646 , the C0 control code, the Universal Character Set , and EBCDIC...
The above command is similar to the former one, but launches the vi
Vi
vi is a screen-oriented text editor originally created for the Unix operating system. The portable subset of the behavior of vi and programs based on it, and the ex editor language supported within these programs, is described by the Single Unix Specification and POSIX.The original code for vi...
editor for each of the files. The -t prints the command to stderr before issuing it. The -r is a GNU extension that tells xargs not to run the command if no input was received.
find . -name "*.foo" -print0 | xargs -0 -I {} mv {} /tmp/trash
The above command uses -I to tell xargs to replace {} with the argument list. Note that not all versions of xargs supports the {} syntax. In those cases you may specify a string after -I that will be replaced, e.g.
find . -name "*.foo" -print0 | xargs -0 -I xxx mv xxx /tmp/trash
The above command uses string xxx instead of {} as the argument list marker.
find . -maxdepth 1 -type f -name "*.ogg" -print0 | xargs -0 -r cp -v -p --target-directory=/home/media
The command above does the same as:
cp -v -p *.ogg /home/media
however, the former command which uses find/xargs/cp is more resource efficient and will not halt with an error if the number of files is too large for the cp command to handle. Another way to do it (choosing where to put your arguments) is:
The -I in the above command tells xargs what replacement string you want to use (otherwise it adds the arguments to the end of the command). You can also use -L to limit the number of arguments. If you do that, the command will be run repeatedly until it is out of arguments. Thus, -L1 runs the command once for each argument (needed for tools like tar and such).
The separator problem
Many UNIX utilities are line oriented. These may work with xargs as long as the lines do not contain ', " or space. Some of the UNIX utilities can use NULL
Null character
The null character , abbreviated NUL, is a control character with the value zero.It is present in many character sets, including ISO/IEC 646 , the C0 control code, the Universal Character Set , and EBCDIC...
Perl is a high-level, general-purpose, interpreted, dynamic programming language. Perl was originally developed by Larry Wall in 1987 as a general-purpose Unix scripting language to make report processing easier. Since then, it has undergone many changes and revisions and become widely popular...
(requires -0 and \0 instead of \n), locate (requires using -0), find
Find
In Unix-like and some other operating systems, find is a command-line utility that searches through one or more directory trees of a file system, locates files based on some user-specified criteria and applies a user-specified action on each matched file...
sort is a standard Unix command line program that prints the lines of its input or concatenation of all files listed in its argument list in sorted order. Sorting is done based on one or more sort keys extracted from each line of input. By default, the entire input is taken as sort key...
(requires using -z)). Using -0 for xargs deals with the problem, but many UNIX utilities cannot use NULL as separator (e.g. head
Head (Unix)
head is a program on Unix and Unix-like systems used to display the first few lines of a text file or piped data. The command syntax is: head [options] <file_name>...
tail is a program on Unix and Unix-like systems used to display the last few lines of a text file or piped data.-Syntax:The command-syntax is: tail [options]...
In computing, ls is a command to list files in Unix and Unix-like operating systems. ls is specified by POSIX and the Single UNIX Specification.- History :An ls utility appeared in the original version of AT&T UNIX...
sed is a Unix utility that parses text and implements a programming language which can apply transformations to such text. It reads input line by line , applying the operation which has been specified via the command line , and then outputs the line. It was developed from 1973 to 1974 as a Unix...
wc is a command in Unix-like operating systems.The program reads either standard input or a list of files and generates one or more of the following statistics: number of bytes, number of words, and number of lines...
which is a Unix command used to identify the location of executables.The command takes one or more arguments; for each of these arguments, it prints to stdout the full path of the executable that would have been executed if this argument had been entered into the shell. It does this by searching...
).
But often people forget this and assume xargs is also line oriented.
Running the above will cause important_file to be removed and will remove neither the directory called 12" records, nor the file called not important_file.
When using the syntax find -print0, entries are separated by a null character instead of a end-of-line. This is equivalent to the more verbose command:
GNU parallel is a command-line driven utility for Linux or other Unix-like operating systems which allows the user to execute shell scripts in parallel. GNU parallel is a free software, written in Perl...
is an alternative to xargs that is designed to have the same options, but be line oriented. Thus, using GNU Parallel
Parallel (software)
GNU parallel is a command-line driven utility for Linux or other Unix-like operating systems which allows the user to execute shell scripts in parallel. GNU parallel is a free software, written in Perl...
instead, the above would work as expected.
For Unix environments where xargs does not support the -0 option (e.g. Solaris), the following can not be used as it does not deal with ' and " (GNU Parallel
Parallel (software)
GNU parallel is a command-line driven utility for Linux or other Unix-like operating systems which allows the user to execute shell scripts in parallel. GNU parallel is a free software, written in Perl...
would work on Solaris, though):
find -name not\* | sed 's/ /\\ /g' | xargs rm
The source of this article is wikipedia, the free encyclopedia. The text of this article is licensed under the GFDL.
