mirror of
https://git.savannah.gnu.org/git/coreutils.git
synced 2026-11-17 18:02:09 +03:00
doc: expand 'join' info section
* doc/coreutils.texi (join invocation): Expand section to add examples and more details. Suggested by Dan Jacobson in https://bugs.gnu.org/25870
This commit is contained in:
1 parent
8b22614b9a
commit
00381d22d4
1 file changed
+389
-35
+389
-35
@@ -6111,48 +6111,27 @@ Either @var{file1} or @var{file2} (but not both) can be @samp{-},
|
||||
meaning standard input. @var{file1} and @var{file2} should be
|
||||
sorted on the join fields.
|
||||
|
||||
@vindex LC_COLLATE
|
||||
Normally, the sort order is that of the
|
||||
collating sequence specified by the @env{LC_COLLATE} locale. Unless
|
||||
the @option{-t} option is given, the sort comparison ignores blanks at
|
||||
the start of the join field, as in @code{sort -b}. If the
|
||||
@option{--ignore-case} option is given, the sort comparison ignores
|
||||
the case of characters in the join field, as in @code{sort -f}.
|
||||
|
||||
The @command{sort} and @command{join} commands should use consistent
|
||||
locales and options if the output of @command{sort} is fed to
|
||||
@command{join}. You can use a command like @samp{sort -k 1b,1} to
|
||||
sort a file on its default join field, but if you select a non-default
|
||||
locale, join field, separator, or comparison options, then you should
|
||||
do so consistently between @command{join} and @command{sort}.
|
||||
If @samp{join -t ''} is specified then the whole line is considered which
|
||||
matches the default operation of sort.
|
||||
|
||||
If the input has no unpairable lines, a GNU extension is
|
||||
available; the sort order can be any order that considers two fields
|
||||
to be equal if and only if the sort comparison described above
|
||||
considers them to be equal. For example:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ cat file1
|
||||
a a1
|
||||
c c1
|
||||
b b1
|
||||
a 1
|
||||
b 2
|
||||
e 5
|
||||
|
||||
$ cat file2
|
||||
a a2
|
||||
c c2
|
||||
b b2
|
||||
a X
|
||||
e Y
|
||||
f Z
|
||||
|
||||
$ join file1 file2
|
||||
a a1 a2
|
||||
c c1 c2
|
||||
b b1 b2
|
||||
a 1 X
|
||||
e 5 Z
|
||||
@end group
|
||||
@end example
|
||||
|
||||
@set JOIN_COMMAND
|
||||
@checkOrderOption{join}
|
||||
@clear JOIN_COMMAND
|
||||
|
||||
The defaults are:
|
||||
@noindent
|
||||
@command{join}'s default behavior (when no options are given):
|
||||
@itemize
|
||||
@item the join field is the first field in each line;
|
||||
@item fields in the input are separated by one or more blanks, with leading
|
||||
@@ -6162,6 +6141,18 @@ blanks on the line ignored;
|
||||
fields from @var{file1}, then the remaining fields from @var{file2}.
|
||||
@end itemize
|
||||
|
||||
|
||||
@menu
|
||||
* General options in join:: Options which affect general program behavior.
|
||||
* Sorting files for join:: Using @command{sort} before @command{join}.
|
||||
* Working with fields:: Joining on different fields.
|
||||
* Paired and unpaired lines:: Controlling @command{join}'s field matching.
|
||||
* Header lines:: Working with header lines in files.
|
||||
* Set operations:: Union, Intersection and Difference of files.
|
||||
@end menu
|
||||
|
||||
@node General options in join
|
||||
@subsection General options
|
||||
The program accepts the following options. Also see @ref{Common options}.
|
||||
|
||||
@table @samp
|
||||
@@ -6262,6 +6253,369 @@ Print a line for each unpairable line in file @var{file-number}
|
||||
|
||||
@exitstatus
|
||||
|
||||
@set JOIN_COMMAND
|
||||
@checkOrderOption{join}
|
||||
@clear JOIN_COMMAND
|
||||
|
||||
|
||||
|
||||
@node Sorting files for join
|
||||
@subsection Pre-sorting
|
||||
|
||||
@command{join} requires sorted input files. Each input file should be
|
||||
sorted according to the key (=field/column number) used in
|
||||
@command{join}. The recommended sorting option is @samp{sort -k 1b,1}
|
||||
(assuming the desired key is in the first column).
|
||||
|
||||
@noindent Typical usage:
|
||||
@example
|
||||
@group
|
||||
$ sort -k 1b,1 file1 > file1.sorted
|
||||
$ sort -k 1b,1 file2 > file2.sorted
|
||||
$ join file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
@vindex LC_COLLATE
|
||||
Normally, the sort order is that of the
|
||||
collating sequence specified by the @env{LC_COLLATE} locale. Unless
|
||||
the @option{-t} option is given, the sort comparison ignores blanks at
|
||||
the start of the join field, as in @code{sort -b}. If the
|
||||
@option{--ignore-case} option is given, the sort comparison ignores
|
||||
the case of characters in the join field, as in @code{sort -f}:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ sort -k 1bf,1 file1 > file1.sorted
|
||||
$ sort -k 1bf,1 file2 > file2.sorted
|
||||
$ join --ignore-case file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
The @command{sort} and @command{join} commands should use consistent
|
||||
locales and options if the output of @command{sort} is fed to
|
||||
@command{join}. You can use a command like @samp{sort -k 1b,1} to
|
||||
sort a file on its default join field, but if you select a non-default
|
||||
locale, join field, separator, or comparison options, then you should
|
||||
do so consistently between @command{join} and @command{sort}.
|
||||
|
||||
@noindent To avoid any locale-related issues, it is recommended to use the
|
||||
@samp{C} locale for both commands:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ LC_ALL=C sort -k 1b,1 file1 > file1.sorted
|
||||
$ LC_ALL=C sort -k 1b,1 file2 > file2.sorted
|
||||
$ LC_ALL=C join file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
|
||||
@node Working with fields
|
||||
@subsection Working with fields
|
||||
|
||||
Use @option{-1},@option{-2} to set the key fields for each of the input files.
|
||||
Ensure the preceeding @command{sort} commands operated on the same fields.
|
||||
|
||||
@noindent
|
||||
The following example joins two files, using the values from seventh field
|
||||
of the first file and the third field of the second file:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ sort -k 7b,7 file1 > file1.sorted
|
||||
$ sort -k 3b,3 file2 > file2.sorted
|
||||
$ join -1 7 -2 3 file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
@noindent
|
||||
If the field number is the same for both files, use @option{-j}:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ sort -k4b,4 file1 > file1.sorted
|
||||
$ sort -k4b,4 file2 > file2.sorted
|
||||
$ join -j4 file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
@noindent
|
||||
Both @command{sort} and @command{join} operate of whitespace-delimited
|
||||
fields. To specify a different delimiter, use @option{-t} in @emph{both}:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ sort -t, -k3b,3 file1 > file1.sorted
|
||||
$ sort -t, -k3b,3 file2 > file2.sorted
|
||||
$ join -t, -j3 file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
@noindent
|
||||
To specify a tab (@sc{ascii} 0x09) character instead of whitespace, use
|
||||
@footnote{the @code{$'\t'} is supported in most modern shells.
|
||||
For older shells, use a literal tab}:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ sort -t$'\t' -k3b,3 file1 > file1.sorted
|
||||
$ sort -t$'\t' -k3b,3 file2 > file2.sorted
|
||||
$ join -t$'\t' -j3 file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
|
||||
@noindent
|
||||
If @samp{join -t ''} is specified then the whole line is considered which
|
||||
matches the default operation of sort:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ sort file1 > file1.sorted
|
||||
$ sort file2 > file2.sorted
|
||||
$ join -t'' file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
|
||||
@node Paired and unpaired lines
|
||||
@subsection Controlling @command{join}'s field matching
|
||||
|
||||
In this section the @command{sort} commands are omitted for brevity.
|
||||
Sorting the files before joining is still required.
|
||||
|
||||
@command{join}'s default behavior is to print only lines common to
|
||||
both input files. Use @option{-a} and @option{-v} to print unpairable lines
|
||||
from one or both files.
|
||||
|
||||
@noindent
|
||||
All examples below use the following two (pre-sorted) input files:
|
||||
|
||||
@multitable @columnfractions .5 .5
|
||||
@item
|
||||
@example
|
||||
$ cat file1
|
||||
a 1
|
||||
b 2
|
||||
@end example
|
||||
|
||||
@tab
|
||||
@example
|
||||
$ cat file2
|
||||
a A
|
||||
c C
|
||||
@end example
|
||||
@end multitable
|
||||
|
||||
|
||||
@c TODO: Find better column widths that work for both HTML and PDF
|
||||
@c and disable indentation of @example.
|
||||
@multitable @columnfractions 0.5 0.5
|
||||
|
||||
@headitem Command @tab Outcome
|
||||
|
||||
|
||||
@item
|
||||
@example
|
||||
$ join file1 file2
|
||||
a 1 A
|
||||
@end example
|
||||
@tab
|
||||
common lines
|
||||
(@emph{intersection})
|
||||
|
||||
|
||||
|
||||
@item
|
||||
@example
|
||||
$ join -a 1 file1 file2
|
||||
a 1 A
|
||||
b 2
|
||||
@end example
|
||||
@tab
|
||||
common lines @emph{and} unpaired
|
||||
lines from the first file
|
||||
|
||||
|
||||
@item
|
||||
@example
|
||||
$ join -a 2 file1 file2
|
||||
a 1 A
|
||||
c C
|
||||
@end example
|
||||
@tab
|
||||
common lines @emph{and} unpaired lines from the second file
|
||||
|
||||
|
||||
@item
|
||||
@example
|
||||
$ join -a 1 -a 2 file1 file2
|
||||
a 1 A
|
||||
b 2
|
||||
c C
|
||||
@end example
|
||||
@tab
|
||||
all lines (paired and unpaired) from both files
|
||||
(@emph{union}).
|
||||
@*
|
||||
see note below regarding @code{-o auto}.
|
||||
|
||||
|
||||
@item
|
||||
@example
|
||||
$ join -v 1 file1 file2
|
||||
b 1
|
||||
@end example
|
||||
@tab
|
||||
unpaired lines from the first file
|
||||
(@emph{difference})
|
||||
|
||||
|
||||
@item
|
||||
@example
|
||||
$ join -v 2 file1 file2
|
||||
c C
|
||||
@end example
|
||||
@tab
|
||||
unpaired lines from the second file
|
||||
(@emph{difference})
|
||||
|
||||
|
||||
@item
|
||||
@example
|
||||
$ join -v 1 -v 2 file1 file2
|
||||
b 2
|
||||
c C
|
||||
@end example
|
||||
@tab
|
||||
unpaired lines from both files, omitting common lines
|
||||
(@emph{symmetric difference}).
|
||||
|
||||
|
||||
@end multitable
|
||||
|
||||
@noindent
|
||||
The @option{-o auto -e X} options are useful when dealing with unpaired lines.
|
||||
The following example prints all lines (common and unpaired) from both files.
|
||||
Without @option{-o auto} it is not easy to discern which fields originate from
|
||||
which file:
|
||||
|
||||
@example
|
||||
$ join -a 1 -a 2 file1 file2
|
||||
a 1 A
|
||||
b 2
|
||||
c C
|
||||
|
||||
$ join -o auto -e X -a 1 -a 2 file1 file2
|
||||
a 1 A
|
||||
b 2 X
|
||||
c X C
|
||||
@end example
|
||||
|
||||
|
||||
If the input has no unpairable lines, a GNU extension is
|
||||
available; the sort order can be any order that considers two fields
|
||||
to be equal if and only if the sort comparison described above
|
||||
considers them to be equal. For example:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ cat file1
|
||||
a a1
|
||||
c c1
|
||||
b b1
|
||||
|
||||
$ cat file2
|
||||
a a2
|
||||
c c2
|
||||
b b2
|
||||
$ join file1 file2
|
||||
a a1 a2
|
||||
c c1 c2
|
||||
b b1 b2
|
||||
@end group
|
||||
@end example
|
||||
|
||||
|
||||
@node Header lines
|
||||
@subsection Header lines
|
||||
|
||||
The @option{--header} option can be used when the files to join
|
||||
have a header line which is not sorted:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ cat file1
|
||||
Name Age
|
||||
Alice 25
|
||||
Charlie 34
|
||||
|
||||
$ cat file2
|
||||
Name Country
|
||||
Alice France
|
||||
Bob Spain
|
||||
|
||||
$ join --header -o auto -e NA -a1 -a2 file1 file2
|
||||
Name Age Country
|
||||
Alice 25 France
|
||||
Bob NA Spain
|
||||
Charlie 34 NA
|
||||
@end group
|
||||
@end example
|
||||
|
||||
|
||||
To sort a file with a header line, use @sc{GNU} @command{sed -u}.
|
||||
The following example sort the files but keeps the first line of each
|
||||
file in place:
|
||||
|
||||
@example
|
||||
@group
|
||||
$ ( sed -u 1q ; sort -k2b,2 ) < file1 > file1.sorted
|
||||
$ ( sed -u 1q ; sort -k2b,2 ) < file2 > file2.sorted
|
||||
$ join --header -o auto -e NA -a1 -a2 file1.sorted file2.sorted > file3
|
||||
@end group
|
||||
@end example
|
||||
|
||||
@node Set operations
|
||||
@subsection Union, Intersection and Difference of files
|
||||
|
||||
Combine @command{sort}, @command{uniq} and @command{join} to
|
||||
perform the equivalent of set operations on files:
|
||||
|
||||
@c From https://www.pixelbeat.org/cmdline.html#sets
|
||||
@multitable @columnfractions 0.5 0.5
|
||||
@headitem Command @tab outcome
|
||||
@item @code{sort -u file1 file2}
|
||||
@tab Union of unsorted files
|
||||
|
||||
@item @code{sort file1 file2 | uniq -d}
|
||||
@tab Intersection of unsorted files
|
||||
|
||||
@item @code{sort file1 file1 file2 | uniq}
|
||||
@tab Difference of unsorted files
|
||||
|
||||
@item @code{sort file1 file2 | uniq -u}
|
||||
@tab Symmetric Difference of unsorted files
|
||||
|
||||
@item @code{join -t'' -a1 -a2 file1 file2}
|
||||
@tab Union of sorted files
|
||||
|
||||
@item @code{join -t'' file1 file2}
|
||||
@tab Intersection of sorted files
|
||||
|
||||
@item @code{join -t'' -v2 file1 file2}
|
||||
@tab Difference of sorted files
|
||||
|
||||
@item @code{join -t'' -v1 -v2 file1 file2}
|
||||
@tab Symmetric Difference of sorted files
|
||||
|
||||
@end multitable
|
||||
|
||||
All examples above operate on entire lines and not on specific fields:
|
||||
@command{sort} without @option{-k} and @command{join -t''} both consider
|
||||
entire lines as the key.
|
||||
|
||||
|
||||
@node Operating on characters
|
||||
@chapter Operating on characters
|
||||
|
||||
Reference in new issue
Block a user