doc: expand 'join' info section

* doc/coreutils.texi (join invocation): Expand section to
add examples and more details.
Suggested by Dan Jacobson in https://bugs.gnu.org/25870
This commit is contained in:
Assaf Gordon authored and Pádraig Brady committed 2017-03-01 18:47:24 -08:00
1 parent 8b22614b9a
commit 00381d22d4
1 file changed
+389 -35
+389 -35
View File
@@ -6111,48 +6111,27 @@ Either @var{file1} or @var{file2} (but not both) can be @samp{-},
meaning standard input. @var{file1} and @var{file2} should be
sorted on the join fields.
@vindex LC_COLLATE
Normally, the sort order is that of the
collating sequence specified by the @env{LC_COLLATE} locale. Unless
the @option{-t} option is given, the sort comparison ignores blanks at
the start of the join field, as in @code{sort -b}. If the
@option{--ignore-case} option is given, the sort comparison ignores
the case of characters in the join field, as in @code{sort -f}.
The @command{sort} and @command{join} commands should use consistent
locales and options if the output of @command{sort} is fed to
@command{join}. You can use a command like @samp{sort -k 1b,1} to
sort a file on its default join field, but if you select a non-default
locale, join field, separator, or comparison options, then you should
do so consistently between @command{join} and @command{sort}.
If @samp{join -t ''} is specified then the whole line is considered which
matches the default operation of sort.
If the input has no unpairable lines, a GNU extension is
available; the sort order can be any order that considers two fields
to be equal if and only if the sort comparison described above
considers them to be equal. For example:
@example
@group
$ cat file1
a a1
c c1
b b1
a 1
b 2
e 5
$ cat file2
a a2
c c2
b b2
a X
e Y
f Z
$ join file1 file2
a a1 a2
c c1 c2
b b1 b2
a 1 X
e 5 Z
@end group
@end example
@set JOIN_COMMAND
@checkOrderOption{join}
@clear JOIN_COMMAND
The defaults are:
@noindent
@command{join}'s default behavior (when no options are given):
@itemize
@item the join field is the first field in each line;
@item fields in the input are separated by one or more blanks, with leading
@@ -6162,6 +6141,18 @@ blanks on the line ignored;
fields from @var{file1}, then the remaining fields from @var{file2}.
@end itemize
@menu
* General options in join:: Options which affect general program behavior.
* Sorting files for join:: Using @command{sort} before @command{join}.
* Working with fields:: Joining on different fields.
* Paired and unpaired lines:: Controlling @command{join}'s field matching.
* Header lines:: Working with header lines in files.
* Set operations:: Union, Intersection and Difference of files.
@end menu
@node General options in join
@subsection General options
The program accepts the following options. Also see @ref{Common options}.
@table @samp
@@ -6262,6 +6253,369 @@ Print a line for each unpairable line in file @var{file-number}
@exitstatus
@set JOIN_COMMAND
@checkOrderOption{join}
@clear JOIN_COMMAND
@node Sorting files for join
@subsection Pre-sorting
@command{join} requires sorted input files. Each input file should be
sorted according to the key (=field/column number) used in
@command{join}. The recommended sorting option is @samp{sort -k 1b,1}
(assuming the desired key is in the first column).
@noindent Typical usage:
@example
@group
$ sort -k 1b,1 file1 > file1.sorted
$ sort -k 1b,1 file2 > file2.sorted
$ join file1.sorted file2.sorted > file3
@end group
@end example
@vindex LC_COLLATE
Normally, the sort order is that of the
collating sequence specified by the @env{LC_COLLATE} locale. Unless
the @option{-t} option is given, the sort comparison ignores blanks at
the start of the join field, as in @code{sort -b}. If the
@option{--ignore-case} option is given, the sort comparison ignores
the case of characters in the join field, as in @code{sort -f}:
@example
@group
$ sort -k 1bf,1 file1 > file1.sorted
$ sort -k 1bf,1 file2 > file2.sorted
$ join --ignore-case file1.sorted file2.sorted > file3
@end group
@end example
The @command{sort} and @command{join} commands should use consistent
locales and options if the output of @command{sort} is fed to
@command{join}. You can use a command like @samp{sort -k 1b,1} to
sort a file on its default join field, but if you select a non-default
locale, join field, separator, or comparison options, then you should
do so consistently between @command{join} and @command{sort}.
@noindent To avoid any locale-related issues, it is recommended to use the
@samp{C} locale for both commands:
@example
@group
$ LC_ALL=C sort -k 1b,1 file1 > file1.sorted
$ LC_ALL=C sort -k 1b,1 file2 > file2.sorted
$ LC_ALL=C join file1.sorted file2.sorted > file3
@end group
@end example
@node Working with fields
@subsection Working with fields
Use @option{-1},@option{-2} to set the key fields for each of the input files.
Ensure the preceeding @command{sort} commands operated on the same fields.
@noindent
The following example joins two files, using the values from seventh field
of the first file and the third field of the second file:
@example
@group
$ sort -k 7b,7 file1 > file1.sorted
$ sort -k 3b,3 file2 > file2.sorted
$ join -1 7 -2 3 file1.sorted file2.sorted > file3
@end group
@end example
@noindent
If the field number is the same for both files, use @option{-j}:
@example
@group
$ sort -k4b,4 file1 > file1.sorted
$ sort -k4b,4 file2 > file2.sorted
$ join -j4 file1.sorted file2.sorted > file3
@end group
@end example
@noindent
Both @command{sort} and @command{join} operate of whitespace-delimited
fields. To specify a different delimiter, use @option{-t} in @emph{both}:
@example
@group
$ sort -t, -k3b,3 file1 > file1.sorted
$ sort -t, -k3b,3 file2 > file2.sorted
$ join -t, -j3 file1.sorted file2.sorted > file3
@end group
@end example
@noindent
To specify a tab (@sc{ascii} 0x09) character instead of whitespace, use
@footnote{the @code{$'\t'} is supported in most modern shells.
For older shells, use a literal tab}:
@example
@group
$ sort -t$'\t' -k3b,3 file1 > file1.sorted
$ sort -t$'\t' -k3b,3 file2 > file2.sorted
$ join -t$'\t' -j3 file1.sorted file2.sorted > file3
@end group
@end example
@noindent
If @samp{join -t ''} is specified then the whole line is considered which
matches the default operation of sort:
@example
@group
$ sort file1 > file1.sorted
$ sort file2 > file2.sorted
$ join -t'' file1.sorted file2.sorted > file3
@end group
@end example
@node Paired and unpaired lines
@subsection Controlling @command{join}'s field matching
In this section the @command{sort} commands are omitted for brevity.
Sorting the files before joining is still required.
@command{join}'s default behavior is to print only lines common to
both input files. Use @option{-a} and @option{-v} to print unpairable lines
from one or both files.
@noindent
All examples below use the following two (pre-sorted) input files:
@multitable @columnfractions .5 .5
@item
@example
$ cat file1
a 1
b 2
@end example
@tab
@example
$ cat file2
a A
c C
@end example
@end multitable
@c TODO: Find better column widths that work for both HTML and PDF
@c and disable indentation of @example.
@multitable @columnfractions 0.5 0.5
@headitem Command @tab Outcome
@item
@example
$ join file1 file2
a 1 A
@end example
@tab
common lines
(@emph{intersection})
@item
@example
$ join -a 1 file1 file2
a 1 A
b 2
@end example
@tab
common lines @emph{and} unpaired
lines from the first file
@item
@example
$ join -a 2 file1 file2
a 1 A
c C
@end example
@tab
common lines @emph{and} unpaired lines from the second file
@item
@example
$ join -a 1 -a 2 file1 file2
a 1 A
b 2
c C
@end example
@tab
all lines (paired and unpaired) from both files
(@emph{union}).
@*
see note below regarding @code{-o auto}.
@item
@example
$ join -v 1 file1 file2
b 1
@end example
@tab
unpaired lines from the first file
(@emph{difference})
@item
@example
$ join -v 2 file1 file2
c C
@end example
@tab
unpaired lines from the second file
(@emph{difference})
@item
@example
$ join -v 1 -v 2 file1 file2
b 2
c C
@end example
@tab
unpaired lines from both files, omitting common lines
(@emph{symmetric difference}).
@end multitable
@noindent
The @option{-o auto -e X} options are useful when dealing with unpaired lines.
The following example prints all lines (common and unpaired) from both files.
Without @option{-o auto} it is not easy to discern which fields originate from
which file:
@example
$ join -a 1 -a 2 file1 file2
a 1 A
b 2
c C
$ join -o auto -e X -a 1 -a 2 file1 file2
a 1 A
b 2 X
c X C
@end example
If the input has no unpairable lines, a GNU extension is
available; the sort order can be any order that considers two fields
to be equal if and only if the sort comparison described above
considers them to be equal. For example:
@example
@group
$ cat file1
a a1
c c1
b b1
$ cat file2
a a2
c c2
b b2
$ join file1 file2
a a1 a2
c c1 c2
b b1 b2
@end group
@end example
@node Header lines
@subsection Header lines
The @option{--header} option can be used when the files to join
have a header line which is not sorted:
@example
@group
$ cat file1
Name Age
Alice 25
Charlie 34
$ cat file2
Name Country
Alice France
Bob Spain
$ join --header -o auto -e NA -a1 -a2 file1 file2
Name Age Country
Alice 25 France
Bob NA Spain
Charlie 34 NA
@end group
@end example
To sort a file with a header line, use @sc{GNU} @command{sed -u}.
The following example sort the files but keeps the first line of each
file in place:
@example
@group
$ ( sed -u 1q ; sort -k2b,2 ) < file1 > file1.sorted
$ ( sed -u 1q ; sort -k2b,2 ) < file2 > file2.sorted
$ join --header -o auto -e NA -a1 -a2 file1.sorted file2.sorted > file3
@end group
@end example
@node Set operations
@subsection Union, Intersection and Difference of files
Combine @command{sort}, @command{uniq} and @command{join} to
perform the equivalent of set operations on files:
@c From https://www.pixelbeat.org/cmdline.html#sets
@multitable @columnfractions 0.5 0.5
@headitem Command @tab outcome
@item @code{sort -u file1 file2}
@tab Union of unsorted files
@item @code{sort file1 file2 | uniq -d}
@tab Intersection of unsorted files
@item @code{sort file1 file1 file2 | uniq}
@tab Difference of unsorted files
@item @code{sort file1 file2 | uniq -u}
@tab Symmetric Difference of unsorted files
@item @code{join -t'' -a1 -a2 file1 file2}
@tab Union of sorted files
@item @code{join -t'' file1 file2}
@tab Intersection of sorted files
@item @code{join -t'' -v2 file1 file2}
@tab Difference of sorted files
@item @code{join -t'' -v1 -v2 file1 file2}
@tab Symmetric Difference of sorted files
@end multitable
All examples above operate on entire lines and not on specific fields:
@command{sort} without @option{-k} and @command{join -t''} both consider
entire lines as the key.
@node Operating on characters
@chapter Operating on characters