scripts/bash-ini-parser
1
0
Fork 0
mirror of https://github.com/albfan/bash-ini-parser.git synced 2025-04-21 00:07:46 +03:00
Code

Squashed 't/sharness/' content from commit 665e154

Browse Source 
git-subtree-dir: t/sharness
git-subtree-split: 665e154c365762867ce537a7ca6328516ccd5b44
This commit is contained in:
albfan 2015-05-16 11:37:33 +02:00
commit 2d24f2698e
15 changed files with 2879 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
.travis.yml Normal file
View File

@ -0,0 +1,16 @@
language: erlang # no shell language support; use least-loaded worker(s)
env:
- TEST_OPTS=-v
- DEFAULT_TEST_TARGET=prove
install:
- sudo make install prefix=/usr/local
script:
- make test
- sudo make -C /usr/local/share/doc/sharness/examples
branches:
only:
- master

302
API.md Normal file
View File

@ -0,0 +1,302 @@
# Sharness API
### SHARNESS_VERSION
Public: Current version of Sharness.
### SHARNESS_TEST_EXTENSION
Public: The file extension for tests. By default, it is set to "t".
### test_set_prereq()
Public: Define that a test prerequisite is available.
The prerequisite can later be checked explicitly using test_have_prereq or
implicitly by specifying the prerequisite name in calls to test_expect_success
or test_expect_failure.
$1 - Name of prerequiste (a simple word, in all capital letters by convention)
Examples
# Set PYTHON prerequisite if interpreter is available.
command -v python >/dev/null && test_set_prereq PYTHON
# Set prerequisite depending on some variable.
test -z "$NO_GETTEXT" && test_set_prereq GETTEXT
Returns nothing.
### test_have_prereq()
Public: Check if one or more test prerequisites are defined.
The prerequisites must have previously been set with test_set_prereq.
The most common use of this is to skip all the tests if some essential
prerequisite is missing.
$1 - Comma-separated list of test prerequisites.
Examples
# Skip all remaining tests if prerequisite is not set.
if ! test_have_prereq PERL; then
skip_all='skipping perl interface tests, perl not available'
test_done
fi
Returns 0 if all prerequisites are defined or 1 otherwise.
### test_debug()
Public: Execute commands in debug mode.
Takes a single argument and evaluates it only when the test script is started
with --debug. This is primarily meant for use during the development of test
scripts.
$1 - Commands to be executed.
Examples
test_debug "cat some_log_file"
Returns the exit code of the last command executed in debug mode or 0
otherwise.
### test_expect_success()
Public: Run test commands and expect them to succeed.
When the test passed, an "ok" message is printed and the number of successful
tests is incremented. When it failed, a "not ok" message is printed and the
number of failed tests is incremented.
With --immediate, exit test immediately upon the first failed test.
Usually takes two arguments:
$1 - Test description
$2 - Commands to be executed.
With three arguments, the first will be taken to be a prerequisite:
$1 - Comma-separated list of test prerequisites. The test will be skipped if
not all of the given prerequisites are set. To negate a prerequisite,
put a "!" in front of it.
$2 - Test description
$3 - Commands to be executed.
Examples
test_expect_success \
'git-write-tree should be able to write an empty tree.' \
'tree=$(git-write-tree)'
# Test depending on one prerequisite.
test_expect_success TTY 'git --paginate rev-list uses a pager' \
' ... '
# Multiple prerequisites are separated by a comma.
test_expect_success PERL,PYTHON 'yo dawg' \
' test $(perl -E 'print eval "1 +" . qx[python -c "print 2"]') == "4" '
Returns nothing.
### test_expect_failure()
Public: Run test commands and expect them to fail. Used to demonstrate a known
breakage.
This is NOT the opposite of test_expect_success, but rather used to mark a
test that demonstrates a known breakage.
When the test passed, an "ok" message is printed and the number of fixed tests
is incremented. When it failed, a "not ok" message is printed and the number
of tests still broken is incremented.
Failures from these tests won't cause --immediate to stop.
Usually takes two arguments:
$1 - Test description
$2 - Commands to be executed.
With three arguments, the first will be taken to be a prerequisite:
$1 - Comma-separated list of test prerequisites. The test will be skipped if
not all of the given prerequisites are set. To negate a prerequisite,
put a "!" in front of it.
$2 - Test description
$3 - Commands to be executed.
Returns nothing.
### test_must_fail()
Public: Run command and ensure that it fails in a controlled way.
Use it instead of "! <command>". For example, when <command> dies due to a
segfault, test_must_fail diagnoses it as an error, while "! <command>" would
mistakenly be treated as just another expected failure.
This is one of the prefix functions to be used inside test_expect_success or
test_expect_failure.
$1.. - Command to be executed.
Examples
test_expect_success 'complain and die' '
do something &&
do something else &&
test_must_fail git checkout ../outerspace
'
Returns 1 if the command succeeded (exit code 0).
Returns 1 if the command died by signal (exit codes 130-192)
Returns 1 if the command could not be found (exit code 127).
Returns 0 otherwise.
### test_might_fail()
Public: Run command and ensure that it succeeds or fails in a controlled way.
Similar to test_must_fail, but tolerates success too. Use it instead of
"<command> || :" to catch failures caused by a segfault, for instance.
This is one of the prefix functions to be used inside test_expect_success or
test_expect_failure.
$1.. - Command to be executed.
Examples
test_expect_success 'some command works without configuration' '
test_might_fail git config --unset all.configuration &&
do something
'
Returns 1 if the command died by signal (exit codes 130-192)
Returns 1 if the command could not be found (exit code 127).
Returns 0 otherwise.
### test_expect_code()
Public: Run command and ensure it exits with a given exit code.
This is one of the prefix functions to be used inside test_expect_success or
test_expect_failure.
$1 - Expected exit code.
$2.. - Command to be executed.
Examples
test_expect_success 'Merge with d/f conflicts' '
test_expect_code 1 git merge "merge msg" B master
'
Returns 0 if the expected exit code is returned or 1 otherwise.
### test_cmp()
Public: Compare two files to see if expected output matches actual output.
The TEST_CMP variable defines the command used for the comparision; it
defaults to "diff -u". Only when the test script was started with --verbose,
will the command's output, the diff, be printed to the standard output.
This is one of the prefix functions to be used inside test_expect_success or
test_expect_failure.
$1 - Path to file with expected output.
$2 - Path to file with actual output.
Examples
test_expect_success 'foo works' '
echo expected >expected &&
foo >actual &&
test_cmp expected actual
'
Returns the exit code of the command set by TEST_CMP.
### test_when_finished()
Public: Schedule cleanup commands to be run unconditionally at the end of a
test.
If some cleanup command fails, the test will not pass. With --immediate, no
cleanup is done to help diagnose what went wrong.
This is one of the prefix functions to be used inside test_expect_success or
test_expect_failure.
$1.. - Commands to prepend to the list of cleanup commands.
Examples
test_expect_success 'test core.capslock' '
git config core.capslock true &&
test_when_finished "git config --unset core.capslock" &&
do_something
'
Returns the exit code of the last cleanup command executed.
### test_done()
Public: Summarize test results and exit with an appropriate error code.
Must be called at the end of each test script.
Can also be used to stop tests early and skip all remaining tests. For this,
set skip_all to a string explaining why the tests were skipped before calling
test_done.
Examples
# Each test script must call test_done at the end.
test_done
# Skip all remaining tests if prerequisite is not set.
if ! test_have_prereq PERL; then
skip_all='skipping perl interface tests, perl not available'
test_done
fi
Returns 0 if all tests passed or 1 if there was a failure.
### cleanup()
Public: Schedule cleanup commands to be run unconditionally when all tests
have run.
This can be used to clean up things like test databases. It is not needed to
clean up temporary files, as test_done already does that.
Examples:
cleanup mysql -e "DROP DATABASE mytest"
Returns the exit code of the last cleanup command executed.
### SHARNESS_TEST_DIRECTORY
Public: Root directory containing tests. Tests can override this variable,
e.g. for testing Sharness itself.
### SHARNESS_BUILD_DIRECTORY
Public: Build directory that will be added to PATH. By default, it is set to
the parent directory of SHARNESS_TEST_DIRECTORY.
### SHARNESS_TEST_FILE
Public: Path to test script currently executed.
### SHARNESS_TRASH_DIRECTORY
Public: Empty trash directory, the test area, provided for each test. The HOME
variable is set to that directory too.
Generated by tomdoc.sh version 0.1.4

116
CHANGELOG.md Normal file
View File

@ -0,0 +1,116 @@
v0.3.0 (2013-04-03)
-------------------
This release is all about bringing upstream fixes and improvements from Git to
Sharness ([GH-7]).
List of merged upstream changes:
* Make test number come first in `not ok $count - $message`.
* Paint known breakages in yellow.
* Paint unexpectedly fixed known breakages in bold red.
* Paint skipped tests in blue.
* Change info messages from yellow/brown to cyan.
* Fix `say_color()` to not interpret `\a\b\c` in the message.
* Add check for invalid use of `skip_all` facility.
* Rename `$satisfied` to `$satisfied_prereq`.
* Allow negation of prerequisites with "!".
* Retain cache file `test/.prove` across prove runs.
* Replace `basic.t` with `sharness.t` which is an adapted version of
`t0000-basic.sh` from upstream.
* Update `README.git` with upstream changes.
Other changes:
* Add [git-integration] to the list of projects using Sharness. Also pay tribute
to Git's test suite.
* Let Travis only test the master branch (and pull requests).
[GH-7]: https://github.com/mlafeldt/sharness/pull/7
[git-integration]: https://github.com/johnkeeping/git-integration
v0.2.5 (2013-03-29)
-------------------
* Allow to install Sharness via `make install` and to uninstall it via
`make uninstall`. See brand-new installation instructions in README. ([GH-5])
* Allow users to override the test extension via `SHARNESS_TEST_EXTENSION` if
they wish to. ([GH-6])
* Don't set a variable and export it at the same time. ([GH-6])
* Remove `TEST_INSTALLED` -- use `SHARNESS_BUILD_DIRECTORY` instead.
* Add vi modeline to `sharness.sh`.
* Add `AGGREGATE_SCRIPT` variable to `test/Makefile`.
* Remove superfluous `SHARNESS_TEST_DIRECTORY` assignments from `test/basic.t`.
* Add [timedb] to the list of projects using Sharness.
* Add Sharness alternatives to README.
* Rename HISTORY.md to CHANGELOG.md.
[GH-5]: https://github.com/mlafeldt/sharness/pull/5
[GH-6]: https://github.com/mlafeldt/sharness/pull/6
[timedb]: http://git.cryptoism.org/cgit.cgi/timedb.git
v0.2.4 (2012-07-13)
-------------------
* Add `simple.t` to tests and README.
* Provide `SHARNESS_TEST_FILE` which is the path to the test script currently
being executed.
* Add [dabba] to the list of projects using Sharness.
[dabba]: https://github.com/eroullit/dabba
v0.2.3 (2012-06-20)
-------------------
* Make `.t` the new test file extension, which is the default extension used by
`prove(1)`. (You can still use the `t????-*` scheme, but you need to rename
the `.sh` ending of all tests.)
* Rename, export, and document public variables `SHARNESS_TEST_DIRECTORY`,
`SHARNESS_BUILD_DIRECTORY`, and `SHARNESS_TRASH_DIRECTORY`.
* TomDoc `SHARNESS_TEST_EXTENSION`.
v0.2.2 (2012-04-27)
-------------------
* Document all public API functions using [TomDoc] and let [tomdoc.sh] generate
documentation in markdown format from it, see `API.md`.
* Rename `test_skip` to `test_skip_` as it is internal.
* Clean up `test/Makefile`.
* Sync Git README with upstream.
[TomDoc]: http://tomdoc.org/
[tomdoc.sh]: https://github.com/mlafeldt/tomdoc.sh
v0.2.1 (2012-03-01)
-------------------
* Fix: Redirect stdin of tests (by @peff).
* Unify coding style across all shell scripts.
* Remove superfluous functions `sane_unset` and `test_declared_prereq`.
* Get rid of variables `DIFF` and `TEST_CMP_USE_COPIED_CONTEXT`.
* Remove dysfunctional smoke testing targets from `test/Makefile`.
* Add Travis CI config.
* Add top-level Makefile to say `make test`.
* Add GPL header to all files from Git.
v0.2.0 (2011-12-13)
-------------------
* Rename `test-lib.sh` to `sharness.sh`.
* Strip more Git-specific functionality.
* Add variable `SHARNESS_VERSION`.
* Move self-tests to `test` folder; keep essential files in root.
* Update README.
* Add this history file.
v0.1.1 (2011-11-02)
-------------------
* Merge changes to test harness library from Git v1.7.8-rc0
v0.1.0 (2011-05-02)
-------------------
* First version based on test harness library from Git v1.7.5
* Remove Git-specific functions, variables, prerequisites, make targets, etc.
* Remove `GIT_` prefix from global variables.

339
COPYING Normal file
View File

@ -0,0 +1,339 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.

36
Makefile Normal file
View File

@ -0,0 +1,36 @@
prefix = $(HOME)
INSTALL_DIR = $(prefix)/share/sharness
DOC_DIR = $(prefix)/share/doc/sharness
EXAMPLE_DIR = $(DOC_DIR)/examples
INSTALL_FILES = aggregate-results.sh sharness.sh
DOC_FILES = API.md CHANGELOG.md COPYING README.git README.md
EXAMPLE_FILES = test/Makefile test/simple.t
INSTALL = install
RM = rm -f
SED = sed
TOMDOCSH = tomdoc.sh
all:
install: all
$(INSTALL) -d -m 755 $(INSTALL_DIR) $(DOC_DIR) $(EXAMPLE_DIR)
$(INSTALL) -m 644 $(INSTALL_FILES) $(INSTALL_DIR)
$(INSTALL) -m 644 $(DOC_FILES) $(DOC_DIR)
$(SED) -e "s!aggregate-results.sh!$(INSTALL_DIR)/aggregate-results.sh!" test/Makefile > $(EXAMPLE_DIR)/Makefile
$(SED) -e "s!. ./sharness.sh!. $(INSTALL_DIR)/sharness.sh!" test/simple.t > $(EXAMPLE_DIR)/simple.t
uninstall:
$(RM) -r $(INSTALL_DIR) $(DOC_DIR) $(EXAMPLE_DIR)
doc: all
{ printf "# Sharness API\n\n"; \
$(TOMDOCSH) -m -a Public sharness.sh; \
printf "Generated by "; $(TOMDOCSH) --version; } >API.md
test: all
$(MAKE) -C test
.PHONY: all install uninstall doc test

709
README.git Normal file
View File

@ -0,0 +1,709 @@
Core GIT Tests
==============
This directory holds many test scripts for core GIT tools. The
first part of this short document describes how to run the tests
and read their output.
When fixing the tools or adding enhancements, you are strongly
encouraged to add tests in this directory to cover what you are
trying to fix or enhance. The later part of this short document
describes how your test scripts should be organized.
Running Tests
-------------
The easiest way to run tests is to say "make". This runs all
the tests.
*** t0000-basic.sh ***
ok 1 - .git/objects should be empty after git init in an empty repo.
ok 2 - .git/objects should have 3 subdirectories.
ok 3 - success is reported like this
...
ok 43 - very long name in the index handled sanely
# fixed 1 known breakage(s)
# still have 1 known breakage(s)
# passed all remaining 42 test(s)
1..43
*** t0001-init.sh ***
ok 1 - plain
ok 2 - plain with GIT_WORK_TREE
ok 3 - plain bare
Since the tests all output TAP (see http://testanything.org) they can
be run with any TAP harness. Here's an example of parallel testing
powered by a recent version of prove(1):
$ prove --timer --jobs 15 ./t[0-9]*.sh
[19:17:33] ./t0005-signals.sh ................................... ok 36 ms
[19:17:33] ./t0022-crlf-rename.sh ............................... ok 69 ms
[19:17:33] ./t0024-crlf-archive.sh .............................. ok 154 ms
[19:17:33] ./t0004-unwritable.sh ................................ ok 289 ms
[19:17:33] ./t0002-gitfile.sh ................................... ok 480 ms
===( 102;0 25/? 6/? 5/? 16/? 1/? 4/? 2/? 1/? 3/? 1... )===
prove and other harnesses come with a lot of useful options. The
--state option in particular is very useful:
# Repeat until no more failures
$ prove -j 15 --state=failed,save ./t[0-9]*.sh
You can give DEFAULT_TEST_TARGET=prove on the make command (or define it
in config.mak) to cause "make test" to run tests under prove.
GIT_PROVE_OPTS can be used to pass additional options, e.g.
$ make DEFAULT_TEST_TARGET=prove GIT_PROVE_OPTS='--timer --jobs 16' test
You can also run each test individually from command line, like this:
$ sh ./t3010-ls-files-killed-modified.sh
ok 1 - git update-index --add to add various paths.
ok 2 - git ls-files -k to show killed files.
ok 3 - validate git ls-files -k output.
ok 4 - git ls-files -m to show modified files.
ok 5 - validate git ls-files -m output.
# passed all 5 test(s)
1..5
You can pass --verbose (or -v), --debug (or -d), and --immediate
(or -i) command line argument to the test, or by setting GIT_TEST_OPTS
appropriately before running "make".
--verbose::
This makes the test more verbose. Specifically, the
command being run and their output if any are also
output.
--debug::
This may help the person who is developing a new test.
It causes the command defined with test_debug to run.
The "trash" directory (used to store all temporary data
during testing) is not deleted even if there are no
failed tests so that you can inspect its contents after
the test finished.
--immediate::
This causes the test to immediately exit upon the first
failed test.
--long-tests::
This causes additional long-running tests to be run (where
available), for more exhaustive testing.
--valgrind::
Execute all Git binaries with valgrind and exit with status
126 on errors (just like regular tests, this will only stop
the test script when running under -i). Valgrind errors
go to stderr, so you might want to pass the -v option, too.
Since it makes no sense to run the tests with --valgrind and
not see any output, this option implies --verbose. For
convenience, it also implies --tee.
Note that valgrind is run with the option --leak-check=no,
as the git process is short-lived and some errors are not
interesting. In order to run a single command under the same
conditions manually, you should set GIT_VALGRIND to point to
the 't/valgrind/' directory and use the commands under
't/valgrind/bin/'.
--tee::
In addition to printing the test output to the terminal,
write it to files named 't/test-results/$TEST_NAME.out'.
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
--with-dashes::
By default tests are run without dashed forms of
commands (like git-commit) in the PATH (it only uses
wrappers from ../bin-wrappers). Use this option to include
the build directory (..) in the PATH, which contains all
the dashed forms of commands. This option is currently
implied by other options like --valgrind and
GIT_TEST_INSTALLED.
--root=<directory>::
Create "trash" directories used to store all temporary data during
testing under <directory>, instead of the t/ directory.
Using this option with a RAM-based filesystem (such as tmpfs)
can massively speed up the test suite.
--chain-lint::
--no-chain-lint::
If --chain-lint is enabled, the test harness will check each
test to make sure that it properly "&&-chains" all commands (so
that a failure in the middle does not go unnoticed by the final
exit code of the test). This check is performed in addition to
running the tests themselves.
You can also set the GIT_TEST_INSTALLED environment variable to
the bindir of an existing git installation to test that installation.
You still need to have built this git sandbox, from which various
test-* support programs, templates, and perl libraries are used.
If your installed git is incomplete, it will silently test parts of
your built version instead.
When using GIT_TEST_INSTALLED, you can also set GIT_TEST_EXEC_PATH to
override the location of the dashed-form subcommands (what
GIT_EXEC_PATH would be used for during normal operation).
GIT_TEST_EXEC_PATH defaults to `$GIT_TEST_INSTALLED/git --exec-path`.
Skipping Tests
--------------
In some environments, certain tests have no way of succeeding
due to platform limitation, such as lack of 'unzip' program, or
filesystem that do not allow arbitrary sequence of non-NUL bytes
as pathnames.
You should be able to say something like
$ GIT_SKIP_TESTS=t9200.8 sh ./t9200-git-cvsexport-commit.sh
and even:
$ GIT_SKIP_TESTS='t[0-4]??? t91?? t9200.8' make
to omit such tests. The value of the environment variable is a
SP separated list of patterns that tells which tests to skip,
and either can match the "t[0-9]{4}" part to skip the whole
test, or t[0-9]{4} followed by ".$number" to say which
particular test to skip.
Note that some tests in the existing test suite rely on previous
test item, so you cannot arbitrarily disable one and expect the
remainder of test to check what the test originally was intended
to check.
Naming Tests
------------
The test files are named as:
tNNNN-commandname-details.sh
where N is a decimal digit.
First digit tells the family:
0 - the absolute basics and global stuff
1 - the basic commands concerning database
2 - the basic commands concerning the working tree
3 - the other basic commands (e.g. ls-files)
4 - the diff commands
5 - the pull and exporting commands
6 - the revision tree commands (even e.g. merge-base)
7 - the porcelainish commands concerning the working tree
8 - the porcelainish commands concerning forensics
9 - the git tools
Second digit tells the particular command we are testing.
Third digit (optionally) tells the particular switch or group of switches
we are testing.
If you create files under t/ directory (i.e. here) that is not
the top-level test script, never name the file to match the above
pattern. The Makefile here considers all such files as the
top-level test script and tries to run all of them. Care is
especially needed if you are creating a common test library
file, similar to test-lib.sh, because such a library file may
not be suitable for standalone execution.
Writing Tests
-------------
The test script is written as a shell script. It should start
with the standard "#!/bin/sh" with copyright notices, and an
assignment to variable 'test_description', like this:
#!/bin/sh
#
# Copyright (c) 2005 Junio C Hamano
#
test_description='xxx test (option --frotz)
This test registers the following structure in the cache
and tries to run git-ls-files with option --frotz.'
Source 'test-lib.sh'
--------------------
After assigning test_description, the test script should source
test-lib.sh like this:
. ./test-lib.sh
This test harness library does the following things:
- If the script is invoked with command line argument --help
(or -h), it shows the test_description and exits.
- Creates an empty test directory with an empty .git/objects database
and chdir(2) into it. This directory is 't/trash
directory.$test_name_without_dotsh', with t/ subject to change by
the --root option documented above.
- Defines standard test helper functions for your scripts to
use. These functions are designed to make all scripts behave
consistently when command line arguments --verbose (or -v),
--debug (or -d), and --immediate (or -i) is given.
Do's, don'ts & things to keep in mind
-------------------------------------
Here are a few examples of things you probably should and shouldn't do
when writing tests.
Do:
- Put all code inside test_expect_success and other assertions.
Even code that isn't a test per se, but merely some setup code
should be inside a test assertion.
- Chain your test assertions
Write test code like this:
git merge foo &&
git push bar &&
test ...
Instead of:
git merge hla
git push gh
test ...
That way all of the commands in your tests will succeed or fail. If
you must ignore the return value of something, consider using a
helper function (e.g. use sane_unset instead of unset, in order
to avoid unportable return value for unsetting a variable that was
already unset), or prepending the command with test_might_fail or
test_must_fail.
- Check the test coverage for your tests. See the "Test coverage"
below.
Don't blindly follow test coverage metrics; if a new function you added
doesn't have any coverage, then you're probably doing something wrong,
but having 100% coverage doesn't necessarily mean that you tested
everything.
Tests that are likely to smoke out future regressions are better
than tests that just inflate the coverage metrics.
- When a test checks for an absolute path that a git command generated,
construct the expected value using $(pwd) rather than $PWD,
$TEST_DIRECTORY, or $TRASH_DIRECTORY. It makes a difference on
Windows, where the shell (MSYS bash) mangles absolute path names.
For details, see the commit message of 4114156ae9.
Don't:
- exit() within a <script> part.
The harness will catch this as a programming error of the test.
Use test_done instead if you need to stop the tests early (see
"Skipping tests" below).
- use '! git cmd' when you want to make sure the git command exits
with failure in a controlled way by calling "die()". Instead,
use 'test_must_fail git cmd'. This will signal a failure if git
dies in an unexpected way (e.g. segfault).
- use perl without spelling it as "$PERL_PATH". This is to help our
friends on Windows where the platform Perl often adds CR before
the end of line, and they bundle Git with a version of Perl that
does not do so, whose path is specified with $PERL_PATH.
- use sh without spelling it as "$SHELL_PATH", when the script can
be misinterpreted by broken platform shell (e.g. Solaris).
- chdir around in tests. It is not sufficient to chdir to
somewhere and then chdir back to the original location later in
the test, as any intermediate step can fail and abort the test,
causing the next test to start in an unexpected directory. Do so
inside a subshell if necessary.
- Break the TAP output
The raw output from your test may be interpreted by a TAP harness. TAP
harnesses will ignore everything they don't know about, but don't step
on their toes in these areas:
- Don't print lines like "$x..$y" where $x and $y are integers.
- Don't print lines that begin with "ok" or "not ok".
TAP harnesses expect a line that begins with either "ok" and "not
ok" to signal a test passed or failed (and our harness already
produces such lines), so your script shouldn't emit such lines to
their output.
You can glean some further possible issues from the TAP grammar
(see http://search.cpan.org/perldoc?TAP::Parser::Grammar#TAP_Grammar)
but the best indication is to just run the tests with prove(1),
it'll complain if anything is amiss.
Keep in mind:
- Inside <script> part, the standard output and standard error
streams are discarded, and the test harness only reports "ok" or
"not ok" to the end user running the tests. Under --verbose, they
are shown to help debugging the tests.
Skipping tests
--------------
If you need to skip tests you should do so by using the three-arg form
of the test_* functions (see the "Test harness library" section
below), e.g.:
test_expect_success PERL 'I need Perl' '
"$PERL_PATH" -e "hlagh() if unf_unf()"
'
The advantage of skipping tests like this is that platforms that don't
have the PERL and other optional dependencies get an indication of how
many tests they're missing.
If the test code is too hairy for that (i.e. does a lot of setup work
outside test assertions) you can also skip all remaining tests by
setting skip_all and immediately call test_done:
if ! test_have_prereq PERL
then
skip_all='skipping perl interface tests, perl not available'
test_done
fi
The string you give to skip_all will be used as an explanation for why
the test was skipped.
End with test_done
------------------
Your script will be a sequence of tests, using helper functions
from the test harness library. At the end of the script, call
'test_done'.
Test harness library
--------------------
There are a handful helper functions defined in the test harness
library for your script to use.
- test_expect_success [<prereq>] <message> <script>
Usually takes two strings as parameters, and evaluates the
<script>. If it yields success, test is considered
successful. <message> should state what it is testing.
Example:
test_expect_success \
'git-write-tree should be able to write an empty tree.' \
'tree=$(git-write-tree)'
If you supply three parameters the first will be taken to be a
prerequisite; see the test_set_prereq and test_have_prereq
documentation below:
test_expect_success TTY 'git --paginate rev-list uses a pager' \
' ... '
You can also supply a comma-separated list of prerequisites, in the
rare case where your test depends on more than one:
test_expect_success PERL,PYTHON 'yo dawg' \
' test $(perl -E 'print eval "1 +" . qx[python -c "print 2"]') == "4" '
- test_expect_failure [<prereq>] <message> <script>
This is NOT the opposite of test_expect_success, but is used
to mark a test that demonstrates a known breakage. Unlike
the usual test_expect_success tests, which say "ok" on
success and "FAIL" on failure, this will say "FIXED" on
success and "still broken" on failure. Failures from these
tests won't cause -i (immediate) to stop.
Like test_expect_success this function can optionally use a three
argument invocation with a prerequisite as the first argument.
- test_debug <script>
This takes a single argument, <script>, and evaluates it only
when the test script is started with --debug command line
argument. This is primarily meant for use during the
development of a new test script.
- test_done
Your test script must have test_done at the end. Its purpose
is to summarize successes and failures in the test script and
exit with an appropriate error code.
- test_tick
Make commit and tag names consistent by setting the author and
committer times to defined state. Subsequent calls will
advance the times by a fixed amount.
- test_commit <message> [<filename> [<contents>]]
Creates a commit with the given message, committing the given
file with the given contents (default for both is to reuse the
message string), and adds a tag (again reusing the message
string as name). Calls test_tick to make the SHA-1s
reproducible.
- test_merge <message> <commit-or-tag>
Merges the given rev using the given message. Like test_commit,
creates a tag and calls test_tick before committing.
- test_set_prereq <prereq>
Set a test prerequisite to be used later with test_have_prereq. The
test-lib will set some prerequisites for you, see the
"Prerequisites" section below for a full list of these.
Others you can set yourself and use later with either
test_have_prereq directly, or the three argument invocation of
test_expect_success and test_expect_failure.
- test_have_prereq <prereq>
Check if we have a prerequisite previously set with
test_set_prereq. The most common use of this directly is to skip
all the tests if we don't have some essential prerequisite:
if ! test_have_prereq PERL
then
skip_all='skipping perl interface tests, perl not available'
test_done
fi
- test_external [<prereq>] <message> <external> <script>
Execute a <script> with an <external> interpreter (like perl). This
was added for tests like t9700-perl-git.sh which do most of their
work in an external test script.
test_external \
'GitwebCache::*FileCache*' \
"$PERL_PATH" "$TEST_DIRECTORY"/t9503/test_cache_interface.pl
If the test is outputting its own TAP you should set the
test_external_has_tap variable somewhere before calling the first
test_external* function. See t9700-perl-git.sh for an example.
# The external test will outputs its own plan
test_external_has_tap=1
- test_external_without_stderr [<prereq>] <message> <external> <script>
Like test_external but fail if there's any output on stderr,
instead of checking the exit code.
test_external_without_stderr \
'Perl API' \
"$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl
- test_expect_code <exit-code> <command>
Run a command and ensure that it exits with the given exit code.
For example:
test_expect_success 'Merge with d/f conflicts' '
test_expect_code 1 git merge "merge msg" B master
'
- test_must_fail <git-command>
Run a git command and ensure it fails in a controlled way. Use
this instead of "! <git-command>". When git-command dies due to a
segfault, test_must_fail diagnoses it as an error; "! <git-command>"
treats it as just another expected failure, which would let such a
bug go unnoticed.
- test_might_fail <git-command>
Similar to test_must_fail, but tolerate success, too. Use this
instead of "<git-command> || :" to catch failures due to segv.
- test_cmp <expected> <actual>
Check whether the content of the <actual> file matches the
<expected> file. This behaves like "cmp" but produces more
helpful output when the test is run with "-v" option.
- test_line_count (= | -lt | -ge | ...) <length> <file>
Check whether a file has the length it is expected to.
- test_path_is_file <path> [<diagnosis>]
test_path_is_dir <path> [<diagnosis>]
test_path_is_missing <path> [<diagnosis>]
Check if the named path is a file, if the named path is a
directory, or if the named path does not exist, respectively,
and fail otherwise, showing the <diagnosis> text.
- test_when_finished <script>
Prepend <script> to a list of commands to run to clean up
at the end of the current test. If some clean-up command
fails, the test will not pass.
Example:
test_expect_success 'branch pointing to non-commit' '
git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
test_when_finished "git update-ref -d refs/heads/invalid" &&
...
'
- test_pause
This command is useful for writing and debugging tests and must be
removed before submitting. It halts the execution of the test and
spawns a shell in the trash directory. Exit the shell to continue
the test. Example:
test_expect_success 'test' '
git do-something >actual &&
test_pause &&
test_cmp expected actual
'
Prerequisites
-------------
These are the prerequisites that the test library predefines with
test_have_prereq.
See the prereq argument to the test_* functions in the "Test harness
library" section above and the "test_have_prereq" function for how to
use these, and "test_set_prereq" for how to define your own.
- PERL & PYTHON
Git wasn't compiled with NO_PERL=YesPlease or
NO_PYTHON=YesPlease. Wrap any tests that need Perl or Python in
these.
- POSIXPERM
The filesystem supports POSIX style permission bits.
- BSLASHPSPEC
Backslashes in pathspec are not directory separators. This is not
set on Windows. See 6fd1106a for details.
- EXECKEEPSPID
The process retains the same pid across exec(2). See fb9a2bea for
details.
- SYMLINKS
The filesystem we're on supports symbolic links. E.g. a FAT
filesystem doesn't support these. See 704a3143 for details.
- SANITY
Test is not run by root user, and an attempt to write to an
unwritable file is expected to fail correctly.
- LIBPCRE
Git was compiled with USE_LIBPCRE=YesPlease. Wrap any tests
that use git-grep --perl-regexp or git-grep -P in these.
- CASE_INSENSITIVE_FS
Test is run on a case insensitive file system.
- UTF8_NFD_TO_NFC
Test is run on a filesystem which converts decomposed utf-8 (nfd)
to precomposed utf-8 (nfc).
Tips for Writing Tests
----------------------
As with any programming projects, existing programs are the best
source of the information. However, do _not_ emulate
t0000-basic.sh when writing your tests. The test is special in
that it tries to validate the very core of GIT. For example, it
knows that there will be 256 subdirectories under .git/objects/,
and it knows that the object ID of an empty tree is a certain
40-byte string. This is deliberately done so in t0000-basic.sh
because the things the very basic core test tries to achieve is
to serve as a basis for people who are changing the GIT internal
drastically. For these people, after making certain changes,
not seeing failures from the basic test _is_ a failure. And
such drastic changes to the core GIT that even changes these
otherwise supposedly stable object IDs should be accompanied by
an update to t0000-basic.sh.
However, other tests that simply rely on basic parts of the core
GIT working properly should not have that level of intimate
knowledge of the core GIT internals. If all the test scripts
hardcoded the object IDs like t0000-basic.sh does, that defeats
the purpose of t0000-basic.sh, which is to isolate that level of
validation in one place. Your test also ends up needing
updating when such a change to the internal happens, so do _not_
do it and leave the low level of validation to t0000-basic.sh.
Test coverage
-------------
You can use the coverage tests to find code paths that are not being
used or properly exercised yet.
To do that, run the coverage target at the top-level (not in the t/
directory):
make coverage
That'll compile Git with GCC's coverage arguments, and generate a test
report with gcov after the tests finish. Running the coverage tests
can take a while, since running the tests in parallel is incompatible
with GCC's coverage mode.
After the tests have run you can generate a list of untested
functions:
make coverage-untested-functions
You can also generate a detailed per-file HTML report using the
Devel::Cover module. To install it do:
# On Debian or Ubuntu:
sudo aptitude install libdevel-cover-perl
# From the CPAN with cpanminus
curl -L http://cpanmin.us | perl - --sudo --self-upgrade
cpanm --sudo Devel::Cover
Then, at the top-level:
make cover_db_html
That'll generate a detailed cover report in the "cover_db_html"
directory, which you can then copy to a webserver, or inspect locally
in a browser.

164
README.md Normal file
View File

@ -0,0 +1,164 @@
# Sharness
Sharness is a portable shell library to write, run, and analyze automated tests
for Unix programs. Since all tests output TAP, the [Test Anything Protocol],
they can be run with any TAP harness.
Each test is written as a shell script, for example:
```sh
#!/bin/sh
test_description="Show basic features of Sharness"
. ./sharness.sh
test_expect_success "Success is reported like this" "
echo hello world | grep hello
"
test_expect_success "Commands are chained this way" "
test x = 'x' &&
test 2 -gt 1 &&
echo success
"
return_42() {
echo "Will return soon"
return 42
}
test_expect_success "You can test for a specific exit code" "
test_expect_code 42 return_42
"
test_expect_failure "We expect this to fail" "
test 1 = 2
"
test_done
```
Running the above test script returns the following (TAP) output:
$ ./simple.t
ok 1 - Success is reported like this
ok 2 - Commands are chained this way
ok 3 - You can test for a specific exit code
not ok 4 - We expect this to fail # TODO known breakage
# still have 1 known breakage(s)
# passed all remaining 3 test(s)
1..4
Alternatively, you can run the test through [prove(1)]:
$ prove simple.t
simple.t .. ok
All tests successful.
Files=1, Tests=4, 0 wallclock secs ( 0.02 usr + 0.00 sys = 0.02 CPU)
Result: PASS
Sharness was derived from the [Git] project - see [README.git] for the original
documentation.
## Installation
First, clone the Git repository:
$ git clone git://github.com/mlafeldt/sharness.git
Then choose an installation method that works best for you:
### Per-project installation
If you like to add Sharness to the sources of a project you want to use it for,
simply copy the files `sharness.sh`, `aggregate-results.sh`, and `test/Makefile`
to a folder named `test` inside that project.
Alternatively, you can also add Sharness as a Git submodule to your project.
### Per-user installation
$ cd sharness
$ make install
This will install Sharness to `$HOME/share/sharness`, and its documentation and
examples to `$HOME/share/doc/sharness`.
### System-wide installation
$ cd sharness
# make install prefix=/usr/local
This will install Sharness to `/usr/local/share/sharness`, and its documentation
and examples to `/usr/local/share/doc/sharness`.
Of course, you can change the _prefix_ parameter to install Sharness to any
other location.
### Installation via Chef
If you want to install Sharness with Opscode Chef, the [Sharness cookbook] is
for you.
## Usage
The following files are essential to using Sharness:
* `sharness.sh` - core shell library providing test functionality, see separate
[API documentation]
* `aggregate-results.sh` - helper script to aggregate test results
* `test/Makefile` - test driver
To learn how to write and run actual test scripts based on `sharness.sh`, please
read [README.git] until I come up with more documentation myself.
## Projects using Sharness
See how Sharness is used in real-world projects:
* [cb2util](https://github.com/mlafeldt/cb2util/tree/master/test)
* [dabba](https://github.com/eroullit/dabba/tree/master/dabba/test)
* [git-integration](https://github.com/johnkeeping/git-integration/tree/master/t)
* [go-ipfs](https://github.com/jbenet/go-ipfs/tree/master/test/sharness)
* [go-multihash](https://github.com/jbenet/go-multihash/tree/master/test/sharness)
* [rdd.py](https://github.com/mlafeldt/rdd.py/tree/master/test/integration)
* [Sharness itself](/test)
* [tomdoc.sh](https://github.com/mlafeldt/tomdoc.sh/tree/master/test)
Furthermore, as Sharness was derived from Git, [Git's test suite](https://github.com/git/git/tree/master/t)
is worth examining as well, especially if you're interested in managing a big
number of tests.
## Alternatives
Here is a list of other shell testing libraries (sorted alphabetically):
* [Bats](https://github.com/sstephenson/bats)
* [Cram](https://bitheap.org/cram)
* [rnt](https://github.com/roman-neuhauser/rnt)
* [roundup](https://github.com/bmizerany/roundup)
* [shUnit2](https://code.google.com/p/shunit2/)
* [shspec](https://github.com/shpec/shpec)
* [testlib.sh](https://gist.github.com/3877539)
* [ts](https://github.com/thinkerbot/ts)
## License
Sharness is licensed under the terms of the GNU General Public License version
2 or higher. See file [COPYING] for full license text.
## Author
Sharness is being developed by [Mathias Lafeldt][twitter]. The library was
derived from the [Git] project.
[API documentation]: https://github.com/mlafeldt/sharness/blob/master/API.md
[COPYING]: https://github.com/mlafeldt/sharness/blob/master/COPYING
[Git]: http://git-scm.com/
[prove(1)]: http://linux.die.net/man/1/prove
[README.git]: https://github.com/mlafeldt/sharness/blob/master/README.git
[Sharness cookbook]: https://github.com/mlafeldt/sharness-cookbook
[Test Anything Protocol]: http://testanything.org/
[twitter]: https://twitter.com/mlafeldt

57
aggregate-results.sh Executable file
View File

@ -0,0 +1,57 @@
#!/bin/sh
#
# Copyright (c) 2008-2012 Git project
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/ .
failed_tests=
fixed=0
success=0
failed=0
broken=0
total=0
while read file; do
while read type value; do
case $type in
'')
continue ;;
fixed)
fixed=$(($fixed + $value)) ;;
success)
success=$(($success + $value)) ;;
failed)
failed=$(($failed + $value))
if test $value != 0; then
test_name=$(expr "$file" : 'test-results/\(.*\)\.[0-9]*\.counts')
failed_tests="$failed_tests $test_name"
fi
;;
broken)
broken=$(($broken + $value)) ;;
total)
total=$(($total + $value)) ;;
esac
done <"$file"
done
if test -n "$failed_tests"; then
printf "\nfailed test(s):$failed_tests\n\n"
fi
printf "%-8s%d\n" fixed $fixed
printf "%-8s%d\n" success $success
printf "%-8s%d\n" failed $failed
printf "%-8s%d\n" broken $broken
printf "%-8s%d\n" total $total

741
sharness.sh Normal file
View File

@ -0,0 +1,741 @@
#!/bin/sh
#
# Copyright (c) 2011-2012 Mathias Lafeldt
# Copyright (c) 2005-2012 Git project
# Copyright (c) 2005-2012 Junio C Hamano
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/ .
# Public: Current version of Sharness.
SHARNESS_VERSION="0.3.0"
export SHARNESS_VERSION
# Public: The file extension for tests. By default, it is set to "t".
: ${SHARNESS_TEST_EXTENSION:=t}
export SHARNESS_TEST_EXTENSION
# Keep the original TERM for say_color
ORIGINAL_TERM=$TERM
# For repeatability, reset the environment to a known state.
LANG=C
LC_ALL=C
PAGER=cat
TZ=UTC
TERM=dumb
EDITOR=:
export LANG LC_ALL PAGER TZ TERM EDITOR
unset VISUAL CDPATH GREP_OPTIONS
# Line feed
LF='
'
[ "x$ORIGINAL_TERM" != "xdumb" ] && (
TERM=$ORIGINAL_TERM &&
export TERM &&
[ -t 1 ] &&
tput bold >/dev/null 2>&1 &&
tput setaf 1 >/dev/null 2>&1 &&
tput sgr0 >/dev/null 2>&1
) &&
color=t
while test "$#" -ne 0; do
case "$1" in
-d|--d|--de|--deb|--debu|--debug)
debug=t; shift ;;
-i|--i|--im|--imm|--imme|--immed|--immedi|--immedia|--immediat|--immediate)
immediate=t; shift ;;
-l|--l|--lo|--lon|--long|--long-|--long-t|--long-te|--long-tes|--long-test|--long-tests)
TEST_LONG=t; export TEST_LONG; shift ;;
-h|--h|--he|--hel|--help)
help=t; shift ;;
-v|--v|--ve|--ver|--verb|--verbo|--verbos|--verbose)
verbose=t; shift ;;
-q|--q|--qu|--qui|--quie|--quiet)
# Ignore --quiet under a TAP::Harness. Saying how many tests
# passed without the ok/not ok details is always an error.
test -z "$HARNESS_ACTIVE" && quiet=t; shift ;;
--chain-lint)
chain_lint=t; shift ;;
--no-chain-lint)
chain_lint=; shift ;;
--no-color)
color=; shift ;;
--root=*)
root=$(expr "z$1" : 'z[^=]*=\(.*\)')
shift ;;
*)
echo "error: unknown test option '$1'" >&2; exit 1 ;;
esac
done
if test -n "$color"; then
say_color() {
(
TERM=$ORIGINAL_TERM
export TERM
case "$1" in
error)
tput bold; tput setaf 1;; # bold red
skip)
tput setaf 4;; # blue
warn)
tput setaf 3;; # brown/yellow
pass)
tput setaf 2;; # green
info)
tput setaf 6;; # cyan
*)
test -n "$quiet" && return;;
esac
shift
printf "%s" "$*"
tput sgr0
echo
)
}
else
say_color() {
test -z "$1" && test -n "$quiet" && return
shift
printf "%s\n" "$*"
}
fi
error() {
say_color error "error: $*"
EXIT_OK=t
exit 1
}
say() {
say_color info "$*"
}
test -n "$test_description" || error "Test script did not set test_description."
if test "$help" = "t"; then
echo "$test_description"
exit 0
fi
exec 5>&1
exec 6<&0
if test "$verbose" = "t"; then
exec 4>&2 3>&1
else
exec 4>/dev/null 3>/dev/null
fi
test_failure=0
test_count=0
test_fixed=0
test_broken=0
test_success=0
die() {
code=$?
if test -n "$EXIT_OK"; then
exit $code
else
echo >&5 "FATAL: Unexpected exit with code $code"
exit 1
fi
}
EXIT_OK=
trap 'die' EXIT
# Public: Define that a test prerequisite is available.
#
# The prerequisite can later be checked explicitly using test_have_prereq or
# implicitly by specifying the prerequisite name in calls to test_expect_success
# or test_expect_failure.
#
# $1 - Name of prerequiste (a simple word, in all capital letters by convention)
#
# Examples
#
# # Set PYTHON prerequisite if interpreter is available.
# command -v python >/dev/null && test_set_prereq PYTHON
#
# # Set prerequisite depending on some variable.
# test -z "$NO_GETTEXT" && test_set_prereq GETTEXT
#
# Returns nothing.
test_set_prereq() {
satisfied_prereq="$satisfied_prereq$1 "
}
satisfied_prereq=" "
# Public: Check if one or more test prerequisites are defined.
#
# The prerequisites must have previously been set with test_set_prereq.
# The most common use of this is to skip all the tests if some essential
# prerequisite is missing.
#
# $1 - Comma-separated list of test prerequisites.
#
# Examples
#
# # Skip all remaining tests if prerequisite is not set.
# if ! test_have_prereq PERL; then
# skip_all='skipping perl interface tests, perl not available'
# test_done
# fi
#
# Returns 0 if all prerequisites are defined or 1 otherwise.
test_have_prereq() {
# prerequisites can be concatenated with ','
save_IFS=$IFS
IFS=,
set -- $*
IFS=$save_IFS
total_prereq=0
ok_prereq=0
missing_prereq=
for prerequisite; do
case "$prerequisite" in
!*)
negative_prereq=t
prerequisite=${prerequisite#!}
;;
*)
negative_prereq=
esac
total_prereq=$(($total_prereq + 1))
case "$satisfied_prereq" in
*" $prerequisite "*)
satisfied_this_prereq=t
;;
*)
satisfied_this_prereq=
esac
case "$satisfied_this_prereq,$negative_prereq" in
t,|,t)
ok_prereq=$(($ok_prereq + 1))
;;
*)
# Keep a list of missing prerequisites; restore
# the negative marker if necessary.
prerequisite=${negative_prereq:+!}$prerequisite
if test -z "$missing_prereq"; then
missing_prereq=$prerequisite
else
missing_prereq="$prerequisite,$missing_prereq"
fi
esac
done
test $total_prereq = $ok_prereq
}
# You are not expected to call test_ok_ and test_failure_ directly, use
# the text_expect_* functions instead.
test_ok_() {
test_success=$(($test_success + 1))
say_color "" "ok $test_count - $@"
}
test_failure_() {
test_failure=$(($test_failure + 1))
say_color error "not ok $test_count - $1"
shift
echo "$@" | sed -e 's/^/# /'
test "$immediate" = "" || { EXIT_OK=t; exit 1; }
}
test_known_broken_ok_() {
test_fixed=$(($test_fixed + 1))
say_color error "ok $test_count - $@ # TODO known breakage vanished"
}
test_known_broken_failure_() {
test_broken=$(($test_broken + 1))
say_color warn "not ok $test_count - $@ # TODO known breakage"
}
# Public: Execute commands in debug mode.
#
# Takes a single argument and evaluates it only when the test script is started
# with --debug. This is primarily meant for use during the development of test
# scripts.
#
# $1 - Commands to be executed.
#
# Examples
#
# test_debug "cat some_log_file"
#
# Returns the exit code of the last command executed in debug mode or 0
# otherwise.
test_debug() {
test "$debug" = "" || eval "$1"
}
test_eval_() {
# This is a separate function because some tests use
# "return" to end a test_expect_success block early.
eval </dev/null >&3 2>&4 "$*"
}
test_run_() {
test_cleanup=:
expecting_failure=$2
test_eval_ "$1"
eval_ret=$?
if test "$chain_lint" = "t"; then
test_eval_ "(exit 117) && $1"
if test "$?" != 117; then
error "bug in the test script: broken &&-chain: $1"
fi
fi
if test -z "$immediate" || test $eval_ret = 0 || test -n "$expecting_failure"; then
test_eval_ "$test_cleanup"
fi
if test "$verbose" = "t" && test -n "$HARNESS_ACTIVE"; then
echo ""
fi
return "$eval_ret"
}
test_skip_() {
test_count=$(($test_count + 1))
to_skip=
for skp in $SKIP_TESTS; do
case $this_test.$test_count in
$skp)
to_skip=t
break
esac
done
if test -z "$to_skip" && test -n "$test_prereq" && ! test_have_prereq "$test_prereq"; then
to_skip=t
fi
case "$to_skip" in
t)
of_prereq=
if test "$missing_prereq" != "$test_prereq"; then
of_prereq=" of $test_prereq"
fi
say_color skip >&3 "skipping test: $@"
say_color skip "ok $test_count # skip $1 (missing $missing_prereq${of_prereq})"
: true
;;
*)
false
;;
esac
}
# Public: Run test commands and expect them to succeed.
#
# When the test passed, an "ok" message is printed and the number of successful
# tests is incremented. When it failed, a "not ok" message is printed and the
# number of failed tests is incremented.
#
# With --immediate, exit test immediately upon the first failed test.
#
# Usually takes two arguments:
# $1 - Test description
# $2 - Commands to be executed.
#
# With three arguments, the first will be taken to be a prerequisite:
# $1 - Comma-separated list of test prerequisites. The test will be skipped if
# not all of the given prerequisites are set. To negate a prerequisite,
# put a "!" in front of it.
# $2 - Test description
# $3 - Commands to be executed.
#
# Examples
#
# test_expect_success \
# 'git-write-tree should be able to write an empty tree.' \
# 'tree=$(git-write-tree)'
#
# # Test depending on one prerequisite.
# test_expect_success TTY 'git --paginate rev-list uses a pager' \
# ' ... '
#
# # Multiple prerequisites are separated by a comma.
# test_expect_success PERL,PYTHON 'yo dawg' \
# ' test $(perl -E 'print eval "1 +" . qx[python -c "print 2"]') == "4" '
#
# Returns nothing.
test_expect_success() {
test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
test "$#" = 2 || error "bug in the test script: not 2 or 3 parameters to test_expect_success"
export test_prereq
if ! test_skip_ "$@"; then
say >&3 "expecting success: $2"
if test_run_ "$2"; then
test_ok_ "$1"
else
test_failure_ "$@"
fi
fi
echo >&3 ""
}
# Public: Run test commands and expect them to fail. Used to demonstrate a known
# breakage.
#
# This is NOT the opposite of test_expect_success, but rather used to mark a
# test that demonstrates a known breakage.
#
# When the test passed, an "ok" message is printed and the number of fixed tests
# is incremented. When it failed, a "not ok" message is printed and the number
# of tests still broken is incremented.
#
# Failures from these tests won't cause --immediate to stop.
#
# Usually takes two arguments:
# $1 - Test description
# $2 - Commands to be executed.
#
# With three arguments, the first will be taken to be a prerequisite:
# $1 - Comma-separated list of test prerequisites. The test will be skipped if
# not all of the given prerequisites are set. To negate a prerequisite,
# put a "!" in front of it.
# $2 - Test description
# $3 - Commands to be executed.
#
# Returns nothing.
test_expect_failure() {
test "$#" = 3 && { test_prereq=$1; shift; } || test_prereq=
test "$#" = 2 || error "bug in the test script: not 2 or 3 parameters to test_expect_failure"
export test_prereq
if ! test_skip_ "$@"; then
say >&3 "checking known breakage: $2"
if test_run_ "$2" expecting_failure; then
test_known_broken_ok_ "$1"
else
test_known_broken_failure_ "$1"
fi
fi
echo >&3 ""
}
# Public: Run command and ensure that it fails in a controlled way.
#
# Use it instead of "! <command>". For example, when <command> dies due to a
# segfault, test_must_fail diagnoses it as an error, while "! <command>" would
# mistakenly be treated as just another expected failure.
#
# This is one of the prefix functions to be used inside test_expect_success or
# test_expect_failure.
#
# $1.. - Command to be executed.
#
# Examples
#
# test_expect_success 'complain and die' '
# do something &&
# do something else &&
# test_must_fail git checkout ../outerspace
# '
#
# Returns 1 if the command succeeded (exit code 0).
# Returns 1 if the command died by signal (exit codes 130-192)
# Returns 1 if the command could not be found (exit code 127).
# Returns 0 otherwise.
test_must_fail() {
"$@"
exit_code=$?
if test $exit_code = 0; then
echo >&2 "test_must_fail: command succeeded: $*"
return 1
elif test $exit_code -gt 129 -a $exit_code -le 192; then
echo >&2 "test_must_fail: died by signal: $*"
return 1
elif test $exit_code = 127; then
echo >&2 "test_must_fail: command not found: $*"
return 1
fi
return 0
}
# Public: Run command and ensure that it succeeds or fails in a controlled way.
#
# Similar to test_must_fail, but tolerates success too. Use it instead of
# "<command> || :" to catch failures caused by a segfault, for instance.
#
# This is one of the prefix functions to be used inside test_expect_success or
# test_expect_failure.
#
# $1.. - Command to be executed.
#
# Examples
#
# test_expect_success 'some command works without configuration' '
# test_might_fail git config --unset all.configuration &&
# do something
# '
#
# Returns 1 if the command died by signal (exit codes 130-192)
# Returns 1 if the command could not be found (exit code 127).
# Returns 0 otherwise.
test_might_fail() {
"$@"
exit_code=$?
if test $exit_code -gt 129 -a $exit_code -le 192; then
echo >&2 "test_might_fail: died by signal: $*"
return 1
elif test $exit_code = 127; then
echo >&2 "test_might_fail: command not found: $*"
return 1
fi
return 0
}
# Public: Run command and ensure it exits with a given exit code.
#
# This is one of the prefix functions to be used inside test_expect_success or
# test_expect_failure.
#
# $1 - Expected exit code.
# $2.. - Command to be executed.
#
# Examples
#
# test_expect_success 'Merge with d/f conflicts' '
# test_expect_code 1 git merge "merge msg" B master
# '
#
# Returns 0 if the expected exit code is returned or 1 otherwise.
test_expect_code() {
want_code=$1
shift
"$@"
exit_code=$?
if test $exit_code = $want_code; then
return 0
fi
echo >&2 "test_expect_code: command exited with $exit_code, we wanted $want_code $*"
return 1
}
# Public: Compare two files to see if expected output matches actual output.
#
# The TEST_CMP variable defines the command used for the comparision; it
# defaults to "diff -u". Only when the test script was started with --verbose,
# will the command's output, the diff, be printed to the standard output.
#
# This is one of the prefix functions to be used inside test_expect_success or
# test_expect_failure.
#
# $1 - Path to file with expected output.
# $2 - Path to file with actual output.
#
# Examples
#
# test_expect_success 'foo works' '
# echo expected >expected &&
# foo >actual &&
# test_cmp expected actual
# '
#
# Returns the exit code of the command set by TEST_CMP.
test_cmp() {
${TEST_CMP:-diff -u} "$@"
}
# Public: Schedule cleanup commands to be run unconditionally at the end of a
# test.
#
# If some cleanup command fails, the test will not pass. With --immediate, no
# cleanup is done to help diagnose what went wrong.
#
# This is one of the prefix functions to be used inside test_expect_success or
# test_expect_failure.
#
# $1.. - Commands to prepend to the list of cleanup commands.
#
# Examples
#
# test_expect_success 'test core.capslock' '
# git config core.capslock true &&
# test_when_finished "git config --unset core.capslock" &&
# do_something
# '
#
# Returns the exit code of the last cleanup command executed.
test_when_finished() {
test_cleanup="{ $*
} && (exit \"\$eval_ret\"); eval_ret=\$?; $test_cleanup"
}
# Public: Schedule cleanup commands to be run unconditionally when all tests
# have run.
#
# This can be used to clean up things like test databases. It is not needed to
# clean up temporary files, as test_done already does that.
#
# Examples:
#
# cleanup mysql -e "DROP DATABASE mytest"
#
# Returns the exit code of the last cleanup command executed.
final_cleanup=
cleanup() {
final_cleanup="{ $*
} && (exit \"\$eval_ret\"); eval_ret=\$?; $final_cleanup"
}
# Public: Summarize test results and exit with an appropriate error code.
#
# Must be called at the end of each test script.
#
# Can also be used to stop tests early and skip all remaining tests. For this,
# set skip_all to a string explaining why the tests were skipped before calling
# test_done.
#
# Examples
#
# # Each test script must call test_done at the end.
# test_done
#
# # Skip all remaining tests if prerequisite is not set.
# if ! test_have_prereq PERL; then
# skip_all='skipping perl interface tests, perl not available'
# test_done
# fi
#
# Returns 0 if all tests passed or 1 if there was a failure.
test_done() {
EXIT_OK=t
if test -z "$HARNESS_ACTIVE"; then
test_results_dir="$SHARNESS_TEST_DIRECTORY/test-results"
mkdir -p "$test_results_dir"
test_results_path="$test_results_dir/${SHARNESS_TEST_FILE%.$SHARNESS_TEST_EXTENSION}.$$.counts"
cat >>"$test_results_path" <<-EOF
total $test_count
success $test_success
fixed $test_fixed
broken $test_broken
failed $test_failure
EOF
fi
if test "$test_fixed" != 0; then
say_color error "# $test_fixed known breakage(s) vanished; please update test(s)"
fi
if test "$test_broken" != 0; then
say_color warn "# still have $test_broken known breakage(s)"
fi
if test "$test_broken" != 0 || test "$test_fixed" != 0; then
test_remaining=$(( $test_count - $test_broken - $test_fixed ))
msg="remaining $test_remaining test(s)"
else
test_remaining=$test_count
msg="$test_count test(s)"
fi
case "$test_failure" in
0)
# Maybe print SKIP message
if test -n "$skip_all" && test $test_count -gt 0; then
error "Can't use skip_all after running some tests"
fi
[ -z "$skip_all" ] || skip_all=" # SKIP $skip_all"
if test $test_remaining -gt 0; then
say_color pass "# passed all $msg"
fi
say "1..$test_count$skip_all"
test_eval_ "$final_cleanup"
test -d "$remove_trash" &&
cd "$(dirname "$remove_trash")" &&
rm -rf "$(basename "$remove_trash")"
exit 0 ;;
*)
say_color error "# failed $test_failure among $msg"
say "1..$test_count"
exit 1 ;;
esac
}
# Public: Root directory containing tests. Tests can override this variable,
# e.g. for testing Sharness itself.
: ${SHARNESS_TEST_DIRECTORY:=$(pwd)}
export SHARNESS_TEST_DIRECTORY
# Public: Build directory that will be added to PATH. By default, it is set to
# the parent directory of SHARNESS_TEST_DIRECTORY.
: ${SHARNESS_BUILD_DIRECTORY:="$SHARNESS_TEST_DIRECTORY/.."}
PATH="$SHARNESS_BUILD_DIRECTORY:$PATH"
export PATH SHARNESS_BUILD_DIRECTORY
# Public: Path to test script currently executed.
SHARNESS_TEST_FILE="$0"
export SHARNESS_TEST_FILE
# Prepare test area.
test_dir="trash directory.$(basename "$SHARNESS_TEST_FILE" ".$SHARNESS_TEST_EXTENSION")"
test -n "$root" && test_dir="$root/$test_dir"
case "$test_dir" in
/*) SHARNESS_TRASH_DIRECTORY="$test_dir" ;;
*) SHARNESS_TRASH_DIRECTORY="$SHARNESS_TEST_DIRECTORY/$test_dir" ;;
esac
test "$debug" = "t" || remove_trash="$SHARNESS_TRASH_DIRECTORY"
rm -rf "$test_dir" || {
EXIT_OK=t
echo >&5 "FATAL: Cannot prepare test area"
exit 1
}
# Public: Empty trash directory, the test area, provided for each test. The HOME
# variable is set to that directory too.
export SHARNESS_TRASH_DIRECTORY
HOME="$SHARNESS_TRASH_DIRECTORY"
export HOME
mkdir -p "$test_dir" || exit 1
# Use -P to resolve symlinks in our working directory so that the cwd
# in subprocesses like git equals our $PWD (for pathname comparisons).
cd -P "$test_dir" || exit 1
this_test=${SHARNESS_TEST_FILE##*/}
this_test=${this_test%.$SHARNESS_TEST_EXTENSION}
for skp in $SKIP_TESTS; do
case "$this_test" in
$skp)
say_color info >&3 "skipping test $this_test altogether"
skip_all="skip all tests in $this_test"
test_done
esac
done
# vi: set ts=4 sw=4 noet :

3
test/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
/trash directory*
/test-results
/.prove

60
test/Makefile Normal file
View File

@ -0,0 +1,60 @@
# Run tests
#
# Copyright (c) 2011-2012 Mathias Lafeldt
# Copyright (c) 2005-2012 Git project
# Copyright (c) 2005-2012 Junio C Hamano
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/ .
SHELL_PATH ?= $(SHELL)
SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
RM ?= rm -f
PROVE ?= prove
AGGREGATE_SCRIPT ?= aggregate-results.sh
DEFAULT_TEST_TARGET ?= test
T = $(wildcard *.t)
all: $(DEFAULT_TEST_TARGET)
test: pre-clean
$(MAKE) aggregate-results-and-cleanup
prove: pre-clean
@echo "*** prove ***"; $(PROVE) --exec '$(SHELL_PATH_SQ)' $(PROVE_OPTS) $(T) :: $(TEST_OPTS)
$(MAKE) clean-except-prove-cache
$(T):
@echo "*** $@ ***"; '$(SHELL_PATH_SQ)' $@ $(TEST_OPTS)
pre-clean:
$(RM) -r test-results
clean-except-prove-cache:
$(RM) -r 'trash directory'.* test-results
clean: clean-except-prove-cache
$(RM) .prove
aggregate-results-and-cleanup: $(T)
$(MAKE) aggregate-results
$(MAKE) clean
aggregate-results:
for f in test-results/*.counts; do \
echo "$$f"; \
done | '$(SHELL_PATH_SQ)' '$(AGGREGATE_SCRIPT)'
.PHONY: all test prove $(T) pre-clean clean
.PHONY: aggregate-results-and-cleanup aggregate-results

1
test/aggregate-results.sh Symbolic link
View File

@ -0,0 +1 @@
../aggregate-results.sh

1
test/sharness.sh Symbolic link
View File

@ -0,0 +1 @@
../sharness.sh

302
test/sharness.t Executable file
View File

@ -0,0 +1,302 @@
#!/bin/sh
#
# Copyright (c) 2011-2013 Mathias Lafeldt
# Copyright (c) 2005-2013 Git project
# Copyright (c) 2005-2013 Junio C Hamano
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/ .
test_description='Test Sharness itself'
. ./sharness.sh
test_expect_success 'success is reported like this' '
:
'
test_expect_failure 'pretend we have a known breakage' '
false
'
run_sub_test_lib_test () {
name="$1" descr="$2" # stdin is the body of the test code
mkdir "$name" &&
(
cd "$name" &&
cat >".$name.t" <<-EOF &&
#!$SHELL_PATH
test_description='$descr (run in sub sharness)
This is run in a sub sharness so that we do not get incorrect
passing metrics
'
# Point to the test/sharness.sh, which isn't in ../ as usual
. "\$SHARNESS_TEST_DIRECTORY"/sharness.sh
EOF
cat >>".$name.t" &&
chmod +x ".$name.t" &&
export SHARNESS_TEST_DIRECTORY &&
./".$name.t" --chain-lint >out 2>err
)
}
check_sub_test_lib_test () {
name="$1" # stdin is the expected output from the test
(
cd "$name" &&
! test -s err &&
sed -e 's/^> //' -e 's/Z$//' >expect &&
test_cmp expect out
)
}
test_expect_success 'pretend we have a fully passing test suite' "
run_sub_test_lib_test full-pass '3 passing tests' <<-\\EOF &&
for i in 1 2 3
do
test_expect_success \"passing test #\$i\" 'true'
done
test_done
EOF
check_sub_test_lib_test full-pass <<-\\EOF
> ok 1 - passing test #1
> ok 2 - passing test #2
> ok 3 - passing test #3
> # passed all 3 test(s)
> 1..3
EOF
"
test_expect_success 'pretend we have a partially passing test suite' "
test_must_fail run_sub_test_lib_test \
partial-pass '2/3 tests passing' <<-\\EOF &&
test_expect_success 'passing test #1' 'true'
test_expect_success 'failing test #2' 'false'
test_expect_success 'passing test #3' 'true'
test_done
EOF
check_sub_test_lib_test partial-pass <<-\\EOF
> ok 1 - passing test #1
> not ok 2 - failing test #2
# false
> ok 3 - passing test #3
> # failed 1 among 3 test(s)
> 1..3
EOF
"
test_expect_success 'pretend we have a known breakage' "
run_sub_test_lib_test failing-todo 'A failing TODO test' <<-\\EOF &&
test_expect_success 'passing test' 'true'
test_expect_failure 'pretend we have a known breakage' 'false'
test_done
EOF
check_sub_test_lib_test failing-todo <<-\\EOF
> ok 1 - passing test
> not ok 2 - pretend we have a known breakage # TODO known breakage
> # still have 1 known breakage(s)
> # passed all remaining 1 test(s)
> 1..2
EOF
"
test_expect_success 'pretend we have fixed a known breakage' "
run_sub_test_lib_test passing-todo 'A passing TODO test' <<-\\EOF &&
test_expect_failure 'pretend we have fixed a known breakage' 'true'
test_done
EOF
check_sub_test_lib_test passing-todo <<-\\EOF
> ok 1 - pretend we have fixed a known breakage # TODO known breakage vanished
> # 1 known breakage(s) vanished; please update test(s)
> 1..1
EOF
"
test_expect_success 'pretend we have fixed one of two known breakages (run in sub sharness)' "
run_sub_test_lib_test partially-passing-todos \
'2 TODO tests, one passing' <<-\\EOF &&
test_expect_failure 'pretend we have a known breakage' 'false'
test_expect_success 'pretend we have a passing test' 'true'
test_expect_failure 'pretend we have fixed another known breakage' 'true'
test_done
EOF
check_sub_test_lib_test partially-passing-todos <<-\\EOF
> not ok 1 - pretend we have a known breakage # TODO known breakage
> ok 2 - pretend we have a passing test
> ok 3 - pretend we have fixed another known breakage # TODO known breakage vanished
> # 1 known breakage(s) vanished; please update test(s)
> # still have 1 known breakage(s)
> # passed all remaining 1 test(s)
> 1..3
EOF
"
test_expect_success 'pretend we have a pass, fail, and known breakage' "
test_must_fail run_sub_test_lib_test \
mixed-results1 'mixed results #1' <<-\\EOF &&
test_expect_success 'passing test' 'true'
test_expect_success 'failing test' 'false'
test_expect_failure 'pretend we have a known breakage' 'false'
test_done
EOF
check_sub_test_lib_test mixed-results1 <<-\\EOF
> ok 1 - passing test
> not ok 2 - failing test
> # false
> not ok 3 - pretend we have a known breakage # TODO known breakage
> # still have 1 known breakage(s)
> # failed 1 among remaining 2 test(s)
> 1..3
EOF
"
test_expect_success 'pretend we have a mix of all possible results' "
test_must_fail run_sub_test_lib_test \
mixed-results2 'mixed results #2' <<-\\EOF &&
test_expect_success 'passing test' 'true'
test_expect_success 'passing test' 'true'
test_expect_success 'passing test' 'true'
test_expect_success 'passing test' 'true'
test_expect_success 'failing test' 'false'
test_expect_success 'failing test' 'false'
test_expect_success 'failing test' 'false'
test_expect_failure 'pretend we have a known breakage' 'false'
test_expect_failure 'pretend we have a known breakage' 'false'
test_expect_failure 'pretend we have fixed a known breakage' 'true'
test_done
EOF
check_sub_test_lib_test mixed-results2 <<-\\EOF
> ok 1 - passing test
> ok 2 - passing test
> ok 3 - passing test
> ok 4 - passing test
> not ok 5 - failing test
> # false
> not ok 6 - failing test
> # false
> not ok 7 - failing test
> # false
> not ok 8 - pretend we have a known breakage # TODO known breakage
> not ok 9 - pretend we have a known breakage # TODO known breakage
> ok 10 - pretend we have fixed a known breakage # TODO known breakage vanished
> # 1 known breakage(s) vanished; please update test(s)
> # still have 2 known breakage(s)
> # failed 3 among remaining 7 test(s)
> 1..10
EOF
"
test_set_prereq HAVEIT
haveit=no
test_expect_success HAVEIT 'test runs if prerequisite is satisfied' '
test_have_prereq HAVEIT &&
haveit=yes
'
donthaveit=yes
test_expect_success DONTHAVEIT 'unmet prerequisite causes test to be skipped' '
donthaveit=no
'
if test $haveit$donthaveit != yesyes
then
say "bug in test framework: prerequisite tags do not work reliably"
exit 1
fi
test_set_prereq HAVETHIS
haveit=no
test_expect_success HAVETHIS,HAVEIT 'test runs if prerequisites are satisfied' '
test_have_prereq HAVEIT &&
test_have_prereq HAVETHIS &&
haveit=yes
'
donthaveit=yes
test_expect_success HAVEIT,DONTHAVEIT 'unmet prerequisites causes test to be skipped' '
donthaveit=no
'
donthaveiteither=yes
test_expect_success DONTHAVEIT,HAVEIT 'unmet prerequisites causes test to be skipped' '
donthaveiteither=no
'
if test $haveit$donthaveit$donthaveiteither != yesyesyes
then
say "bug in test framework: multiple prerequisite tags do not work reliably"
exit 1
fi
clean=no
test_expect_success 'tests clean up after themselves' '
test_when_finished clean=yes
'
if test $clean != yes
then
say "bug in test framework: basic cleanup command does not work reliably"
exit 1
fi
test_expect_success 'tests clean up even on failures' "
test_must_fail run_sub_test_lib_test \
failing-cleanup 'Failing tests with cleanup commands' <<-\\EOF &&
test_expect_success 'tests clean up even after a failure' '
touch clean-after-failure &&
test_when_finished rm clean-after-failure &&
(exit 1)
'
test_expect_success 'failure to clean up causes the test to fail' '
test_when_finished \"(exit 2)\"
'
test_done
EOF
check_sub_test_lib_test failing-cleanup <<-\\EOF
> not ok 1 - tests clean up even after a failure
> # Z
> # touch clean-after-failure &&
> # test_when_finished rm clean-after-failure &&
> # (exit 1)
> # Z
> not ok 2 - failure to clean up causes the test to fail
> # Z
> # test_when_finished \"(exit 2)\"
> # Z
> # failed 2 among 2 test(s)
> 1..2
EOF
"
test_expect_success 'cleanup functions tun at the end of the test' "
run_sub_test_lib_test cleanup-function 'Empty test with cleanup function' <<-\\EOF &&
cleanup 'echo cleanup-function-called >&5'
test_done
EOF
check_sub_test_lib_test cleanup-function <<-\\EOF
1..0
cleanup-function-called
EOF
"
test_expect_success 'We detect broken && chains' "
test_must_fail run_sub_test_lib_test \
broken-chain 'Broken && chain' <<-\\EOF
test_expect_success 'Cannot fail' '
true
true
'
test_done
EOF
"
test_done
# vi: set ft=sh :

32
test/simple.t Executable file
View File

@ -0,0 +1,32 @@
#!/bin/sh
test_description="Show basic features of Sharness"
. ./sharness.sh
test_expect_success "Success is reported like this" "
echo hello world | grep hello
"
test_expect_success "Commands are chained this way" "
test x = 'x' &&
test 2 -gt 1 &&
echo success
"
return_42() {
echo "Will return soon"
return 42
}
test_expect_success "You can test for a specific exit code" "
test_expect_code 42 return_42
"
test_expect_failure "We expect this to fail" "
test 1 = 2
"
test_done
# vi: set ft=sh :