mirror of
https://github.com/sstephenson/bats.git
synced 2024-11-17 11:42:33 +01:00
Merge remote-tracking branch 'ahippo/print-failed-command' into print-failed-command
Conflicts: libexec/bats-exec-test
This commit is contained in:
commit
c010220e7a
28
README.md
28
README.md
|
@ -1,6 +1,6 @@
|
||||||
# Bats: Bash Automated Testing System
|
# Bats: Bash Automated Testing System
|
||||||
|
|
||||||
Bats is a [TAP](http://testanything.org/)-compliant testing framework
|
Bats is a [TAP](http://testanything.org)-compliant testing framework
|
||||||
for Bash. It provides a simple way to verify that the UNIX programs
|
for Bash. It provides a simple way to verify that the UNIX programs
|
||||||
you write behave as expected.
|
you write behave as expected.
|
||||||
|
|
||||||
|
@ -30,6 +30,7 @@ Bash's `errexit` (`set -e`) option when running test cases. If every
|
||||||
command in the test case exits with a `0` status code (success), the
|
command in the test case exits with a `0` status code (success), the
|
||||||
test passes. In this way, each line is an assertion of truth.
|
test passes. In this way, each line is an assertion of truth.
|
||||||
|
|
||||||
|
|
||||||
## Running tests
|
## Running tests
|
||||||
|
|
||||||
To run your tests, invoke the `bats` interpreter with a path to a test
|
To run your tests, invoke the `bats` interpreter with a path to a test
|
||||||
|
@ -48,9 +49,10 @@ an "X" if it fails.
|
||||||
2 tests, 0 failures
|
2 tests, 0 failures
|
||||||
|
|
||||||
If Bats is not connected to a terminal—in other words, if you
|
If Bats is not connected to a terminal—in other words, if you
|
||||||
run it from a continuous integration system or redirect its output to
|
run it from a continuous integration system, or redirect its output to
|
||||||
a file—the results are displayed in human-readable, machine-parsable
|
a file—the results are displayed in human-readable, machine-parsable
|
||||||
[TAP format](http://testanything.org).
|
[TAP format](http://testanything.org).
|
||||||
|
|
||||||
You can force TAP output from a terminal by invoking Bats with the
|
You can force TAP output from a terminal by invoking Bats with the
|
||||||
`--tap` option.
|
`--tap` option.
|
||||||
|
|
||||||
|
@ -66,18 +68,19 @@ arguments, or with a path to a directory containing multiple `.bats`
|
||||||
files. Bats will run each test file individually and aggregate the
|
files. Bats will run each test file individually and aggregate the
|
||||||
results. If any test case fails, `bats` exits with a `1` status code.
|
results. If any test case fails, `bats` exits with a `1` status code.
|
||||||
|
|
||||||
|
|
||||||
## Writing tests
|
## Writing tests
|
||||||
|
|
||||||
Each Bats test file is evaluated n+1 times, where _n_ is the number of
|
Each Bats test file is evaluated _n+1_ times, where _n_ is the number of
|
||||||
test cases in the file. The first run counts the number of test cases,
|
test cases in the file. The first run counts the number of test cases,
|
||||||
then iterates over the test cases and executes each one in its own
|
then iterates over the test cases and executes each one in its own
|
||||||
process.
|
process.
|
||||||
|
|
||||||
For details about exactly how Bats evaluates test files, see [Bats
|
For more details about how Bats evaluates test files, see
|
||||||
Evaluation Process](https://github.com/sstephenson/bats/wiki/Bats-Evaluation-Process)
|
[Bats Evaluation Process](https://github.com/sstephenson/bats/wiki/Bats-Evaluation-Process)
|
||||||
on the wiki.
|
on the wiki.
|
||||||
|
|
||||||
### The _run_ helper
|
### `run`: Test other commands
|
||||||
|
|
||||||
Many Bats tests need to run a command and then make assertions about
|
Many Bats tests need to run a command and then make assertions about
|
||||||
its exit status and output. Bats includes a `run` helper that invokes
|
its exit status and output. Bats includes a `run` helper that invokes
|
||||||
|
@ -114,7 +117,7 @@ the first line:
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
### The _load_ command
|
### `load`: Share common code
|
||||||
|
|
||||||
You may want to share common code across multiple test files. Bats
|
You may want to share common code across multiple test files. Bats
|
||||||
includes a convenient `load` command for sourcing a Bash source file
|
includes a convenient `load` command for sourcing a Bash source file
|
||||||
|
@ -129,7 +132,7 @@ will source the script `test/test_helper.bash` in your test file. This
|
||||||
can be useful for sharing functions to set up your environment or load
|
can be useful for sharing functions to set up your environment or load
|
||||||
fixtures.
|
fixtures.
|
||||||
|
|
||||||
### The _skip_ command
|
### `skip`: Easily skip tests
|
||||||
|
|
||||||
Tests can be skipped by using the `skip` command at the point in a
|
Tests can be skipped by using the `skip` command at the point in a
|
||||||
test you wish to skip.
|
test you wish to skip.
|
||||||
|
@ -165,9 +168,9 @@ Or you can skip conditionally:
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
### Setup and teardown functions
|
### `setup` and `teardown`: Pre- and post-test hooks
|
||||||
|
|
||||||
You can define special `setup` and `teardown` functions which run
|
You can define special `setup` and `teardown` functions, which run
|
||||||
before and after each test case, respectively. Use these to load
|
before and after each test case, respectively. Use these to load
|
||||||
fixtures, set up your environment, and clean up when you're done.
|
fixtures, set up your environment, and clean up when you're done.
|
||||||
|
|
||||||
|
@ -199,6 +202,7 @@ in the test file.
|
||||||
* `$BATS_TMPDIR` is the location to a directory that may be used to
|
* `$BATS_TMPDIR` is the location to a directory that may be used to
|
||||||
store temporary files.
|
store temporary files.
|
||||||
|
|
||||||
|
|
||||||
## Installing Bats from source
|
## Installing Bats from source
|
||||||
|
|
||||||
Check out a copy of the Bats repository. Then, either add the Bats
|
Check out a copy of the Bats repository. Then, either add the Bats
|
||||||
|
@ -213,6 +217,7 @@ Bats. For example, to install Bats into `/usr/local`,
|
||||||
Note that you may need to run `install.sh` with `sudo` if you do not
|
Note that you may need to run `install.sh` with `sudo` if you do not
|
||||||
have permission to write to the installation prefix.
|
have permission to write to the installation prefix.
|
||||||
|
|
||||||
|
|
||||||
## Support
|
## Support
|
||||||
|
|
||||||
The Bats source code repository is [hosted on
|
The Bats source code repository is [hosted on
|
||||||
|
@ -227,7 +232,8 @@ To learn how to set up your editor for Bats syntax highlighting, see
|
||||||
[Syntax Highlighting](https://github.com/sstephenson/bats/wiki/Syntax-Highlighting)
|
[Syntax Highlighting](https://github.com/sstephenson/bats/wiki/Syntax-Highlighting)
|
||||||
on the wiki.
|
on the wiki.
|
||||||
|
|
||||||
### Version history
|
|
||||||
|
## Version history
|
||||||
|
|
||||||
*0.3.1* (October 28, 2013)
|
*0.3.1* (October 28, 2013)
|
||||||
|
|
||||||
|
|
|
@ -118,13 +118,23 @@ bats_capture_stack_trace() {
|
||||||
|
|
||||||
bats_print_stack_trace() {
|
bats_print_stack_trace() {
|
||||||
local frame
|
local frame
|
||||||
|
local filename
|
||||||
local lineno
|
local lineno
|
||||||
local index=1
|
local index=1
|
||||||
local count="${#@}"
|
local count="${#@}"
|
||||||
|
local failed_line
|
||||||
|
|
||||||
for frame in "$@"; do
|
for frame in "$@"; do
|
||||||
|
if [ $index -eq $count ]; then
|
||||||
|
filename="$BATS_TEST_FILENAME"
|
||||||
|
else
|
||||||
|
filename="$(bats_frame_filename "$frame")"
|
||||||
|
fi
|
||||||
|
|
||||||
lineno="$(bats_frame_lineno "$frame")"
|
lineno="$(bats_frame_lineno "$frame")"
|
||||||
|
|
||||||
if [ $index -eq 1 ]; then
|
if [ $index -eq 1 ]; then
|
||||||
|
failed_line="$(bats_extract_line "$filename" "$lineno")"
|
||||||
echo -n "# ("
|
echo -n "# ("
|
||||||
else
|
else
|
||||||
echo -n "# "
|
echo -n "# "
|
||||||
|
@ -136,9 +146,10 @@ bats_print_stack_trace() {
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if [ $index -eq $count ]; then
|
if [ $index -eq $count ]; then
|
||||||
echo "in test file $BATS_TEST_FILENAME, line $lineno)"
|
local failed_command="$(bats_strip_string "$failed_line")"
|
||||||
|
echo "in test file $filename, line $lineno)"
|
||||||
|
echo "# \`${failed_command}' failed"
|
||||||
else
|
else
|
||||||
local filename="$(bats_frame_filename "$frame")"
|
|
||||||
echo "in file $filename, line $lineno,"
|
echo "in file $filename, line $lineno,"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
@ -171,6 +182,17 @@ bats_frame_filename() {
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
bats_extract_line() {
|
||||||
|
local filename="$1"
|
||||||
|
local lineno="$2"
|
||||||
|
sed -n "${lineno}p" "$filename"
|
||||||
|
}
|
||||||
|
|
||||||
|
bats_strip_string() {
|
||||||
|
local string="$1"
|
||||||
|
printf "%s" "$string" | sed -e "s/^[ "$'\t'"]*//" -e "s/[ "$'\t'"]*$//"
|
||||||
|
}
|
||||||
|
|
||||||
bats_debug_trap() {
|
bats_debug_trap() {
|
||||||
if [ "$BASH_SOURCE" != "$1" ]; then
|
if [ "$BASH_SOURCE" != "$1" ]; then
|
||||||
bats_capture_stack_trace
|
bats_capture_stack_trace
|
||||||
|
|
9
package.json
Normal file
9
package.json
Normal file
|
@ -0,0 +1,9 @@
|
||||||
|
{
|
||||||
|
"name": "bats",
|
||||||
|
"version": "0.3.1",
|
||||||
|
"description": "Bash Automated Testing System",
|
||||||
|
"global": "true",
|
||||||
|
"install": "./install.sh /usr/local",
|
||||||
|
"scripts": [ "libexec/bats", "libexec/bats-exec-suite", "libexec/bats-exec-test", "libexec/bats-format-tap-stream", "libexec/bats-preprocess", "bin/bats" ]
|
||||||
|
}
|
||||||
|
|
|
@ -43,26 +43,29 @@ fixtures bats
|
||||||
@test "one failing test" {
|
@test "one failing test" {
|
||||||
run bats "$FIXTURE_ROOT/failing.bats"
|
run bats "$FIXTURE_ROOT/failing.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ ${lines[0]} = "1..1" ]
|
[ "${lines[0]}" = '1..1' ]
|
||||||
[ ${lines[1]} = "not ok 1 a failing test" ]
|
[ "${lines[1]}" = 'not ok 1 a failing test' ]
|
||||||
[ ${lines[2]} = "# (in test file $FIXTURE_ROOT/failing.bats, line 4)" ]
|
[ "${lines[2]}" = "# (in test file $FIXTURE_ROOT/failing.bats, line 4)" ]
|
||||||
|
[ "${lines[3]}" = "# \`false' failed" ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "one failing and one passing test" {
|
@test "one failing and one passing test" {
|
||||||
run bats "$FIXTURE_ROOT/failing_and_passing.bats"
|
run bats "$FIXTURE_ROOT/failing_and_passing.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ ${lines[0]} = "1..2" ]
|
[ "${lines[0]}" = '1..2' ]
|
||||||
[ ${lines[1]} = "not ok 1 a failing test" ]
|
[ "${lines[1]}" = 'not ok 1 a failing test' ]
|
||||||
[ ${lines[2]} = "# (in test file $FIXTURE_ROOT/failing_and_passing.bats, line 2)" ]
|
[ "${lines[2]}" = "# (in test file $FIXTURE_ROOT/failing_and_passing.bats, line 2)" ]
|
||||||
[ ${lines[3]} = "ok 2 a passing test" ]
|
[ "${lines[3]}" = "# \`false' failed" ]
|
||||||
|
[ "${lines[4]}" = 'ok 2 a passing test' ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "failing helper function logs the test case's line number" {
|
@test "failing helper function logs the test case's line number" {
|
||||||
run bats "$FIXTURE_ROOT/failing_helper.bats"
|
run bats "$FIXTURE_ROOT/failing_helper.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ "${lines[1]}" = "not ok 1 failing helper function" ]
|
[ "${lines[1]}" = 'not ok 1 failing helper function' ]
|
||||||
[ "${lines[2]}" = "# (from function \`failing_helper' in file $FIXTURE_ROOT/test_helper.bash, line 6," ]
|
[ "${lines[2]}" = "# (from function \`failing_helper' in file $FIXTURE_ROOT/test_helper.bash, line 6," ]
|
||||||
[ "${lines[3]}" = "# in test file $FIXTURE_ROOT/failing_helper.bats, line 5)" ]
|
[ "${lines[3]}" = "# in test file $FIXTURE_ROOT/failing_helper.bats, line 5)" ]
|
||||||
|
[ "${lines[4]}" = "# \`false' failed" ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "test environments are isolated" {
|
@test "test environments are isolated" {
|
||||||
|
@ -89,22 +92,25 @@ fixtures bats
|
||||||
@test "setup failure" {
|
@test "setup failure" {
|
||||||
run bats "$FIXTURE_ROOT/failing_setup.bats"
|
run bats "$FIXTURE_ROOT/failing_setup.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ "${lines[1]}" = "not ok 1 truth" ]
|
[ "${lines[1]}" = 'not ok 1 truth' ]
|
||||||
[ "${lines[2]}" = "# (from function \`setup' in test file $FIXTURE_ROOT/failing_setup.bats, line 2)" ]
|
[ "${lines[2]}" = "# (from function \`setup' in test file $FIXTURE_ROOT/failing_setup.bats, line 2)" ]
|
||||||
|
[ "${lines[3]}" = "# \`false' failed" ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "passing test with teardown failure" {
|
@test "passing test with teardown failure" {
|
||||||
PASS=1 run bats "$FIXTURE_ROOT/failing_teardown.bats"
|
PASS=1 run bats "$FIXTURE_ROOT/failing_teardown.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ "${lines[1]}" = "not ok 1 truth" ]
|
[ "${lines[1]}" = 'not ok 1 truth' ]
|
||||||
[ "${lines[2]}" = "# (from function \`teardown' in test file $FIXTURE_ROOT/failing_teardown.bats, line 2)" ]
|
[ "${lines[2]}" = "# (from function \`teardown' in test file $FIXTURE_ROOT/failing_teardown.bats, line 2)" ]
|
||||||
|
[ "${lines[3]}" = "# \`false' failed" ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "failing test with teardown failure" {
|
@test "failing test with teardown failure" {
|
||||||
PASS=0 run bats "$FIXTURE_ROOT/failing_teardown.bats"
|
PASS=0 run bats "$FIXTURE_ROOT/failing_teardown.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ "${lines[1]}" = "not ok 1 truth" ]
|
[ "${lines[1]}" = 'not ok 1 truth' ]
|
||||||
[ "${lines[2]}" = "# (in test file $FIXTURE_ROOT/failing_teardown.bats, line 6)" ]
|
[ "${lines[2]}" = "# (in test file $FIXTURE_ROOT/failing_teardown.bats, line 6)" ]
|
||||||
|
[ "${lines[3]}" = $'# `[ "$PASS" = "1" ]\' failed' ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "load sources scripts relative to the current test file" {
|
@test "load sources scripts relative to the current test file" {
|
||||||
|
@ -130,9 +136,9 @@ fixtures bats
|
||||||
@test "output is discarded for passing tests and printed for failing tests" {
|
@test "output is discarded for passing tests and printed for failing tests" {
|
||||||
run bats "$FIXTURE_ROOT/output.bats"
|
run bats "$FIXTURE_ROOT/output.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ "${lines[5]}" = "# failure stdout 1" ]
|
[ "${lines[6]}" = '# failure stdout 1' ]
|
||||||
[ "${lines[6]}" = "# failure stdout 2" ]
|
[ "${lines[7]}" = '# failure stdout 2' ]
|
||||||
[ "${lines[9]}" = "# failure stderr" ]
|
[ "${lines[11]}" = '# failure stderr' ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "-c prints the number of tests" {
|
@test "-c prints the number of tests" {
|
||||||
|
@ -172,10 +178,10 @@ fixtures bats
|
||||||
@test "extended syntax" {
|
@test "extended syntax" {
|
||||||
run bats-exec-test -x "$FIXTURE_ROOT/failing_and_passing.bats"
|
run bats-exec-test -x "$FIXTURE_ROOT/failing_and_passing.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ "${lines[1]}" = "begin 1 a failing test" ]
|
[ "${lines[1]}" = 'begin 1 a failing test' ]
|
||||||
[ "${lines[2]}" = "not ok 1 a failing test" ]
|
[ "${lines[2]}" = 'not ok 1 a failing test' ]
|
||||||
[ "${lines[4]}" = "begin 2 a passing test" ]
|
[ "${lines[5]}" = 'begin 2 a passing test' ]
|
||||||
[ "${lines[5]}" = "ok 2 a passing test" ]
|
[ "${lines[6]}" = 'ok 2 a passing test' ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "pretty and tap formats" {
|
@test "pretty and tap formats" {
|
||||||
|
@ -200,9 +206,10 @@ fixtures bats
|
||||||
@test "single-line tests" {
|
@test "single-line tests" {
|
||||||
run bats "$FIXTURE_ROOT/single_line.bats"
|
run bats "$FIXTURE_ROOT/single_line.bats"
|
||||||
[ $status -eq 1 ]
|
[ $status -eq 1 ]
|
||||||
[ "${lines[1]}" = "ok 1 empty" ]
|
[ "${lines[1]}" = 'ok 1 empty' ]
|
||||||
[ "${lines[2]}" = "ok 2 passing" ]
|
[ "${lines[2]}" = 'ok 2 passing' ]
|
||||||
[ "${lines[3]}" = "ok 3 input redirection" ]
|
[ "${lines[3]}" = 'ok 3 input redirection' ]
|
||||||
[ "${lines[4]}" = "not ok 4 failing" ]
|
[ "${lines[4]}" = 'not ok 4 failing' ]
|
||||||
[ "${lines[5]}" = "# (in test file $FIXTURE_ROOT/single_line.bats, line 9)" ]
|
[ "${lines[5]}" = "# (in test file $FIXTURE_ROOT/single_line.bats, line 9)" ]
|
||||||
|
[ "${lines[6]}" = $'# `@test "failing" { false; }\' failed' ]
|
||||||
}
|
}
|
||||||
|
|
Loading…
Reference in New Issue
Block a user