While the performance impact of these changes are in the noise under
macOS 10.12.3 on a on a MacBook Pro with a 2.9GHz Intel Core i5 CPU and
8GB 1867MHz DDR3 RAM, eliminating these subshells makes the code more
consistent.
I did try removing `buffer` to eliminate yet more subshells, but the
flickering of the output did prove annoying, so I'm not removing it.
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.16s) from the current test suite.
Before this change:
46 tests, 0 failures
real 0m3.541s
user 0m2.125s
sys 0m0.937s
After this change:
real 0m3.372s
user 0m2.031s
sys 0m0.894s
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.04-0.05s) from the current test suite. Very minor, but it's a
straightforward change that may provide a minor-yet-noticeable effect on
Windows.
Before this change:
46 tests, 0 failures
real 0m3.588s
user 0m2.171s
sys 0m1.046s
After this change:
real 0m3.538s
user 0m2.119s
sys 0m0.941s
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.25s) from the current test suite.
Before the change:
46 tests, 0 failures
real 0m3.851s
user 0m2.273s
sys 0m1.166s
After the change:
real 0m3.595s
user 0m2.171s
sys 0m1.048s
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.1s) from the current test suite.
Before the change:
46 tests, 0 failures
real 0m3.983s
user 0m2.320s
sys 0m1.241s
After the change:
real 0m3.861s
user 0m2.276s
sys 0m1.174s
Under Bash 3.2.57(1)-release and 4.4.12(1)-release on a MacBook Pro with
a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off
O(0.4s) from the current test suite.
Before the change:
46 tests, 0 failures
real 0m4.392s
user 0m2.489s
sys 0m1.467s
After the change:
real 0m3.980s
user 0m2.312s
sys 0m1.233s
This is part of the effort to improve performance by reducing the number
of command substitutions/subshells.
Under Bash 3.2.57(1)-release on a MacBook Pro with a 2.9GHz Intel Core
i5 CPU and 8GB 1867MHz DDR3 RAM, this shaves off O(0.15s) from the test
suite at the previous commit, but I anticipate this effect being
magnified on Windows platforms.
This is part of the effort to improve performance by reducing the number
of command substitutions/subshells spawned by `bats_debug_trap`.
Under Bash 3.2.57(1)-release on a MacBook Pro with a 2.9GHz Intel Core
i5 CPU and 8GB 1867MHz DDR3 RAM, this makes `bin/bats test/` go from the
following for the previous commit:
44 tests, 0 failures
real 0m5.293s
user 0m2.853s
sys 0m2.087s
to:
real 0m4.319s
user 0m2.559s
sys 0m1.454s
This is part of the effort to improve performance by reducing the number
of command substitutions/subshells spawned by `bats_debug_trap`.
Under Bash 3.2.57(1)-release on a MacBook Pro with a 2.9GHz Intel Core
i5 CPU and 8GB 1867MHz DDR3 RAM, this makes `bin/bats test/` go from:
44 tests, 0 failures
real 0m7.565s
user 0m3.664s
sys 0m3.368s
to:
real 0m6.449s
user 0m3.290s
sys 0m2.665s
When running under Bash 3.2.57(1)-release on macOS, the following tests
would fail because `BATS_ERROR_STACK_TRACE` would be empty, and hence no
information about the actual error would get printed:
- one failing test
- failing test with significant status
- failing test file outside of BATS_CWD
This is because each of these cases use `FIXTURE_ROOT/failing.bats`, and
the `ERR` trap would not fire for its `eval "( exit ${STATUS:-1} )"`
line. Changing it to `exit ${STATUS:-1}` produced the same effect, and
changing it to `return ${STATUS:-1}` would cause the output to point to
the previous line, which executes `true`.
However, the correct status would be reported to the `EXIT` trap, so now
we call `bats_error_trap` at the very beginning of `bats_teardown_trap`.
All the existing tests now pass under Bash 3.2.57(1)-release, Bash
4.2.25(1)-release (the version from the default Ubuntu 12.04.5/Precise
image on Travis CI), and Bash 4.4.12(1)-release.
IFS was modified by run() becoming '\n' and so relying to its bash default
was failing tests.
Also some wrong tests corrected because was relying on this behavior to pass.
Fix#89
This also update the behaviour of the summary, now it only display the
number of failures, and skipped tests also, if the numbers are greater
than zero.
The outermost command—i.e. the line that failed inside the test case function itself—is more likely to be meaningful at a glance than the innermost command, which might be e.g. the implementation of a helper assertion.
