-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
cli-test: a tool for testing command-line programs.
Not looking for other users right now, this is just enough to test unzip/zip/zipinfo. This includes tests for unzip and ziptool, along with a change to unzip's behavior to fix AOSP `make dist` when using ziptool unzip. Also add the boilerplate to run these tests on the device, in presubmit. Fix command name in --help output. Test: atest ziptool-tests Change-Id: I5c0215a3ab8cb2cd5fc517ed9c188f81a7bf4520 Former-commit-id: f276140d0fc0d0f0be8bce3356d69389eb8ab914
- Loading branch information
1 parent
5a4a698
commit f928131
Showing
11 changed files
with
688 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../.clang-format-2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
cc_binary { | ||
name: "cli-test", | ||
host_supported: true, | ||
srcs: ["cli-test.cpp"], | ||
cflags: ["-Wall", "-Werror"], | ||
shared_libs: ["libbase"], | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
# cli-test | ||
|
||
## What? | ||
|
||
`cli-test` makes integration testing of command-line tools easier. | ||
|
||
## Goals | ||
|
||
* Readable syntax. Common cases should be concise, and pretty much anyone | ||
should be able to read tests even if they've never seen this tool before. | ||
|
||
* Minimal issues with quoting. The toybox tests -- being shell scripts -- | ||
quickly become a nightmare of quoting. Using a non ad hoc format (such as | ||
JSON) would have introduced similar but different quoting issues. A custom | ||
format, while annoying, side-steps this. | ||
|
||
* Sensible defaults. We expect your exit status to be 0 unless you say | ||
otherwise. We expect nothing on stderr unless you say otherwise. And so on. | ||
|
||
* Convention over configuration. Related to sensible defaults, we don't let you | ||
configure things that aren't absolutely necessary. So you can't keep your test | ||
data anywhere except in the `files/` subdirectory of the directory containing | ||
your test, for example. | ||
|
||
## Non Goals | ||
|
||
* Portability. Just being able to run on Linux (host and device) is sufficient | ||
for our needs. macOS is probably easy enough if we ever need it, but Windows | ||
probably doesn't make sense. | ||
|
||
## Syntax | ||
|
||
Any all-whitespace line, or line starting with `#` is ignored. | ||
|
||
A test looks like this: | ||
``` | ||
name: unzip -l | ||
command: unzip -l $FILES/example.zip d1/d2/x.txt | ||
after: [ ! -f d1/d2/x.txt ] | ||
expected-stdout: | ||
Archive: $FILES/example.zip | ||
Length Date Time Name | ||
--------- ---------- ----- ---- | ||
1024 2017-06-04 08:45 d1/d2/x.txt | ||
--------- ------- | ||
1024 1 file | ||
--- | ||
``` | ||
|
||
The `name:` line names the test, and is only for human consumption. | ||
|
||
The `command:` line is the command to be run. Additional commands can be | ||
supplied as zero or more `before:` lines (run before `command:`) and zero or | ||
more `after:` lines (run after `command:`). These are useful for both | ||
setup/teardown but also for testing post conditions (as in the example above). | ||
|
||
Any `command:`, `before:`, or `after:` line is expected to exit with status 0. | ||
Anything else is considered a test failure. | ||
|
||
The `expected-stdout:` line is followed by zero or more tab-prefixed lines that | ||
are otherwise the exact output expected from the command. (There's magic behind | ||
the scenes to rewrite the test files directory to `$FILES` because otherwise any | ||
path in the output would depend on the temporary directory used to run the test.) | ||
|
||
There is currently no `expected-stderr:` line. Standard error is implicitly | ||
expected to be empty, and any output will cause a test failure. (The support is | ||
there, but not wired up because we haven't needed it yet.) | ||
|
||
The fields can appear in any order, but every test must contain at least a | ||
`name:` line and a `command:` line. | ||
|
||
## Output | ||
|
||
The output is intended to resemble gtest. | ||
|
||
## Future Directions | ||
|
||
* It's often useful to be able to *match* against stdout/stderr/a file rather | ||
than give exact expected output. We might want to add explicit support for | ||
this. In the meantime, it's possible to use an `after:` with `grep -q` if | ||
you redirect in your `command:`. | ||
|
||
* In addition to using a `before:` (which will fail a test), it can be useful | ||
to be able to specify tests that would cause us to *skip* a test. An example | ||
would be "am I running as root?". | ||
|
||
* It might be useful to be able to make exit status assertions other than 0? | ||
|
||
* There's currently no way (other than the `files/` directory) to share repeated | ||
setup between tests. |
Oops, something went wrong.