Skip to content

Commit

Permalink
Move large categories of tests to their own folders (#1913)
Browse files Browse the repository at this point in the history
The `tests` folder contains a lot of tests for various things. In
particular, there are a lot of tests for the interpreter and a lot of
tests related to VHLO, so IMO it makes sense to give these their own
folders. I find that this helps navigate the tests.

I considered creating folders for CHLO, printing related tests, and
verification related tests, but IMO there aren't enough of those to
really warrant it.
  • Loading branch information
mlevesquedion authored Jan 16, 2024
1 parent a918afc commit 2a79b7c
Show file tree
Hide file tree
Showing 116 changed files with 96 additions and 97 deletions.
2 changes: 1 addition & 1 deletion build_tools/github_actions/lint_version.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ set_version_var() {
set_version_var

## Check if compatibility tests exist for version `X_Y_0`
COMPAT_TEST_BASE="stablehlo/tests/stablehlo_legalize_to_vhlo"
COMPAT_TEST_BASE="stablehlo/tests/vhlo/stablehlo_legalize_to_vhlo"
COMPAT_TEST="$COMPAT_TEST_BASE.mlir"
TEST_VERSION="${VERSION[0]}_${VERSION[1]}_0"
VERSIONED_COMPAT_TEST="$COMPAT_TEST_BASE.$TEST_VERSION.mlir"
Expand Down
4 changes: 2 additions & 2 deletions docs/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,8 @@ for roundtrip examples of using the Python Serialization APIs.

## Tests

We have a compatibility suite in [stablehlo/tests](../stablehlo/tests)
that involves [a comprehensive compendium of StableHLO ops](../stablehlo/tests/stablehlo_legalize_to_vhlo.mlir)
We have a compatibility suite in [stablehlo/tests/vhlo](../stablehlo/tests/vhlo)
that involves [a comprehensive compendium of StableHLO ops](../stablehlo/tests/vhlo/stablehlo_legalize_to_vhlo.mlir)
serialized for all supported StableHLO versions. For every pull request, we are
testing both backward and forward compatibility - i.e. that the suite can be
deserialized targeting HEAD (backward compatibility), that the compendium
Expand Down
8 changes: 4 additions & 4 deletions docs/interpreter_status.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,11 +147,11 @@ for op semantics and example usage).
<!-- markdownlint-disable line-length -->
We use LLVM's [lit](https://llvm.org/docs/CommandGuide/lit.html) tool to run and
compare against generated file to diff against the output of the interpreter
(see [stablehlo/tests](https://github.com/openxla/stablehlo/tree/main/stablehlo/tests)
(see [stablehlo/tests/interpret](https://github.com/openxla/stablehlo/tree/main/stablehlo/tests/interpret)
for example tests).

Testing `AddOp` (sample from
[interpret_add.mlir](https://github.com/openxla/stablehlo/blob/main/stablehlo/tests/interpret_add.mlir)):
[interpret_add.mlir](https://github.com/openxla/stablehlo/blob/main/stablehlo/tests/interpret/add.mlir)):
<!-- markdownlint-enable line-length -->

```mlir
Expand All @@ -171,7 +171,7 @@ Testing ops in the Distribution category requires running it via the

<!-- markdownlint-disable line-length -->
Testing `AllReduceOp` (sample from
[interpret_all_reduce.mlir](https://github.com/openxla/stablehlo/blob/main/stablehlo/tests/interpret_all_reduce.mlir)):
[all_reduce.mlir](https://github.com/openxla/stablehlo/blob/main/stablehlo/tests/interpret/all_reduce.mlir)):
<!-- markdownlint-enable line-length -->

```mlir
Expand Down Expand Up @@ -209,7 +209,7 @@ Following the StableHLO build steps, the StableHLO binaries for tools in
GDB can be used to step through the code:

```sh
gdb --args ./build/bin/stablehlo-translate -allow-unregistered-dialect --interpret ./stablehlo/tests/<test>.mlir
gdb --args ./build/bin/stablehlo-translate -allow-unregistered-dialect --interpret ./stablehlo/tests/interpret/<test>.mlir
```

## Appendix
Expand Down
3 changes: 1 addition & 2 deletions docs/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,7 @@ A test utility `stablehlo-translate --interpret`
is responsible for parsing the program, interpreting each function including the
operations constituting the function. We have a dedicated test-suite, consisting
of several tests exercising various runtime behaviors, for each StableHLO Op.
The tests can be found [here](https://github.com/openxla/stablehlo/tree/main/stablehlo/tests/)
(e.g. interpret\_\*.mlir).
The tests can be found [here](https://github.com/openxla/stablehlo/tree/main/stablehlo/tests/interpret).

### Testing guidelines

Expand Down
4 changes: 2 additions & 2 deletions docs/reference_checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ on those fronts alongside the interpreter implementation.
1. In case there is a mismatch between the constraints in the
implementation VS and those in the specification, make sure there is
an open issue reflecting that discrepancy.
1. In [interpreter tests](https://github.com/openxla/stablehlo/tree/main/stablehlo/tests):
1. Add a file called `interpret_<op_mnemonic>.mlir`.
1. In [interpreter tests](https://github.com/openxla/stablehlo/tree/main/stablehlo/tests/interpret):
1. Add a file called `<op_mnemonic>.mlir`.
1. Write tests following the [testing guidelines](https://github.com/openxla/stablehlo/blob/main/docs/reference.md#testing-guidelines).
1. In the [testdata directory](https://github.com/openxla/stablehlo/tree/main/stablehlo/testdata):
1. Run any disabled tests that are covered by the newly added operation.
Expand Down
Loading

0 comments on commit 2a79b7c

Please sign in to comment.