Match cargo test package/workspace handling; add --package --workspace (

#161)

- Add `--package`, `--workspace`
- Match Cargo's heuristics for what to test in a package or a workspace
- Remove not-quite-right `Tool` abstraction

DESIGN.md

@@ -50,31 +50,57 @@ the content based on those addresses.
 replacements. The interface to the `syn` parser is localized here, and also the
 core of cargo-mutants logic to guess at valid replacements.
 
-## Relative dependencies
+## Major processing stages
 
-After copying the tree, cargo-mutants scans the top-level `Cargo.toml` and any
-`.cargo/config.toml` for relative dependencies. If there are any, the paths are
-rewritten to be absolute, so that they still work when cargo is run in the
-scratch directory.
+1. Find the workspace enclosing the start directory, and the packages within it.
+1. Determine which packages to mutate.
+1. Generate mutants by walking each package.
+1. Copy the source tree.
+1. Run baseline tests.
+1. Test each mutant in parallel.
+
+### Finding the workspace and packages
 
-## Enumerating the source tree
+cargo-mutants is invoked from within, or given with `-d`, a single directory, called the _start directory_. To find mutants and run tests we first need to find the enclosing workspace and the packages within it.
 
-A source tree may be a single crate or a Cargo workspace. There are several levels of nesting:
+This is done basically by parsing the output of `cargo locate-project` and `cargo metadata`.
 
-* A workspace contains one or more packages.
-* A package contains one or more targets.
-* Each target names one (or possibly-more) top level source files, whose directories are walked to find more source files.
-* Each source file contains some functions.
-* For each function we generate some mutants.
+We often want to test only one or a subset of packages in the workspace. This can be set explicitly with `--package` and `--workspace`, or heuristically depending on the project metadata and the start directory.
 
-The name of the containing package is passed through to the `SourceFile` and the `Mutant` objects.
+For each package, cargo tells us the build targets including tests and the main library or binary. The tests are not considered for mutation, so this leaves us with
+some targets of interest, and for each of them cargo tells us one top source file, typically something like `src/lib.rs` or `src/main.rs`.
 
-For source tree and baseline builds and tests, we pass Cargo `--workspace` to build and test everything. For mutant builds and tests, we pass `--package` to build and test only the package containing the mutant, on the assumption that each mutant should be caught by its own package's tests.
+### Discovering mutants
 
-Currently source files are discovered by finding any `bin` or `lib` targets in the package, then taking every `*.rs` file in the same directory as their top-level source file. (This is a bit approximate and will pick up files that might not actually be referenced by a `mod` statement, so may change in the future.)
+After discovering packages and before running any tests, we discover all the potential mutants.
+
+Starting from the top files for each package, we parse each source file using `syn`
+and then walk its AST. In the course of that walk we can find three broad categories of patterns:
+
+* A `mod` statement (without a block), which tells us of another source file we must remember
+  to walk.
+* A source pattern that cargo-mutants knows how to mutate, such as a function returning a value.
+* A pattern that tells cargo-mutants not to look further into this branch of the tree, such as `#[test]` or `#[mutants::skip]`.
+
+For baseline builds and tests, we test all the packages that will later be mutated. For mutant builds and tests, we pass `--package` to build and test only the package containing the mutant, on the assumption that each mutant should be caught by its own package's tests.
 
 We may later mutate at a granularity smaller than a single function, for example by cutting out an `if` statement or a loop, but that is not yet implemented. (<https://github.com/sourcefrog/cargo-mutants/issues/73>)
 
+### Copying the source tree
+
+Mutations are tested in copies of the source tree. (An option could be added to test in-place, which would be nice for CI.)
+
+Initially, one copy is made to run baseline tests; if they succeed then additional copies are made as necessary for each parallel job.
+
+After copying the tree, cargo-mutants scans the top-level `Cargo.toml` and any
+`.cargo/config.toml` for relative dependencies. If there are any, the paths are
+rewritten to be absolute, so that they still work when cargo is run in the
+scratch directory.
+
+Currently, the whole workspace tree is copied. In future, possibly only the package to be mutated could be copied: this would require changes to the code that fixes up dependencies.
+
+(This current approach assumes that all the packages are under the workspace directory, which is common but not actually required.)
+
 ## Handling timeouts
 
 Mutations can cause a program to go into an infinite (or just very long) loop:

NEWS.md

@@ -2,6 +2,8 @@
 
 ## Unreleased
 
+- Changed: `cargo mutants` now tries to match the behavior of `cargo test` when run within a workspace. If run in a package directory, it tests only that package. If run in a workspace that is not a package (a "virtual workspace"), it tests the configured default packages, or otherwise all packages. This can all be overridden with the `--package` or `--workspace` options.
+
 - New: generate key-value map values from types like `BTreeMap<String, Vec<u8>>`.
 
 - Changed: Send trace messages to stderr rather stdout, in part so that it won't pollute json output.

book/src/workspaces.md

@@ -2,17 +2,20 @@
 
 cargo-mutants supports testing Cargo workspaces that contain multiple packages. The entire workspace tree is copied.
 
-By default, all source files in all packages in the workspace are tested.
+By default, cargo-mutants has [the same behavior as Cargo](https://doc.rust-lang.org/cargo/reference/workspaces.html):
 
-**NOTE: This behavior is likely to change in future: see <https://github.com/sourcefrog/cargo-mutants/issues/156>.**
-
-You can use the `--file` options to restrict cargo-mutants to testing only files
-from some subdirectory, e.g. with `-f "utils/**/*.rs"`. (Remember to quote globs
-on the command line, so that the shell doesn't expand them.) You can use `--list` or
-`--list-files` to preview the effect of filters.
+* If `--workspace` is given, all packages in the workspace are tested.
+* If `--package` is given, the named packages are tested.
+* If the starting directory (or `-d` directory) is in a package, that package is tested.
+* Otherwise, the starting directory must be in a virtual workspace. If it specifies default members, they are tested. Otherwise, all packages are tested.
 
 For each mutant, only the containing package's tests are run, on the theory that
 each package's tests are responsible for testing the package's code.
 
 The baseline tests exercise all and only the packages for which mutants will
 be generated.
+
+You can also use the `--file` options to restrict cargo-mutants to testing only files
+from some subdirectory, e.g. with `-f "utils/**/*.rs"`. (Remember to quote globs
+on the command line, so that the shell doesn't expand them.) You can use `--list` or
+`--list-files` to preview the effect of filters.

src/build_dir.rs

@@ -152,10 +152,8 @@ mod test {
     #[test]
     fn build_dir_debug_form() {
         let options = Options::default();
-        let root = CargoTool::new()
-            .find_root("testdata/tree/factorial".into())
-            .unwrap();
-        let build_dir = BuildDir::new(&root, &options, &Console::new()).unwrap();
+        let workspace = Workspace::open("testdata/tree/factorial").unwrap();
+        let build_dir = BuildDir::new(&workspace.dir, &options, &Console::new()).unwrap();
         let debug_form = format!("{build_dir:?}");
         assert!(
             Regex::new(r#"^BuildDir \{ path: "[^"]*[/\\]cargo-mutants-factorial[^"]*" \}$"#)