Skip to content

Commit

Permalink
Show the unnormalized output on output mismatch failure
Browse files Browse the repository at this point in the history
  • Loading branch information
oli-obk committed Feb 25, 2025
1 parent a9a8ea0 commit 73518d2
Show file tree
Hide file tree
Showing 13 changed files with 50 additions and 25 deletions.
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Removed

## [0.29.0] - 2025-02-25

### Added

### Fixed

### Changed

* output conflict handling now only takes the unnormalized output and is expected to normalize itself if desired.

### Removed

## [0.28.0] - 2025-01-27

### Added
Expand Down
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "ui_test"
version = "0.28.0"
version = "0.29.0"
edition = "2021"
license = "MIT OR Apache-2.0"
description = "A test framework for testing rustc diagnostics output"
Expand Down
23 changes: 12 additions & 11 deletions src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use crate::{
use crate::{
diagnostics::{self, Diagnostics},
parser::CommandParserFunc,
per_test_config::{Comments, Condition},
per_test_config::{Comments, Condition, TestConfig},
CommandBuilder, Error, Errored, Errors,
};
use color_eyre::eyre::Result;
Expand Down Expand Up @@ -85,7 +85,7 @@ impl AbortCheck {
}

/// Function that performs the actual output conflict handling.
pub type OutputConflictHandling = fn(&Path, Vec<u8>, &mut Errors, &Config);
pub type OutputConflictHandling = fn(&Path, &[u8], &mut Errors, &TestConfig);

impl Config {
/// Create a blank configuration that doesn't do anything interesting
Expand Down Expand Up @@ -484,36 +484,37 @@ impl Config {
/// test.
pub fn error_on_output_conflict(
path: &Path,
actual: Vec<u8>,
output: &[u8],
errors: &mut Errors,
config: &Config,
config: &TestConfig,
) {
let expected = std::fs::read(path).unwrap_or_default();
if actual != expected {
if config.normalize(output, &path.extension().unwrap().to_string_lossy()) != expected {
errors.push(Error::OutputDiffers {
path: path.to_path_buf(),
actual,
actual: output.to_vec(),
expected,
bless_command: config.bless_command.clone(),
bless_command: config.config.bless_command.clone(),
});
}
}

/// Ignore mismatches in the stderr/stdout files.
pub fn ignore_output_conflict(
_path: &Path,
_actual: Vec<u8>,
_output: &[u8],
_errors: &mut Errors,
_config: &Config,
_config: &TestConfig,
) {
}

/// Instead of erroring if the stderr/stdout differs from the expected
/// automatically replace it with the found output (after applying filters).
pub fn bless_output_files(path: &Path, actual: Vec<u8>, _errors: &mut Errors, _config: &Config) {
if actual.is_empty() {
pub fn bless_output_files(path: &Path, output: &[u8], _errors: &mut Errors, config: &TestConfig) {
if output.is_empty() {
let _ = std::fs::remove_file(path);
} else {
let actual = config.normalize(output, &path.extension().unwrap().to_string_lossy());
std::fs::write(path, &actual).unwrap();
}
}
3 changes: 1 addition & 2 deletions src/per_test_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -186,9 +186,8 @@ impl TestConfig {
}

pub(crate) fn check_output(&self, output: &[u8], errors: &mut Errors, kind: &str) -> PathBuf {
let output = self.normalize(output, kind);
let path = self.output_path(kind);
(self.config.output_conflict_handling)(&path, output, errors, &self.config);
(self.config.output_conflict_handling)(&path, output, errors, &self);
path
}

Expand Down
2 changes: 1 addition & 1 deletion tests/integrations/basic-bin/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tests/integrations/basic-fail-mode/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tests/integrations/basic-fail/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 16 additions & 3 deletions tests/integrations/basic-fail/Cargo.stdout
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,13 @@ Execute `DO NOT BLESS. These are meant to fail` to update `tests/actual_tests/ba
+++ <stderr output>
error[E0308]: mismatched types
--> tests/actual_tests/bad_pattern.rs:4:9
... 10 lines skipped ...
... 5 lines skipped ...

Check failure on line 33 in tests/integrations/basic-fail/Cargo.stdout

View workflow job for this annotation

GitHub Actions / tests (windows-latest, i686-pc-windows-msvc)

actual output differs from expected

this line was expected to be `- --> tests/actual_tests/bad_pattern.rs:4:9`
|
note: function defined here
- --> $DIR/tests/integrations/basic-fail/src/lib.rs:LL:CC

Check failure on line 36 in tests/integrations/basic-fail/Cargo.stdout

View workflow job for this annotation

GitHub Actions / tests (windows-latest, i686-pc-windows-msvc)

missing line in output

bless the test to create a line containing ` 4 | add("42", 3);`
+ --> $DIR/tests/integrations/basic-fail/src/lib.rs:LL:CC

Check failure on line 37 in tests/integrations/basic-fail/Cargo.stdout

View workflow job for this annotation

GitHub Actions / tests (windows-latest, i686-pc-windows-msvc)

actual output differs from expected

this line was expected to be `- --> $DIR/tests/integrations/basic-fail/src/lib.rs:1:8`
|
1 | pub fn add(left: usize, right: usize) -> usize {
| ^^^

-error: aborting due to previous error
Expand Down Expand Up @@ -227,7 +233,13 @@ Execute `DO NOT BLESS. These are meant to fail` to update `tests/actual_tests/fo
+++ <stderr output>
error[E0308]: mismatched types
--> tests/actual_tests/foomp2.rs:4:9
... 10 lines skipped ...
... 5 lines skipped ...
|
note: function defined here
- --> $DIR/tests/integrations/basic-fail/src/lib.rs:LL:CC
+ --> $DIR/tests/integrations/basic-fail/src/lib.rs:LL:CC
|
1 | pub fn add(left: usize, right: usize) -> usize {
| ^^^

-error: aborting due to previous error
Expand Down Expand Up @@ -1429,7 +1441,8 @@ Execute `DO NOT BLESS. These are meant to fail` to update `tests/actual_tests/ru
- | --- ^^^^ expected `usize`, found `&str`
+ | ^^^^^^^^^^ use of undeclared crate or module `basic_fail`

thread 'rustc' panicked
-thread 'rustc' panicked
+thread 'rustc' panicked at compiler/rustc_errors/src/lib.rs:


error: `mismatched types` not found in diagnostics on line 8
Expand Down
2 changes: 1 addition & 1 deletion tests/integrations/basic/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tests/integrations/cargo-run/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tests/integrations/dep-fail/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tests/integrations/ui_test_dep_bug/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 73518d2

Please sign in to comment.