Refactor DatasetIssues and Issue classes

[rwblair]

Towards #2043

This PR flattens the structure of the DatasetIssues and Issue types. It adds a new subCode and rule field for further differentiation of issues. The issues of DatasetIssues are an array instead of as a map on . The nested files maps were removed in favor of creating a separate issue for each infraction file, code and all. With that there was no need to do any look ups of existing issues during validation.

addNonSchemaIssue was removed, its functionality combined with the regular add. Both took arrays of files as an argument, but in practice only one file was ever passed to it.

The old outputs would have these file objects with path, filenam, and relative path. It wasn't clear to me that those were really necessary. I tried to get away with just using the path to the file as a string for simplicity. But I may be missing something and having a proper File object in there would be better, when we have one.

I renamed reason to codeMessage since the text was always being drawn from where the code is defined be it schema or nonschema.
I renamed evidence to issueMessage since these were always generated at issue creation time during validation.
I don't know that either of these is actually clearer.

filter and groupBy have new DatasetIssues objects as part of their returns,
while get returns just an array of issues. should get return a new DatasetIssues?

You can chain calls to filter, is that useful? Chaining calls to groupBy isn't so easy, should it be easy? I had a version of groupBy that accepted an array of Issue keys to nest that would nest the groups of issues in a object for all the unique values of the keys in the order they were given, it was a mess so I removed it.

I had played with array of Issue keys being passed into the DatasetIssues constructor and used to set up a bunch of independent maps of values to be indexed on stored in this.groups (Map<keyof Issue, Map<Issue[keyof Issue]>, Issue[]>) but then I couldn't imagine it ever being used so I gave up on it.

Should DatasetIssues extend array? Push issues to this instead of this.issues. At its heart it is an array with some helper functions.

Currently issues store codeMessages which leads to quite a bit of duplication, they are consistently the longest field in an issue object and make this issues object 10x larger than old one. ThecodeMessages map in DatasetIssues could be used to keep a single copy of each message so they could be dropped from the Issue object, but then the issue object needs some assemblly when accessed.

If only we could store a reference to fields in codeMessages in issues.

[codecov]

Codecov Report

Attention: Patch coverage is 70.35714% with 83 lines in your changes missing coverage. Please review.

Project coverage is 87.23%. Comparing base (941f36d) to head (ec4e1d9).

Files	Patch %	Lines
bids-validator/src/schema/applyRules.ts	50.48%	51 Missing ⚠️
bids-validator/src/validators/filenameValidate.ts	66.66%	14 Missing ⚠️
bids-validator/src/issues/datasetIssues.ts	93.81%	6 Missing ⚠️
bids-validator/src/schema/context.ts	0.00%	4 Missing ⚠️
...ds-validator/src/validators/internal/unusedFile.ts	0.00%	4 Missing ⚠️
bids-validator/src/tests/local/common.ts	33.33%	2 Missing ⚠️
bids-validator/src/schema/associations.ts	0.00%	1 Missing ⚠️
bids-validator/src/validators/hed.ts	91.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2067      +/-   ##
==========================================
+ Coverage   85.69%   87.23%   +1.53%     
==========================================
  Files          91      139      +48     
  Lines        3782     6806    +3024     
  Branches     1220     1604     +384     
==========================================
+ Hits         3241     5937    +2696     
- Misses        455      779     +324     
- Partials       86       90       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[effigies]

@rwblair Just a note that I've opened https://github.com/rwblair/bids-validator/pull/10 against this. Issues discovered while attempting to address #2007, but the conflict with this PR was too high to be worth addressing separately.

[effigies]

Stopped making codeMessage suggestions. Grabbing food, so submitting now.

[effigies]

Needs more than I can do in a suggestion, due to diff, but what about:

for (const col of extraCols) {
  context.dataset.issues.add({
    code: 'TSV_ADDITIONAL_COLUMNS_NOT_ALLOWED',
    subCode: col,
    location: context.path,
    rule: schemaPath,
  })
}

[effigies]

Again, I think a foreach using subcodes would be good here.