need to run contrast manifest check earlier in the process; improve error message

[slsevilla]

Description of the bug

Process errored because a sample was listed on the contrast manifest that wasn't included in the sample manifest. Need to implement a catch for this.

Command used and terminal output

No response

Relevant files

No response

System information

No response

[kelly-sovacool]

which process errored?

[slsevilla]

which process errored?

Welp. the process that it should have. CHIPSEQ:DIFF:CHECK_CONTRASTS

I guess two things from this then

1. Needs to happen earlier. This should error during the input_check and not mid-way through running.
2. an output file should be created that has this error inside. it'll make it easier to see the error because right now it's hidden within the log. It should also include what the error is, so the user doesn't have to guess for themselves.

Example, part of the output log

Command exit status:
  1

Command output:
  (empty)

Command error:
  
  Attaching package: 'dplyr'
  
  The following objects are masked from 'package:stats':
  
      filter, lag
  
  The following objects are masked from 'package:base':
  
      intersect, setdiff, setequal, union
  
  Rows: 3 Columns: 8
  -- Column specification --------------------------------------------------------
  Delimiter: ","
  chr (5): sample, sample_basename, fastq_1, antibody, control
  dbl (1): single_end
  lgl (2): rep, fastq_2
  
  i Use `spec()` to retrieve the full column specification for this data.
  i Specify the column types or set `show_col_types = FALSE` to quiet this message.
  Error: All sample names in contrasts must also be in the sample sheet
  In addition: Warning message:
  In readLines(file, warn = readLines.warn) :
    incomplete final line found on 'chip_contrast.yml'
  Execution halted

Work dir:
  /vf/users/CMBS_POB_JEE/analysis/chip/champagne_240221/work/25/03b7b9f28d859e50d61e189374ebe7

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details

[kelly-sovacool]

Thanks for the info! Yeah I think moving this inside the input check workflow should force it to run earlier.

Do you think we should be publishing log files separately from what nextflow captures?

[slsevilla]

Hmmm... I think that's a whole separate issue. Maybe? I haven't spent a lot of time looking at ways to better compile nextflow failures (just found a command to parse errors today innate to nextflow nextflow log last -f hash,workdir,stderr -F 'exit != 0') so I dont know if we need to do something else OR for modules that error, we have a way to display this easily.

i know a lot of the requests for help on pipelines I get are debugging... the easier it is for users to debug themselves the better. i can see a 'poweruser' being able to dig into the NF logs, but not an every-day user.

[kelly-sovacool]

resolved by #192