MLTraining output status is not resolved correctly

[mafrahm]

Currently (probably due to some law update), our training is seen as completed even if outputs are missing. Apparently, we count the existence of each individual requested directory/file (even if they are optional) and the task is defined until 5 (number of folds) outputs are present. Since, however, we require ~8 directories/files per branch, the task is seen as completed as soon as one of the 5 trainings is finished

hh2bbww/hbw/ml/base.py

Lines 287 to 305 in e90add5

	def output(self, task: law.Task) -> dict[str, law.FileSystemTarget]:
	# declare the main target
	target = task.target(f"mlmodel_f{task.branch}of{self.folds}", dir=True)
	# TODO: cleanup (produce plots, stats in separate task)
	outp = {
	"mlmodel": target,
	"plots": target.child("plots", type="d", optional=True),
	"checkpoint": target.child("checkpoint", type="d", optional=True),
	}
	# define all files that need to be present
	outp["required_files"] = [
	target.child(fname, type="f") for fname in
	("saved_model.pb", "keras_metadata.pb", "fingerprint.pb", "parameters.yaml", "input_features.pkl")
	]
	return outp

[mafrahm]

with the newest columnflow/law version (not yet pushed to this repository), the outputs are counted correctly except that the optional tag of the targets seems to be ignored.