stdlib: Move file filters into runners

[V-FEXrt]

Moves the fnInput/fnOutput functions into the runners so they can directly control them. Allows things like the cache runners to cache only actual output files

[colinschmidt]

Is it valid not to use fnInputs or fnOutputs? Is that not part of the runner's contract with the Plan and Job?
Or are we not using them here because this function calls another run function that does call those input/output functions?

[V-FEXrt]

Is it valid not to use fnInputs or fnOutputs?

Nope, it must always be called at least once. In this case, the cache runners don't call it because the runner they are wrapping is expected to call it

[ag-eitilt]

Seems like a reasonable thing for it to own. Slight documentation expansion would be nice, but the code seems good.

[colinschmidt]

LGTM

[colinschmidt]

LGTM

[ag-eitilt]

How does the cleanable list interact with the FnOutputs, for existing code? I definitely prefer how much more explicit this setup is, but I want to be sure that existing code which obviously doesn't interact with CleanableOutputs doesn't break, seeing as the function only seems to be applied as def fileOutputs = fnOutputs cleanable in the runners:

wake/share/wake/lib/system/path.wake

Lines 278 to 293 in 05d35ba

	# Allow an untracked file to be removed via `wake --clean`
	export target markFileCleanable (filepath: String): Result Unit Error =
	def hashPlan cmd =
	Plan "" cmd Nil Nil "." "" logNever logError logDebug ReRun Nil hashUsage identity identity False
	def job =
	hashPlan ("<hash>", filepath, Nil)
	| setPlanLabel "hash: {filepath}"
	| setPlanFnOutputs (\_ filepath, Nil)
	| runJobWith localRunner
	| setJobInspectVisibilityHidden
	if job.isJobOk then
	Pass Unit
	else
	failWithError "Failed to hash {filepath}"

And, actually, do we want to introduce a new Plan.FnCleanable to provide end users with more control over that list as well?

[colinschmidt]

How does the cleanable list interact with the FnOutputs, for existing code? I definitely prefer how much more explicit this setup is, but I want to be sure that existing code which obviously doesn't interact with CleanableOutputs doesn't break, seeing as the function only seems to be applied as def fileOutputs = fnOutputs cleanable in the runners:

Existing code calls CleanableOutputs All_Outputs or something and is always populated with the unfiltered outputs in runAlways. User code that implements its own runner will now have to fill in this field in the RunnerOutput to update to this version of wake which should not be a high burden given they are maintaining their own Runner already.

And, actually, do we want to introduce a new Plan.FnCleanable to provide end users with more control over that list as well?

I think this is a good idea but needs to be carefully described as something even less likely the user wants to play with than Plan.FnOutputs.

[V-FEXrt]

How does the cleanable list interact with the FnOutputs, for existing code? I definitely prefer how much more explicit this setup is, but I want to be sure that existing code which obviously doesn't interact with CleanableOutputs doesn't break

Yep its a tiny bit subtle but the tl;dr is that previously Runners were required to return "All outputs" that they knew about then runAlways did two things. 1. Pass the unfiltered list of all outputs into the now renamed but identical "AllOutputs" 2. run fnOuputs over the all outputs list and pass that into "outputs". Now the runner directly controls both lists but trivially does the exact same thing.

One subtle and interesting thing is that "AllOutputs" was a very strong misnomer since that list is always empty for the localRunner while the "filtered" outputs of the local runner has the actual discovered outputs. So in the runtime we actually get something like

cleanable_outputs = "all outputs as reported by runner"
hashed_real_outputs = FnOutputs cleanable_outputs (which may increase the size of the list)
unhashed_real_cleanable = cleanable_outputs - hashed_real_outputs

So the only things actually processed separately are items that are in cleanable_outputs but not in "filtered" outputs

And, actually, do we want to introduce a new Plan.FnCleanable to provide end users with more control over that list as well?

Lets punt on that for a bit. I can see it being useful but this change is already large, not one is currently asking for it, and its rather hard to explain what should go into it (esp since the cache won't store files in that list).