Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify greedy scanners #608

Open
AntonyBlakey opened this issue Oct 4, 2023 · 1 comment
Open

Unify greedy scanners #608

AntonyBlakey opened this issue Oct 4, 2023 · 1 comment

Comments

@AntonyBlakey
Copy link
Contributor

I suggest we treat terminators and separators as delimiters with an empty opening delimiter and hence unify the three error correction strategies

@Xanewok
Copy link
Contributor

Xanewok commented Oct 20, 2023

For context, in terms of greedy scan logic itself:

  1. both DelimitedBy and TerminatedBy use ParserResult::recover_until_with_nested_delims
  2. SeparatedBy uses a dedicated SeparatedHelper::run

which all use skip_until_with_nested_delims underneath for the actual skipping/greedy scan, so these are unified in that sense.

I'll see if I can simplify the SeparatedBy to not require that much extra code (in that case I think we'd benefit from #600) but the question remains for the TerminatedBy.

If we change it so that we stop the greedy scan on the terminator as well, we might end up with imbalanced delimiters, since, for example, there may be a partial match that eats an opening delimiter which we will attempt to recover at an inner semicolon and not skip over the closing delimiter as the result. We only keep a local stack of delimiters during a recovery itself, so we still might accept the terminator

I've opened a https://github.com/Xanewok/slang/tree/stop-skipping-on-terminator as PoC (see here how it affects the { unchecked { x = 1; } } example).

One idea that pops to my head is that we could modify the parser to keep the delimiter stack for the parse and not only during recovery - we would accept terminator only if the parsed delimiters are not unclosed. I'll have to think a bit more about this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

2 participants