Skip to content

Commit

Permalink
tokens.md: say that lifetime-like tokens can't be immediately followe…
Browse files Browse the repository at this point in the history
…d by '

Forms like 'ab'c are rejected, so we need some way to explain why they
don't tokenise as two consecutive LIFETIME_OR_LABEL tokens.

Address this by adding "not immediately followed by `'`" to each of the
lexer rules for the lifetime-like tokens.

This also means there can be no ambiguity between CHAR_LITERAL and these
tokens (at present we don't say how such ambiguities are resolved).
  • Loading branch information
mattheww committed Mar 4, 2024
1 parent 1afcfd9 commit 7bd81a6
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion src/tokens.md
Original file line number Diff line number Diff line change
Expand Up @@ -630,11 +630,14 @@ Examples of reserved forms:

> **<sup>Lexer</sup>**\
> LIFETIME_TOKEN :\
> &nbsp;&nbsp; &nbsp;&nbsp; `'` [IDENTIFIER_OR_KEYWORD][identifier]\
> &nbsp;&nbsp; &nbsp;&nbsp; `'` [IDENTIFIER_OR_KEYWORD][identifier]
> _(not immediately followed by `'`)_\
> &nbsp;&nbsp; | `'_`
> _(not immediately followed by `'`)_
>
> LIFETIME_OR_LABEL :\
> &nbsp;&nbsp; &nbsp;&nbsp; `'` [NON_KEYWORD_IDENTIFIER][identifier]
> _(not immediately followed by `'`)_
Lifetime parameters and [loop labels] use LIFETIME_OR_LABEL tokens. Any
LIFETIME_TOKEN will be accepted by the lexer, and for example, can be used in
Expand Down

0 comments on commit 7bd81a6

Please sign in to comment.