What is the difference between importing a langium grammar and just putting that grammar into your file?

[Austin-Fulbright]

So if I have a common file with this inside of it:

  accDescr?: string;
  accTitle?: string;
  title?: string;
}

fragment TitleAndAccessibilities:
  ((accDescr=ACC_DESCR | accTitle=ACC_TITLE | title=TITLE) EOL)+
;

fragment EOL returns string:
  NEWLINE+ | EOF
;

terminal NEWLINE: /\r?\n/;
terminal ACC_DESCR: /[\t ]*accDescr(?:[\t ]*:([^\n\r]*?(?=%%)|[^\n\r]*)|\s*{([^}]*)})/;
terminal ACC_TITLE: /[\t ]*accTitle[\t ]*:(?:[^\n\r]*?(?=%%)|[^\n\r]*)/;
terminal TITLE: /[\t ]*title(?:[\t ][^\n\r]*?(?=%%)|[\t ][^\n\r]*|)/;

hidden terminal WHITESPACE: /[\t ]+/;
hidden terminal YAML: /---[\t ]*\r?\n(?:[\S\s]*?\r?\n)?---(?:\r?\n|(?!\S))/;
hidden terminal DIRECTIVE: /[\t ]*%%{[\S\s]*?}%%(?:\r?\n|(?!\S))/;
hidden terminal SINGLE_LINE_COMMENT: /[\t ]*%%[^\n\r]*/;

and I have a langium file with this:

import "../common/common";

entry GitGraph:
  NEWLINE*
  'gitGraph' Direction? ':'?
  NEWLINE*
  (
    NEWLINE*
    (TitleAndAccessibilities |
    statements+=Statement |
     NEWLINE)*
  )
;

Statement
: Commit
| Branch
| Merge 
| Checkout 
| CherryPicking
;

Direction:
    dir=('LR' | 'TB' | 'BT');

Commit:
    'commit' 
    (
        'id:' id=STRING 
        |'msg:'? message=STRING
        |'tag:' tags+=STRING
        |'type:' type=('NORMAL' | 'REVERSE' | 'HIGHLIGHT')
    )* EOL;
Branch:
    'branch' name=(ID|STRING)
    ('order:' order=INT)?
    EOL;

Merge:
    'merge' branch=(ID|STRING)
    (
        'id:' id=STRING
        |'tag:' tags+=STRING
        |'type:' type=('NORMAL' | 'REVERSE' | 'HIGHLIGHT')
    )* EOL;

Checkout:
    ('checkout'|'switch') branch=(ID|STRING) EOL;

CherryPicking:
    'cherry-pick' 
    (
        'id:' id=STRING
        |'tag:' tags+=STRING
        |'parent:' parent=STRING
    )* EOL;



terminal INT returns number: /[0-9]+(?=\s)/;
terminal ID returns string: /\w([-\./\w]*[-\w])?/;
terminal STRING: /"[^"]*"|'[^']*'/;

what is the difference between just doing this and having the common file directly inside of the grammer gitGraph file. Because for whatever reason it seems to create different rules.

[msujew]

Hey @Austin-Fulbright,

the only real difference is that there no guarantee that the order of terminals is preserved. Generally, Terminals are lexed in the order they appear in the grammar. I.e. if a word could be tokenized as two different terminals, the first terminal that appears in the grammar receives precedence.

However, when importing a grammar, all imported terminal rules get put to the back of the list, and consequently get the lowest precedence.

What kind of different behavior are you seeing with the import compared to the inlined grammar?

[Austin-Fulbright]

So the grammer rules that involve collecting TitleAndAccesibilty tokens. So when I import the ../common/common it will not recognize any of the title and accesability tokens. For example:

gitGraph
accTitle: example acces. title
accDescr: example acces. description
commit
commit

inputed into the language with the imports would throw an error such as expecting EOF but found 'accTitle'

However if in the playground or my langium project I use all of the code in the same file, it will work just fine.

There is also some services that I am using from my common. This is probably a misunderstanding on my part how the modules work.

Here are some examples from my project:

gitGraph/gitGraph.langium

gitGraph/module.ts

common/matcher.ts

common/tokenBulder.ts

common/valueConverter.ts

Thanks for taking the time to answer my question :)

[msujew]

I've put your grammar into the playground and simulated what happens when using grammar imports here. I've mentioned this in my earlier comment:

However, when importing a grammar, all imported terminal rules get put to the back of the list, and consequently get the lowest precedence.

Since terminals that appear later in the grammar get a lower precedence, the lexer and consequently the parser behave differently. Note that you can adjust the TokenBuilder instance to modify that behavior.