A question about "context"

[1185307269]

What does the 'context' parameter in this command represent?
If I change --context "1" to --context "2"，the generated files differ in the number at the beginning of the sequence.

python3 sample.py --model ${model} --t 0.8 --p 0.9 --max-length 1024 --num-samples 2 --context "1"

[Sireesiru]

I have the same question.

[aeolianine]

I first thought it indicates whether the sequence should match sequences from RefSeq (1) or BFD (2). On a second thought, after looking at some of their examples, I think it signals "forward" vs "reverse". They fed both directions into the training, and with "1" and "2" you can preserve knowledge of that direction.

I find the training on reverse and forward at the same time a bit counter-intuitive. Only the "forward protein" is functional in nature. Now if we pretended everything runs in reverse and learned on that, it should be equivalent to "training in the forward direction". The model can learn the relationships regardless of the direction. But if you give both, then the model learns that both directions are possible (likely). That is not true. But maybe the "1" and "2" tokens are a way of dealing with that.

So yeah, why train in both directions?