is_async logic may be broken after changes in Parallel branch

[balta2ar]

As discussed in #602 (comment), it may be the case that is_async logic has changed after Parallel branch was implemented. My plugin is rather a cumbersome piece of unstructured code, so if you won't be able to reproduce the issue yourself, I'll try to tidy it up and come with an example.

Quoting the comment:

But there is still a problem. My plugin relies on the is_async flag logic: I expect that deoplete will keep calling my gather_candidates method until I set context["is_async"] to False. When I process data in the background, I keep setting is_async to True. But it appears that deoplete does not call me again when my results are ready. Could you please check this behavior?

[Shougo]

Please upload the example what I can test.

And the reproduce instructions are needed.

[balta2ar]

Sample async Source:

from .base import Base


import re
RT_PATTERN = r'RT:?\w*$'
RX_RT = re.compile(RT_PATTERN, re.IGNORECASE)


def log(msg):
    with open('/tmp/async.log', 'a') as file_:
        file_.write('%s\n' % msg)


class Source(Base):
    def __init__(self, vim):
        Base.__init__(self, vim)

        self.debug_enabled = True
        self.name = 'request_tracker'
        self.mark = '[RT]'

        self.is_volatile = True
        self.matchers = []
        self.sorters = []

        self.max_menu_width = 120
        self.max_abbr_width = 120
        self.input_pattern = RT_PATTERN

        self._cached_input = None
        self._counter = 10

    def get_complete_position(self, context):
        match = RX_RT.search(context['input'])
        return match.start() if match else -1

    def gather_candidates(self, context):
        if context['input'] != self._cached_input:
            log('RESET INPUT')
            self._cached_input = context['input']
            self._counter = 10

        if self._counter == 0:
            log('RESULT READY')
            context['is_async'] = False
            return ['RTfirst', 'RTsecond', 'RTthird']

        self._counter -= 1
        log('RESULT IS NOT READY: counter %s' % self._counter)
        context['is_async'] = True
        return []

Reproduce: start nvim and type RTfir.

I get these messages in /tmp/async.log on commit 865747e:

RESET INPUT                                                                                                                                                                   
RESULT IS NOT READY: counter 9                                                                                                                                                
RESULT IS NOT READY: counter 8                                                                                                                                                
RESET INPUT                                                                                                                                                                   
RESULT IS NOT READY: counter 9                                                                                                                                                
RESULT IS NOT READY: counter 8                                                                                                                                                
RESULT IS NOT READY: counter 7                                                                                                                                                
RESULT IS NOT READY: counter 6                                                                                                                                                
RESULT IS NOT READY: counter 5                                                                                                                                                
RESULT IS NOT READY: counter 4                                                                                                                                                
RESULT IS NOT READY: counter 3                                                                                                                                                
RESULT IS NOT READY: counter 2                                                                                                                                                
RESULT IS NOT READY: counter 1                                                                                                                                                
RESULT IS NOT READY: counter 0                                                                                                                                                
RESULT READY

Here are the messages in the current master:

RESET INPUT                                                                                                                                                                   
RESULT IS NOT READY: counter 9                                                                                                                                                
RESULT IS NOT READY: counter 8                                                                                                                                                
RESET INPUT                                                                                                                                                                   
RESULT IS NOT READY: counter 9                                                                                                                                                
RESULT IS NOT READY: counter 8                                                                                                                                                
RESET INPUT                                                                                                                                                                   
RESULT IS NOT READY: counter 9                                                                                                                                                
RESULT IS NOT READY: counter 8

[Shougo]

Please test the patch.

#471 (comment)

I think it fixes the problem.

[balta2ar]

Unfortunately, it doesn't.

RESET INPUT
RESULT IS NOT READY: counter 9
RESULT IS NOT READY: counter 8
RESULT IS NOT READY: counter 7

Looks like a kind of a race condition, because sometimes counter goes down to 4, if I type more characters, e.g. RTfirst.

[Shougo]

Unfortunately, I cannot reproduce the problem with the patch.
You can investigate the problem.

[Shougo]

The completion will be skipped if you changed input.

[Shougo]

Looks like a kind of a race condition, because sometimes counter goes down to 4, if I type more characters, e.g. RTfirst.

If so, it depends on your environment. I cannot reproduce/fix the problem.
Fortunately, deoplete is Python3 based plugin.
You can fix the problem easily, can't you?
Because you can reproduce the problem easily.

Of course, if it is reproduced for me, it can be fixed easily.

[balta2ar]

Could you please try this repo to reproduce the issue locally? https://github.com/balta2ar/deoplete-async

docker build -t deoplete .
docker run -it --rm deoplete
nvim

After that in nvim type "RT" only. I can reproduce the issue in this environment. This isn't exactly a minimal setup, but at least you could reproduce it yourself, then we can think how to minimize it. I couldn't point out a single plugin that's breaking things, I think it could be a collection of plugins.

[balta2ar]

In the repo example I forgot to apply the patch you mentioned... As you say, it may be crucial. I'll try to do that tomorrow, or if you get to this sooner than me, please do it yourself.

[Shougo]

I have merged the patch in the master.
Please test it again.

[balta2ar]

I confirm that the issue is gone for now both on my actual machine and in the docker env. As it sometimes happen with race conditions, may it be because of "timeout clearance" is higher now (i.e. it's just chances for that to happen are lower but they are still there)?

For my edification, could you please explain why that patch helps and what it does, @Shougo? It's vimscript, I'd never even thought to go there if I were to track it down, I'd be checking python code only.

[Shougo]

For my edification, could you please explain why that patch helps and what it does, @Shougo? It's vimscript, I'd never even thought to go there if I were to track it down, I'd be checking python code only.

Because, the async completion is skipped if previous candidates are same.
I have fixed the problem.