Optimization: Cancel pending futures on new LSP requests.

[JoshuaBatty]

The language server should cancel any pending futures that have yet to yield a result if a new request comes in from the client. For example, on every did_change event, we should stop compilation and start again immediately.

As this event happens on every keystroke, this optimization in itself should lead to a drastic speed improvement.

[JoshuaBatty]

This definitely isn't a trivial thing to implement. Ideally, we would be able to take advantage of the cancelRequest part of the LSP spec to drop any current work. Unfortunately, parse_project is getting triggered by didOpen which is a notification and not an LSP request. It looks like rust-analyzer also bumped into this problem with notifications, see 666 and 584

There is also a really good discussion here where tower-lsp is considering ditching concurrent execution and also notes the same issues above.

[JoshuaBatty]

Here you can see just how backed up the server can get in it's current state.

Screen.Recording.2023-04-10.at.11.17.00.am.mov

[JoshuaBatty]

A good stress test for this is to rename the Option type from within sway-lib-std/src/option.sw. In doing so, the client sends out a didChange notification for each of the files that are impacted, so roughly 10 notifications one after the other.

If we implement this correctly, each subsequent didChange notification will cancel any current work done in parse_project until the final one comes in, allowing us to complete parsing of the project and be ready to receive new requests from the Client.

[JoshuaBatty]

This was mostly addressed in #5429

We can now bail out of compilation if a newer did_change notification message is received