AMD AMF: Last frames not showing up when screen updates are ending [Windows]

[RPJoshL]

Describe the Bug

When using the H.264 or the H.265 AMD codec the last frames of a screen refresh aren't transmitted and get "lost". The screen is only refreshed when the mouse is moved or something else is displayed on the screen.

That means for example:

• when typing you have a delay of one letter / until the text cursor is blinking again
• you can't imemdiately see when marking some text with a double click
• when scrolling a website the content makes a "jump" when moving the mouse / the screen is updated
• the mouse feels a litte bit laggy because of the delay
• in Games every ~45 seconds the screen hangs for a second. But i can't really explain why this does happen or if its connected to this issue

For demonstrational purporses I have created a simple demo video showing the correct and excpected result on the left side and the output of moonlight on the right side. Mabe its helpful to understand the problem.

Expected Behavior

All frames are transmitted and showing in real time.

Additional Context

When using software encoding or VAAPI in Linux this issue ist not present. Also on my old RX 570 this issue did not occure, but there were other problems ... 😅

You can improve the problem drasticly with activating the tracking overlay with a transparency of 99% and a sampling rate of 0,25 seconds. Because then all 0,25 seconds the screen ist updated, and you can't actually feel the listed problems earlier.

Sunshine Host Operating System and Version

Windows 11

Architecture

64 Bit

Sunshine Version

0.12

GPU Type

AMD

GPU Model

RX 6600 XT

GPU Driver/Mesa Version

22.3.1 (Newest)

Capture Method (Linux Only)

No response

[TheElixZammuto]

When using software encoding or VAAPI in Linux

Do you have the same problem on Windows with software encoder? (libx264)
Probably it's more of a Desktop API issue than an encoding one

[RPJoshL]

When using the H.264 software encoding on Windows everything works just fine as excpected and the describe issue is not present.

Additional today I made some tests on Windows with all the various codecs (with 60 fps and 75 fps because I thought that this might be the issue - but got the same results in both cases).

Encoder	Issue present
AMD HEVC	Yes
AMD H.264	Yes
Software HEVC	Yes
Software H.264	No

[skaface]

This issue is happening with me also in windows 10, using the Nvidia hardware encoding. I am not seeing the mentioned issue with games hanging every ~45 seconds though so I do not think that is related.

[jorismak]

I don't know if it's the same issue, but I was trying nvenc methods and all felt like having a huge latency or delay (while looking at real host computer, I see the screen responding there almost latency-free to my mouse movements, so it isn't network lag).

Switching to the software encoder on 'ultrafast / zerolatency' and it feels instant, but now my CPU is doing work.

I didn't have any issues when GeforceExperience was doing the streaming, and I tried 'that other fork' of sunshine first - by accident - and got it working once on nvenc, but also without any delay problems.

Windows 10 x64
GTX 1060
0.13 release downloaded from Github

[b]edit:[/b] The videostream delay seems 'fixed' now on nvenc. I had moonlight set to force h264, and apparently the nvenc h264 encoder causes delays. Setting both back to auth and restarting / reconnecting, gives me hevc and fixes a global delay (huge latency) in the videostream.

But, now I end up with the same description as this issue: It seems new frames are only sent when the mouse moves or every second as a fallback or something. It makes typing feel really laggy, unless you keep moving the mouse at the same time.
Sort of the same with scrolling. It seems scroll-events don't cause an instant refresh of the video or something, and they only happen at lower intervals, making the scrolling feel weird and delayed. But if you keep moving the mouse, the scrolling is fine. It looks the same - or at least very similar - to the issue opened here, at least the video posted.
But I have the feeling it is not that 'the last frames from the buffer not displayed' or something like that, it feels that mouse movements cause the video to be 'refreshed' and changes sent over, while keyboard events (and mousewheel events) don't trigger an instant refresh, so the screen gets updated at a lower 'fall back' interval or something.

[Dr4goonD3mon]

I have the same Issue using Nvidia Hardware encoding, switch to software and there's no issue there the problem is the same: if there's no movement in the image like a video or animations the picture loses fps to single digits, this bothers me with menus in different from ends (playnite and BigBox) but also present on other non-game apps

[LizardByte-bot]

This issue is stale because it has been open for 90 days with no activity. Comment or remove the stale label, otherwise this will be closed in 10 days.

[MichaelMKenny]

This is still an issue with nvidia hardware encoding.

[psyke83]

This is likely the same issue as #412

[ReenigneArcher]

@psyke83 I believe so. I asked them to open that issue as I didn't realize there was already one open about it, but this topic comes up a lot.

[LizardByte-bot]

This issue has been fixed and will be available in the next release.

[gnattu]

May I kindly ask how this is fixed for non-AMD users? I only see the patches for amfenc but lots of nvidia users reported similar behavior using nvenc as well.

[MichaelMKenny]

May I kindly ask how this is fixed for non-AMD users? I only see the patches for amfenc but lots of nvidia users reported similar behavior using nvenc as well.

It's fixed in this pull request: #507

[gnattu]

May I kindly ask how this is fixed for non-AMD users? I only see the patches for amfenc but lots of nvidia users reported similar behavior using nvenc as well.

It's fixed in this pull request: #507

Thank you for explanation!

[jorubyp]

Sunshine Version: automated nightly release - 2023-01-01T14:36:47Z - 97b1790
Host OS: Windows 10 22H2 (19045.2364)
Host GPU: NVIDIA GeForce GTX 970, 1 Screen & 1 Dummy Plug (Only the dummy plug is present when using the Steam Link described below)
Host Encoder: NVENC

I believe symptoms of this problem are still present as of the nightly release specified above.

All three of my clients (Windows 10, iOS and Steam Link) will not receive video upon attempting to connect to a host streaming a completely static image. A screen update must occur or they will time out before displaying anything (iOS will receive sound before disconnecting, however)
With no access to the hosts mouse this makes connecting impossible.

Additionally where my other clients have had the other symptoms of this issue seemingly fixed, the Steam Link still waits for mouse movement in many cases, or another 'significant enough' visual difference, to display a new frame.

The problem is still apparent doing most things, even watching videos, but a reliable way for me to reproduce this is as follows:

1. Be using my iOS client as an input device, and my Steam Link as a display.
2. Scroll a webpage using a quick flick of my two fingers on the iOS client, so that my fingers leave the screen before the scroll finishes.

The iOS client will have no problem, and will now be displaying the correct image. The image displayed by the Steam Link however will only update for part of the scroll, freezing before the scroll finishes, and will not update until the next mouse movement.

The last version to have none of these issues is v0.9.0