Rewriting renderers in Vulkan.

[bryanedds]

Even at the outset of developing our current OpenGL renderers, it was planned to 'some day' rewrite it in Vulkan.

So far, at least on windows and linux nvidia machines, we've been happy enough with this approach. However, AMD's support for OpenGL on windows has turned out to be unusable in anything but 2D and toy 3D applications. (UPDATE: we got AMD working again by ripping out the use of bindless textures in the engine.)

We are not looking to rewrite the renderers now or likely even any time this year (2024). But at some point, if OpenGL continues to die the death that - at the outset of Vulkan - Khronos promised us it wouldn't, we'll need to start investigating this.

It may still be possible that if vendors realize that the thousands of non-gaming software shops cannot afford to spend man-years rewriting their renderers nor need the features that Vulkan offers over OpenGL, OpenGL may again becomes a target for proper vendor support. Though I think it is the most reasonable outcome, like many similar things in life, we can't count on it.

Hence, this investigation issue.

EDIT: we are tacking some bugs reported to AMD related to our particular opengl issues -

https://community.amd.com/t5/newcomers-start-here/opengl-4-1-this-api-capture-crash-amd-gpu-driver-on-window-linux/m-p/664656

https://gitlab.freedesktop.org/mesa/mesa/-/issues/10581

DiligentGraphics/DiligentSamples#140

At this point, I wouldn't even rule out a hardware bug.

EDIT: ORIAL: At the end of the day, it doesn't matter which API one targets, nor how theoretically perfect that API is, if vendors don't do their basic jobs at supporting it's operation. There's no specific reason to think that some time in the future, AMD could become just as negligent in their support for Vulkan operation as their support has become with OpenGL. Even for what works now, they could easily regress their Vulkan support with things like rewrites and complex internal patches, landing end users with bugs as severe as the ones they're currently serving up with OpenGL. The fault lies with the vendors, and no amount of rewrites from this API to that API will overcome the problems they cause.

[bryanedds]

Investigation task breakdown (the tasks involved in investigation) -

1. Find an existing vulkan renderer with similar rendering techniques as Nu's (deferred, physically-based) that hopefully is small and clean enough to study, pulling in help from the developers as well as Bryan as needed. Possible examples can be found and used as needed. Submit for Bryan's approval on these resources as implementation references.

2. Compare said reference renderer with Nu's existing renderer to understand the essential and inessential differences in implementation. Would not be surprised if Nu's existing overall structure need not be altered much, just need to add additional layers slotted in to do what the OpenGL driver used to do but Vulkan does not.

3. Understand how vulkan will integrate with the multi-platform .Net environment, fsharp, SDL and IMGUI.

First pass implementation break-down -

1. Prepare Nu for Vulkan. The concrete goal here is to set up an OpenGL/Vulkan selection option. This may involve non-trivial restructuring. In particular, it is questionable whether the existing multi-rendering-subsystem approach (2D, 3D, IMGUI) is practical for Vulkan, which imposes an insane amount of highly integrated state at its core.

2. Set up core Vulkan infrastructure. This basically means adapting the triangle tutorial at https://vulkan-tutorial.com. It involves creating a big fat abstract data type, setting up validation layers and a shader-compiling process. Note that, unfortunately, Vulkan shaders are NOT identical to OpenGL ones, so we must figure out how to handle this. An especially delicate issue for this stage is making sure the swap chain is perfectly integrated with Nu's Timing/Synchronization/Framerate.

3. Implement IMGUI rendering. This precedes 2D rendering for two reasons. First, it's a potential challenge that should be promptly resolved, and second, Gaia is our principle tool for developing and testing graphics, compared with e.g. Elmario.

4. Implement 2D rendering with bindless textures. This more simple task is ideal preparation for the 3D work, and pulling the bindless textures forward avoids some time waste and allows us to begin grappling the associated problems right off the bat.

5. Implement skybox. Allows us to tackle cube maps nice and early as well.

6. Implement minimal forward rendering (without light mapping), i.e. minimal 3D.

7. Implement minimal deferred static rendering (without light mapping). The goal is not to achieve any deferred specific features, but just to set up the main, multiple render pass, architecture.

8. Implement animation.

9. Implement shadows. TODO: add more detail.

10. Implement lighting mapping. TODO: add more detail.

11. Implement terrain. Should be a (relatively) trivial task of no foundational importance.

12. Implement FXAA. Foundational feature but not obviously complex.

13. Investigate HDR affordances that might be specific to Vulkan.

[bryanedds]

TODO: provide action items for dean, specifically related to breaking down tasks 1 and 2 and any others.

EDIT: turns out, no action items for me here. Dean, proceed as is and ask any clarifying questions that might arise.

[danilw]

I can share some experience about Vulkan 1.3:

1. Vulkan 1.3 is so much better than Vulkan <=1.2 - you can bind textures/shaders without rebuilding entire pipeline in single line of code- it is huge "less amount of code". Also bindelss-stuff - works.
2. I saw this blog post modern Vulkan with descriptor indexing, dynamic rendering and shader objects tutorial.
   This repository vknew - can be used as proper example of small Vulkan 1.3 code. That example use SDL and vma external libs - but this can be considered as "production standard" to use glfw or sdl with vma.
3. Only problem of Vulkan 1.3 - only PC, when Vulkan <=1.2 is smartphones.
   Full list of Vulkan Conformant Products from Khronos from Vulkan 1.0 to 1.3 - for example Nintendo Switch support Vulkan 1.3.
4. vulkan gpuinfo - Nvidia 1050 support Vulkan 1.3, even Nvidia 750 support it, that is about 10yo.
   AMD Vega also seems to support it, and something like RX 580 - so AMD also have atleast 8 years of gpus with Vulkan 1.3

From what I saw - it is maybe even 100x faster to write complex code/engine on Vulkan 1.3 than in previous - because instead of having full spaghetti-logic of resource initialization/cleaning and writing huge code to do literally anything - in Vulkan 1.3 that "huge hundred lines of code" replaced literally by 1 line of code, especially if you will use bindless.

[deanjl]

I can share some experience about Vulkan 1.3:

1. Vulkan 1.3 is so much better than Vulkan <=1.2 - you can bind textures/shaders without rebuilding entire pipeline in single line of code- it is huge "less amount of code". Also bindelss-stuff - works.
2. I saw this blog post modern Vulkan with descriptor indexing, dynamic rendering and shader objects tutorial.
   This repository vknew - can be used as proper example of small Vulkan 1.3 code. That example use SDL and vma external libs - but this can be considered as "production standard" to use glfw or sdl with vma.
3. Only problem of Vulkan 1.3 - only PC, when Vulkan <=1.2 is smartphones.
   Full list of Vulkan Conformant Products from Khronos from Vulkan 1.0 to 1.3 - for example Nintendo Switch support Vulkan 1.3.
4. vulkan gpuinfo - Nvidia 1050 support Vulkan 1.3, even Nvidia 750 support it, that is about 10yo.
   AMD Vega also seems to support it, and something like RX 580 - so AMD also have atleast 8 years of gpus with Vulkan 1.3

From what I saw - it is maybe even 100x faster to write complex code/engine on Vulkan 1.3 than in previous - because instead of having full spaghetti-logic of resource initialization/cleaning and writing huge code to do literally anything - in Vulkan 1.3 that "huge hundred lines of code" replaced literally by 1 line of code, especially if you will use bindless.

Thanks for that, it's very helpful!

I was surprised to discover recently that VK_KHR_dynamic_rendering is now included, even pushed, in Vulkan 1.0. VK_EXT_shader_object and VK_EXT_descriptor_indexing are too.

Pending investigation, I'm wondering if all 3 extensions are supported on smartphones and other devices that don't support Vulkan 1.3?

So you have convinced us on using these modern features if we can, but actually using Vulkan 1.3 seems to be a separate question: if Vulkan 1.3 adds device support restrictions on top of the modern features alone, then what advantage does Vulkan 1.3 offer present developers beyond avoiding a small amount of annoying boilerplate and suffixes?

If I'm missing something here please let me know!

[danilw]

If I'm missing something here please let me know!

I have no solution to "support <=1.2 and 1.3".

Vulkan 1.3 project code base will be completely different to Vulkan <=1.2

if Vulkan 1.3 adds device support restrictions on top of the modern features alone

Vulkan 1.3 include most of those "EXT" that save so much time in code/logic, those "EXT" is core in Vulkan 1.3 so you dont need to check for them.

Knowing how much "pain" it to make Vulkan <=1.2 complex code - my point of message was - just make Vulkan 1.3 project, and then maybe if you actually need do <=1.2 with check for extensions and different render pipelines per extension.

https://docs.vulkan.org/spec/latest/appendices/versions.html - VK_KHR_dynamic_rendering is in core 1.3.