Add OpenVINO AI Plugins to Preview Track

[frenchwr]

This merges in the OpenVINO AI plugins to the preview branch. Until recently the plugins only supported GIMP 2.99 and therefore we published the snaps in a dedicated 2.99-openvino branch and track. Now that the plugins support GIMP 3.0 I propose putting them into preview for broader reach.

Note about the 3.0-openvino branch: I created this from the current 2.99-openvino branch, then merged in the preview branch, and finally added a handful of commits for 3.0 support.

I've tested on a few different devices (each with a different generation of Intel Core Ultra CPU) and verified that the plugins all work as expected (including with GPU and NPU acceleration).

[jnsgruk]

Thank you for this!

My only question is whether we should maintain a separate branch for this - I don't know that we want to ship openvino bits to every device, even when they may not support it.

We could go forward with an openvino/* track or similar?

WDYT? Perhaps @merlijn-sebrechts has an opinion?

Keeping it all in the same branch is certainly simpler for us 🤷‍♂️ but I'm not sure what that would like like in the future if we were to integrate toolkits from other vendors?

[frenchwr]

Thank you for this!

My only question is whether we should maintain a separate branch for this - I don't know that we want to ship openvino bits to every device, even when they may not support it.

We could go forward with an openvino/* track or similar?

WDYT? Perhaps @merlijn-sebrechts has an opinion?

Keeping it all in the same branch is certainly simpler for us 🤷‍♂️ but I'm not sure what that would like like in the future if we were to integrate toolkits from other vendors?

My preference is to not use a dedicated branch/track, primarily for broader reach but also to minimize the maintenance cost (with separate tracks someone will need to keep everything in sync, build everything twice, etc).

I also intentionally designed the packaging of the plugins to minimize the maintenance burden to the GIMP snap itself, i.e. I used snapd's content interface to allow the plugins to be maintained and updated independently from GIMP.

The risk of the plugins somehow breaking GIMP I think is also low.

In terms of hardware support, it's true that not all devices will support acceleration available only through Intel NPUs and GPUs; however, I'm publishing the OpenVINO snaps built for both AMD64 and ARM64 (I'm currently enlisting a colleague to test on ARM64) so the coverage for CPU support is fairly large!

That's my two cents - admittedly I'm somewhat biased. :)

[frenchwr]

Heads up: my coworker uncovered a small issue on ARM64 that I'm fixing now. I'll update here when it's done.

[merlijn-sebrechts]

+1 for adding this to the regular snap

[jnsgruk]

Give me a shout when you've had a chance to test the new changes and I'll get this merged - thank you!

[frenchwr]

In terms of hardware support, it's true that not all devices will support acceleration available only through Intel NPUs and GPUs; however, I'm publishing the OpenVINO snaps built for both AMD64 and ARM64 (I'm currently enlisting a colleague to test on ARM64) so the coverage for CPU support is fairly large!

It seems I was a bit too optimistic here. While the OpenVINO runtime supports ARM64, the plugins themselves do not, sadly. In fact, they only support Intel silicon (CPU, GPU, and NPU). We've tested a bit and I'm fairly certain two of the three plugins will run fine on ARM64 or an AMD CPU, but I don't think that's something we want to commit to supporting without testing/validation upstream.

I still think there is a case to including the plugins in preview for the other reasons I originally mentioned. We can control which devices get the new plugins like so:

• ARM64 machines: I'm going to delete the ARM64 releases for openvino-ai-plugins-gimp, so ARM64 machines will never get the plugins.
• non-Intel AMD64 machines: We can ensure that the openvino-ai-plugins-gimp snap never auto-installs by keeping it in beta or not listing it as a default-provider for the content plug in GIMP's snapcraft.yaml. This means users need to intentionally install it. I could make it more clear in the README that the plugins only run on Intel silicon.

[jnsgruk]

You could also use an advanced grammar is a bunch of places to ensure we don't ship bits to arm64 machines...

[lucyllewy]

I really like this, and that you have abstracted the vino dependency libraries to a separate snap so it isn't restricted to just gimp's consumption - I can foresee a usecase to have an openvino extension in snapcraft (similar to the gnome extension)

aside: if canonical can get the licensing agreements with nVidia it would also be handy to have a CUDA runtime snap and snapcraft extension to utilise it in other snaps.

[frenchwr]

I really like this, and that you have abstracted the vino dependency libraries to a separate snap so it isn't restricted to just gimp's consumption - I can foresee a usecase to have an openvino extension in snapcraft (similar to the gnome extension)

Thanks! The extension is definitely something we've considered.

aside: if canonical can get the licensing agreements with nVidia it would also be handy to have a CUDA runtime snap and snapcraft extension to utilise it in other snaps.

Completely agree. I know there are a fair number of NVIDIA snaps floating around but I am not so familiar with them.

[frenchwr]

@jnsgruk Just a heads up that I've been updating a few of the content producer snaps to better handle some of the Intel hardware detection. I should have an update on this PR in the next few days.

[frenchwr]

Ok - this is now ready for re-review.

Here are some updates I made to try to clarify hardware requirements and ensure a good user experience:

• The arm64 build uses empty command chain scripts for openvino-launch and openvino-ai-plugins-gimp-launch, since the plugins are not available for this arch.
• The openvino-ai-plugins-gimp-launch command-chain script has been updated to only run if it detects an Intel CPU (by reading /proc/cpuinfo). This will ensure users with AMD or other non-Intel X86_64 machines do not have any models installed or see any confusing log messages from the script.
• The openvino-launch command-chain script now includes a number of checks for Intel hardware and only updates a user's environment and emits logs if it detects Intel hardware.

[jnsgruk]

Would it be better to just include this check in the script itself and keep the snapcraft.yaml simple? In the end - what matters most is what happens at runtime, so having the command-chain script itself check the arch feels appropriate?

[frenchwr]

For the openvino-launch command-chain script, my goal is to keep the door open for ARM64 support in the future since the snap itself does support ARM64, even though there is not an existing snap that consumes the ARM64 version. However, we will soon be providing similar plugin support for a few consumer snaps (Audacity and OBS) which may need ARM64 support.

For the openvino-ai-plugins-gimp-launch command-chain script, there is an Intel CPU check so we don't necessarily need the uname -m block in snapcraft.yaml but I chose to keep it there for consistency with the other command-chain section above. I could remove it if you prefer.

[jnsgruk]

Makes sense, thanks

[jnsgruk]

As above :)

[frenchwr]

Commented above