Add Feature to Select Devices to Execute Kernels On

[matinraayai]

It would be great in a multi-device system, the device id that will run a KA kernel could be set through a function call.
cc: @vchuravy

[vchuravy]

We can do this in a backwards compatible manor.

What function do we need?

• device(::Backend)
• device!(::Backend)
• ?devices or ndevices

[matinraayai]

Those, as well as synchronization functions on a single device.
I'll look into CUDA standards to see if there's something else I'm missing.

[vchuravy]

Those, as well as synchronization functions on a single device.

Can you expand on that? We already have synchronize w.r.t to the current active device.

[matinraayai]

Having a synchronize function that takes a device would have been nice, but if there's one w.r.t active device, I think it does the job just as well.

[luraess]

It may also depend if you need stream/queue or device sync. Currently, it implements stream/queue sync for GPU backends.

[miniskar]

How to configure CuStream / hipStream to backend?

[luraess]

How to configure CuStream / hipStream to backend?

As for now, kernels in KA will execute on default Stream. You can change priority using

KernelAbstractions.jl/src/KernelAbstractions.jl

Line 532 in 766ab7e

priority!(::Backend, prio::Symbol)

. If you need to perform asynchronous kernel launch, you could spawn new Julia tasks to launch kernels as they would then get another Stream to execute on. As far as I know, there is no KA implementation (yet) of fine grained stream management functionality, to e.g. explicitly switch or select new (other) streams (as in https://amdgpu.juliagpu.org/stable/streams/ -- and CUDA's analog).