Q&A Can sionna is accelerated by GPU when only run once ?

[cheerkwah]

Hi dear sionna experts,
I just feel puzzled that how to use GPU to accelerated sionna with Keras layer and model ? Is that due to the simulation is run batchedly and each run is performed as the CUDA kernal? So if only one TTI (batch size = 1) is run, it can still get accelerated by GPU? How to understands the Keras and Cuda's relationship?

Why I ask that is we have our own RX receiver in python, now we are thinking how to adapted into GPU platform and get the acceleration (it can only run once or several times). One way is to is Numba, or write in Cuda python, shall sionna can fulfill that purpose easier ?

[faycalaa]

Thank you for your interest in Sionna!

To benefit from GPU acceleration, you can implement your receiver as a Keras layer in Python using the TensorFlow/Keras framework. You can find more information about this here:
https://keras.io/guides/making_new_layers_and_models_via_subclassing

Once this is done, your new layer can be integrated into a Sionna-based communication system. We have provided a notebook that shows how to implement a neural network-based receiver as a Keras layer with Sionna, which could be helpful:
https://nvlabs.github.io/sionna/examples/Neural_Receiver.html

GPU acceleration happens independently of the batch size, as long as your receiver is implemented using Keras/TensorFlow. Moreover, TensorFlow provides different execution modes that enable different levels of acceleration. You can find more information in this notebook (section Eager vs Graph Mode):
https://nvlabs.github.io/sionna/examples/Sionna_tutorial_part1.html

Note that If you need to implement your receiver in C++/CUDA, we plan to provide a tutorial soon.

[cheerkwah]

Hi Faycal, Thanks for your quick response and information. So you mean if we want to accelerate our python receiver by GPU, it can be realized as followings: 1. use keras layer in Python with Keras/Tensorflow framework, it can be automatically accelerated by GPU. (even it only runs once? It can speed up by allocated different layers in different thread/block? How to understand that ?) 2. Use Nvidia Cuda python? As https://developer.nvidia.com/how-to-cuda-python ? What’s your opinions and recommendation of these two ways? Does ② execution is faster than ①， but easy to go ？ Could you please share more information about that, that’s really helpful. Thanks. Br. Sophie From: Fayçal Ait-Aoudia ***@***.***> Sent: Wednesday, April 27, 2022 4:18 PM To: NVlabs/sionna ***@***.***> Cc: Sophie Wei ***@***.***>; Author ***@***.***> Subject: Re: [NVlabs/sionna] Q&A Can sionna is accelerated by GPU when only run once ? (Discussion #10) Thank you for your interest in Sionna! To benefit from GPU acceleration, you can implement your receiver as a Keras layer in Python using the TensorFlow/Keras framework. You can find more information about this here: https://keras.io/guides/making_new_layers_and_models_via_subclassing Once this is done, your new layer can be integrated into a Sionna-based communication system. We have provided a notebook that shows how to implement a neural network-based receiver as a Keras layer with Sionna, which could be helpful: https://nvlabs.github.io/sionna/examples/Neural_Receiver.html GPU acceleration happens independently of the batch size, as long as your receiver is implemented using Keras/TensorFlow. Moreover, TensorFlow provides different execution modes that enable different levels of acceleration. You can find more information in this notebook (section Eager vs Graph Mode): https://nvlabs.github.io/sionna/examples/Sionna_tutorial_part1.html Note that If you need to implement your receiver in C++/CUDA, we plan to provide a tutorial soon. — Reply to this email directly, view it on GitHub<#10 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AY5FGEBSJ3WV3IUT2QOL4ZLVHDZ3RANCNFSM5UOAO26A>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[Inv4lidn4m3]

Hello,

My 2 cents:

• Numba is really powerful, but it brings many constrains to your code and, trust me, it may be not easy to adapt your code to make it numba compliant. Morevover, depending on the numba version, it will have a limited numpy functions support. I would recommand Numba if you do not plan to modify your code furthera and you just want to make it faster or if you really know what you are doing.

• Tensorflow-gpu will automatically detect and support the GPU. But this also brings some constraints. You need to implement the code you wish to execute as a tensorflow model or at least sucession of tf layers. To have an efficient run you need to enable the graph-mode which means remove any dynamic loops and conditions (these kind of process can be implemented in TF but it is not the spirit of the library). Also best pratice is to remove numpy arrays and only work with tensors, but most of the numpy functions have their counter part in tf. Or at least you can find a way to express your algorithm with TF functions. Usually you will use the batch to automatically parallelize the runs. Again, if your code is compatible, TF will handle the parallelization and use of GPU. Even with a single batch (no batch parallelization) and depending on the kind of instructions you need in your code, the GPU can still bring acceleration (for instance matmul operation is a one shot instruction in the GPU).

I don't know which option will be the fastest at the end since it will greatly depend on the quality of the code and the kind of process you need. Since I'm not a developer, I prefer to use TF since it will efficiently handle everything for me.

[cheerkwah]

Hello, thanks a lot for your reply.
We finally setup the sionna on our GPU environment and can run the examples now.
And we tried the different BATCH_SIZE and test the running time of sionna on CPU and GPU.
We tried the example/Sionna_tutorial_part1.jpynb

when set BATCH_SIZE = 2000, it seems this case run on GPU is about 5 times faster than it on the CPU
when we changed the BATCH_SIZE = 1, it seems it on GPU is about 1.5 times slower than it on the CPU ....

BATCH_SIZE = 2000:

BATCH_SIZE = 1:

As we are just starter of sionna, could you please confirm this results is reasonable?
And can it imply that most speeding of sionna is due to the parallelization of the BATCH_SIZE ?

[SebastianCa]

Your results are reasonable, however, they depend on your exact CPU/GPU configuration and the simulated model.

As usually done in deep learning, one fundamental design paradigm of Sionna is to make use of parallel batch-processing whenever possible.

Please note that for the GPU-accelerated experiments, your simulation time is almost constant (39s vs. 44s per SNR point), although 2000x more bits have been simulated. The reason is simply that the GPU utilization is low (you can check the GPU load with nvidia-smi).

Keep in mind that XLA acceleration may further speed-up your simulation.

[cheerkwah]

Got it, thanks for your reply.
So if we can break down our receiver algorithm into repeated keras model/layer (if it is suitable), then it can be speed up by set multiple batch size by utilizing GPU ?

[Inv4lidn4m3]

It is not that easy as it depends on the actual operations you are doing within your code, but generally speaking as long as it feet in the GPU memory, it should be faster.

[chrisDeCstV]

Hi cheerkwak, everybody,

Eventually, how did you made the code run on GPU ? I am running Sionna_tutorial_part4 on my local windows machine and it is using a single CPU core (11.2%) and no GPU. I am new to this, so no answer is trivial for me.

As a side thing, there are also a bunch of warnings like tensorflow:AutoGraph could not transform <bound method OFDMSystemNeuralReceiver.call of <tensorflow.python.eager.function.TfMethodTarget object at 0x00000199C72DE400>> and will run it as-is. Cause 'arguments' object has no attribute 'posonlyargs'

Thank you !

[SebastianCa]

Hi,

For GPU support you need to ensure that you have installed the latest CUDA drivers & CUDA Toolkit and that TensorFlow can access the GPU. We recommend that you follow the step-by-step instructions from [1] depending on your OS.

Sionna itself does not need any specific configuration for GPU acceleration (as it uses TF as backend).

[1] https://www.tensorflow.org/install/pip