hydro tile size

[indra098124]

I have a question regarding how hydro_tile_size is utilized in Castro_ctu_hydro.cpp and the benefits it provides. I noticed that it is set to 16 unless the memory footprint exceeds a certain threshold.

Could you clarify how this impacts performance or memory efficiency? Specifically, I see it being used in the following loop:

for (MFIter mfi(S_new, hydro_tile_size); mfi.isValid(); ++mfi) { 
    // ...
}

I’d appreciate any insights into the rationale behind this implementation. Thanks

[zingale]

Take a look at the docs here:
https://amrex-astro.github.io/Castro/docs/mpi_plus_x.html#running-on-gpus

In short, the hydro creates a lot of temporary FABs, which can consume a lot of memory. To reduce this effect, we can use tiling to make the region of a box that hydro is working on smaller, so we use a different tile size there than in the rest of the code. For GPUs, this can be especially important, and we can try to pick an optimal hydro tile size to ensure that we don't exceed the GPU memory. This is controlled via the castro.hydro_memory_footprint_ratio parameter.

[indra098124]

Thanks @zingale