unimplemented for this system in file for_each.h

[ToddThomson]

I have a storage container class that essentially wraps a private thrust::host_vector<T> buffer_;
The storage class is part of a static library. The library compiles without any issues.
When the static library is linked to an application and a compilation unit references the class containing the storage class I get a c2338 compile error with "unimplemented for this system" in file for_each.h. This occurs in the creation of the storage container class.
I've read the docs and I pretty certain that I need to set the Host and Device systems, but I can't figure it out. It seems the defaults are CPP and CUDA which is what I want.
If I replace the host_vector with a std::vector then the storage class is created without a compiler error.
It shouldn't matter but the static library uses C++23 modules.
Thanks in advance for any help!

[ToddThomson]

Okay, I've got a solution to the problem, but I can't say why it works...
If I add an #include <thrust/host_vector.h> in the executable translation unit that references the static library storage class and also have the executable link to CUDA::cudart the issue is resolved.
I'd appreciate any explanation. I'd gladly give additional context if you feel there is not enough information to understand the problem.

[bernhardmgruber]

Hi! My estimate is that the static library was compiled with a CUDA comiler, and the application with a non-CUDA compiler. Also, your "storage container class" may be declared in a header file that is included into the non-CUDA application. But it's hard to judge. Can you give us a sketch of your code please?

[ToddThomson]

Hi Bernhard, thank-you for your input!

Here is the link to the GitHub repo: Mila

I've pushed a commit to help document the issue.

The issue arises in the Tensor.ixx class which uses TensorBuffer.cuh. If you change the TensorBuffer to default to a VectorType of thrust::device_vector all executables which link to Mila get the compilation error.

As background, I was quite interested in the llm.c repos, both the original Karpathy code and the Thrust port. I wanted to utilize both to have a general c++23 modules based port. I'd really like to use Thrust, but this issue is slowing me down. I'll soon just write my own storage class if I cannot figure out this issue.

Thank-you in advance for any help.

[bernhardmgruber]

I don't immediately spot any wrong usage :/ Unfortunately, I don't have the bandwidth right now to dig into this.

[ToddThomson]

I couldn't figure out how to use Thrust in a static library when attempting to use thrust::device_vector. It is too bad as I feel that Thrust is exactly the software I need. I will revisit this issue, but for now I've written my own memory resources derived from std::pmr::memory_resource.
Thank-you Bernhard.

[ToddThomson]

Hi Bernhard, I'm hoping that you may be able to help me out.
I've updated Mila to use MRs to manage/allocate the Tensor buffer. This works perfectly for keeping most of my code base as modules.
I would still like to use Thust. My question is: given that I now have a pointer to Cuda memory from using the Cuda specific MRs could I create a Thrust pointer and then somehow call Thrust operations? For example Thrust::Fill().
Thanks in advance for any help!

[bernhardmgruber]

Hi! Sorry for the late response!

I've updated Mila to use MRs to manage/allocate the Tensor buffer.

That's great! MRs are the future!

given that I now have a pointer to Cuda memory from using the Cuda specific MRs could I create a Thrust pointer and then somehow call Thrust operations?

There is no need to create a Thrust pointer, just pass the raw T* to Thrust. Since Thrust tries to detect the execution space from the pointer, and a raw pointer will default to host, you have to pass thrust::device as first argument to Thrust algorithms.

For example Thrust::Fill().

T* ptr_to_gpu_memory = ...
thrust::fill(thrust::device, ptr_to_gpu_memory, ptr_to_gpu_memory + size, value);

should do what you need.

[ToddThomson]

Hi Bernhard. No worries. I understand that you're busy. Thank-you for your response.

I will definitely revisit using Thrust in the near future. For now, I couldn't figure out the compilation problems so I had to press on. I've kept a small, experimental thrust based Tensor project and I'll play with your suggestions above.

Thanks!