Error: Integrating LibTorch C++ Library in a Kratos Application

[ashwakumar]

Hello Kratos Community,

I am working on integrating the LibTorch C++ library (PyTorch C++ frontend) into a custom Kratos application.
To achieve this, I modified the CMakeLists.txt file in the Kratos folder to include the necessary flags and link the LibTorch library. However, during the compilation, I encountered an error, and the build fails. The error message suggests issues with linking or target definitions. Here is a summary of what I’ve done:
I am compiling on Linux platform (WSL).
This is the change I made in CMakeLists.txt in kratos folder.

find_package(Torch REQUIRED)
if(Torch_FOUND)
  message("-- LibTorch found")
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")
  #include_directories( ${KRATOS_SOURCE_DIR}/external_libraries/libtorch )
  target_link_libraries(KratosMultiphysics PRIVATE "${TORCH_LIBRARIES}")
  #set_property(TARGET KratosMultiphysics PROPERTY CXX_STANDARD 17)
else()
  message(FATAL_ERROR "LibTorch not found. Please install or specify the correct path.")
endif()

This is the change in configure.sh file

-DCMAKE_PREFIX_PATH="/mnt/d/Kratos_All/Kratos/external_libraries/libtorch"

This is the error message:

-- LibTorch found
CMake Error at CMakeLists.txt:591 (target_link_libraries):
  Cannot specify link libraries for target "KratosMultiphysics" which is not
  built by this project.

Questions:

1. Has anyone successfully integrated LibTorch into Kratos? If so, could you share your approach?
2. Are there specific changes required in the Kratos build system for adding external libraries like LibTorch?
3. Any advice on debugging or resolving the target_link_libraries error?
   Thank you in advance for your help!

• Compilation
• KratosCore

[loumalouomega]

Hello, first advice, do not put your library in external_libraries folder, that's for external dependencies integrated into Kratos.

Your problem looks like you are linking with something that doesn't exist: target_link_libraries(KratosMultiphysics PRIVATE "${TORCH_LIBRARIES}"), you should do it with KratosCore. In teh same file you have multiple examples of libraries linked that you can use as reference.

[ashwakumar]

Hi @loumalouomega, Thank you for your suggestions, It worked :)

[loumalouomega]

@roigcarlo FYI

[roigcarlo]

Hi @ashwakumar

First of all, @loumalouomega suggestions are correct.

About the integration of TPL (third party libraries) inside Kratos, It is possible and in fact something that we do in a regular bases. I will try to detail all the cases and how to approach each.

First, it is important to mention that you can integrate a library in kratos in two different ways:
1 - Compiled alongside Kratos: This will result in having one single shared object containing both Kratos and your lib. This is the approach that we use for example for our json parser, etc etc...
2 - As an independent library. This will result into making Kratos.so dependent on the TPL. This is the approach that we follow for Testing, External Meshers (MMG), Linear Solver libraries (Eigen, Pardion, Trilinos...) and many others.

For what I can see, you have tried to go for a mix between the to solutions, but I recommend that you go for the second.

While PyTorch has never been tested, the procedure should be very similar to any other TPL with some details:
1 - Download the precompiled binarys for torch from they website. They need to be the cxx11 ABI ones. Otherwise it wont work.
Note that this step is completely independent on Kratos

2 - Deifne the CMake block that allows to find the lib. You do that by adding just the lines below in $KRATOS_ROOT/CMakeLists.txt:

##*****************************
# Finding pyTorch if needed
if(${USE_TORCH} MATCHES ON)
  find_package(Torch REQUIRED)
endif(${USE_TORCH} MATCHES ON)

3 - Once this is done, you need to link those libraries to the Kratos project. This is what you tried to do with the target_link_libraries but that needs to be done in: $KRATOS_ROOT/kratos/CMakeLists.txt:122 (note that is a different file)

target_link_libraries(KratosCore PUBLIC ${KRATOS_EXTERNAL_LIBRARIES} ${TORCH_LIBRARIES})

4 - Change the configure to accommodate all the modifications. Should look something like (ofc change /home/roigcarlo/libtorch for your dir):

# Configure
cmake -H"${KRATOS_SOURCE}" -B"${KRATOS_BUILD}/${KRATOS_BUILD_TYPE}" -G Ninja    \
-DUSE_CUDA=OFF                                                                  \
-DUSE_TORCH=ON                                                                 \
-DKRATOS_GENERATE_PYTHON_STUBS=OFF                                              \
-DCMAKE_PREFIX_PATH="/home/roigcarlo/libtorch"                                  \
-DCMAKE_UNITY_BUILD=OFF

If done correctly, you should see that the resulting libKratosCore.so is indeed dependent on torch and cuda:

libs ❯ ldd libKratosCore.so                                                                                                                                                           test/torch_lib
	linux-vdso.so.1 (0x00007fd144223000)
	libtorch.so => /home/roigcarlo/libtorch/lib/libtorch.so (0x00007fd144218000)
	libc10.so => /home/roigcarlo/libtorch/lib/libc10.so (0x00007fd144124000)
	libnvrtc.so.12 => /opt/cuda/lib64/libnvrtc.so.12 (0x00007fd13ce00000)
	libc10_cuda.so => /home/roigcarlo/libtorch/lib/libc10_cuda.so (0x00007fd140c82000)
	libz.so.1 => /usr/lib/libz.so.1 (0x00007fd140c69000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007fd140b7a000)
	libtorch_cpu.so => /home/roigcarlo/libtorch/lib/libtorch_cpu.so (0x00007fd127400000)
	libtorch_cuda.so => /home/roigcarlo/libtorch/lib/libtorch_cuda.so (0x00007fd0d0000000)
	libcudart.so.12 => /opt/cuda/lib64/libcudart.so.12 (0x00007fd0cfa00000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fd0cf600000)
	libomp.so => /usr/lib/libomp.so (0x00007fd140a5e000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fd140a2e000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007fd0cfe0f000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007fd144225000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fd13cdfb000)
	librt.so.1 => /usr/lib/librt.so.1 (0x00007fd13cdf6000)
	libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fd13cdf1000)
	libgomp-98b21ff3.so.1 => /home/roigcarlo/libtorch/lib/libgomp-98b21ff3.so.1 (0x00007fd13cdaa000)
	libcudart-9335f6a2.so.12 => /home/roigcarlo/libtorch/lib/libcudart-9335f6a2.so.12 (0x00007fd0cf000000)
	libcublas-37d11411.so.12 => /home/roigcarlo/libtorch/lib/libcublas-37d11411.so.12 (0x00007fd0c8400000)
	libcublasLt-f97bfc2c.so.12 => /home/roigcarlo/libtorch/lib/libcublasLt-f97bfc2c.so.12 (0x00007fd0a6200000)
	libcudnn.so.9 => /home/roigcarlo/libtorch/lib/libcudnn.so.9 (0x00007fd0a5c00000)

[ashwakumar]

Hi @roigcarlo, thank you so much for the detailed solution. It compiled perfectly :-).