Update descriptions and comments

BUILD_FOR_CRYSTFEL.md

@@ -1,6 +1,6 @@
 # Building the Fast Feedback Indexer Library for CrystFEL on Linux Systems
 
-Since CrystFEL uses meson as it's build system, the following will assume meson is installed. The code must be compiled with a C++17 and gnu compiler option compatible compiler (like g++-13 or clang++-18). A CUDA installation is also required. The code requires the Eigen library.
+Since CrystFEL uses meson as it's build system, the following will assume meson is installed. The code must be compiled with a C++17 and gnu compiler option compatible compiler (like g++-13 or clang++-18). A CUDA installation, compatible with your compiler and GPU driver, is also required. The code requires the Eigen library.
 
 ## Install the Fast Feedback Indexer Library
 
@@ -11,6 +11,10 @@ Start with cloning the fast feedback indexer library repository:
     $ git clone https://github.com/paulscherrerinstitute/fast-feedback-indexer
     $ cd fast-feedback-indexer
 
+Then you can use the newest code as is, or check out a release, e.g. v1.1.2, with
+
+   $ git checkout v1.1.2
+
 ### Eigen Library (Option eigen-source-dir)
 
 If you have a working installation of the Eigen library, this option can be set to 'ignore'. Otherwise you can set it to 'eigen' after checking out the Eigen library source code with
@@ -29,9 +33,16 @@ In order to determin the compute capability of the GPUs the code should be able
 
     $ nvidia-smi --query-gpu=compute_cap --format=csv,noheader
 
-This information will end up as a single --gencode='xxx' nvidia compiler option, see [nvcc compiler options](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-steering-gpu-code-generation). So if the output of the nvidia-smi comand is 8.9, the value of 'xxx' can be set to 'arch=compute_89,code=sm_89'
+This information will end up as a single --gencode='xxx' nvidia compiler option, see [nvcc compiler options](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-steering-gpu-code-generation). So if the output of the nvidia-smi comand is 8.9, the value of 'xxx' can be set to 'arch=compute_89,code=sm_89'. The value can be set to 'ignore', then no --gencode option will be given to the Nvidia compiler.
+
+If you optimize the code this way, it's important to be aware of the fact that the code will only be able to run on the devices compiled for. The value can be set to 'ignore', then no --gencode option will be given to the Nvidia compiler.
+
+In general, the Nvidia GPU driver must be compatible with the output produced by the nvcc CUDA compiler. The version of the driver and compiler can be printed with
+
+    $ nvidia-smi --query-gpu=driver_version --format=csv,noheader
+    $ nvcc --version
 
-The value can be set to 'ignore', then no --gencode option will be given to the Nvidia compiler.
+If the driver is not compatible (e.g. too old), you'll have to ask the system administrator to upgrade the driver. Alternatively, you may downgrade your CUDA installation (of which nvcc is a part of) to a version compatible with the driver, see [CUDA compatibility](https://docs.nvidia.com/deploy/cuda-compatibility).
 
 #### Determine CPU Architecture (Option cpu-arch)
 
@@ -73,22 +84,29 @@ Amongst others, this sets up the LD_LIBRARY_PATH and PKG_CONFIG_PATH. The pkg-co
 
 ## Build and Install CrystFEL
 
-Presently, a [forked version](https://github.com/hcstadler/crystfel) of CrystFEL by *Filip Leonarski* and others is required to integrate the fast feedback indexer into CrystFEL. Clone the code with
+Clone the code with
 
-    git clone https://github.com/hcstadler/crystfel
+    git clone https://github.com/taw10/crystfel.git
     cd crystfel
 
 Configure the build with
 
     meson setup --reconfigure --buildtype=release --prefix=${BASE_DIR}/crystfel build-crystfel
 
-Then compile and install the CrystFEL code with
+Make sure the runtime dependency 'fast_indexer' is found by checking the output. Then compile and install the CrystFEL code with
 
     cd build-crystfel
     meson compile
     meson install
 
-Then you can add "--indexing=ffbidx" to indexamajig options. Fast Feedback Indexer and CrystFEL lib directories have to be in the LD_LIBRARY_PATH for the execution, for the Fast Feedback Indexer this is easiest achieved by setting up the sh compatible shell environment as described above under **USE**.
+Make sure '${BASE_DIR}/crystfel/bin' is in your PATH and '${BASE_DIR}/crystfel/lib/x86_64-linux-gnu' in your LD_LIBRARY_PATH.
+Then you can add '--indexing=ffbidx' to indexamajig options to use the Fast Feedback Indexer.
+
+Fast Feedback Indexer and CrystFEL lib directories have to be in the LD_LIBRARY_PATH for the execution, for the Fast Feedback Indexer this is easiest achieved by setting up the sh compatible shell environment as described above under **USE**.
+
+To print possible configuration options for the Fast Feedback Indexer, you can run
+
+    $ indexamajig --help-ffbidx
+
+The indexer algorithm and its options are described in the LaTeX documents in the 'doc' folder of the Fast Feedback Indexer repository.
 
-To print possible configuration options, you can run:
-indexamajig --help-ffbidx

README.md

@@ -2,7 +2,7 @@
 
 Develop an indexer for fast feedback
 
-*Status*: We're optimistic that it does what it's supposed to. *Filip Leonarski* will integrate this code into CrystFEL.
+*Status*: We're very optimistic that it does what it's supposed to. *David Waterman* has integrated this indexer into DIALS. *Thomas White* has integrated this indexer into CrystFEL.
 
 *Luis Barbas* PyTorch implementation has been benchmarked extensively here at PSI and shows indexing quality on par with other known indexers and superior speed. This CUDA version has been tested internally by *Duan Jiaxin*, shows indexing quality on par with XGandalf and superior speed.
 
@@ -11,8 +11,7 @@ Develop an indexer for fast feedback
 *Issues*:
 
 * The refinement algorithms misbehave with electron diffraction data
-* The refinement algorithms misbehave with small molecule data
-* Implemented in CUDA, so only Nvidia GPUs are supported currently. With a suitable development machine, porting the code to HIP looks like a very realistic possibility. Porting to SYCL could also be an option.
+* Implemented in CUDA, so only Nvidia GPUs are supported currently.
 
 ### Alternative Implementations
 
@@ -44,7 +43,7 @@ This is the result of the REDML Project, and other more or less informal collabo
 ### External Build Dependencies
 
 * C++17 compatible compiler
-* cmake > 3.21 (not so sure if it works with earlier versions as well)
+* cmake > 3.21 (not so sure if it works with earlier versions as well) or meson >= 1.1
 * Eigen3 header only library
 * *BUILD_FAST_INDEXER* needs a compiler compatible CUDA toolkit
 * *PYTHON_MODULE* needs Python3 and NumPy (also see https://cmake.org/cmake/help/latest/module/FindPython3.html)
@@ -53,6 +52,8 @@ This is the result of the REDML Project, and other more or less informal collabo
 
 Cmake should complain and tell you what to add to the cmake commandline if internal dependencies are not met. In general tests require what is tested, executables need the simple data reader and indexer libraries, and the python module needs the indexer library.
 
+Meson should also complain if requested features cannot be built and tell you what to add to the commandline.
+
 ### Build Instructions
 
 If the default value of *CMAKE_CUDA_ARCHITECTURES*=\"75;80\" is inappropriate for your GPU, set it correctly, or go to the bottom of the README file for the indexer to get more info.
@@ -73,6 +74,19 @@ $ make
 $ ctest
 ```
 
+or with meson from the repository top directory after adapting these lines (especially the cpu-arch and gpu-arch) to your own needs:
+```
+$ FFBIDX_INSTALL_DIR=${HOME}/ffbidx
+$ CXX=g++-13 meson setup --reconfigure --buildtype=release --prefix=${FFBIDX_INSTALL_DIR} \
+  --libdir=lib -D install-simple-data-reader=enabled -D install-simple-data-files=enabled \
+  -D build-tests=enabled -D build-simple-indexers=enabled -D include-python-api=enabled \
+  -D python-installation=python3 -D default_library=both \
+  -D cpu-arch=native -D gpu-arch='arch=compute_89,code=sm_89' meson
+$ cd meson
+$ meson compile -v
+$ meson test
+```
+
 ### Installation Instructions
 
 Take note of the CMAKE_INSTALL_PREFIX value above.
@@ -84,7 +98,14 @@ $ # Adapt the script if you're using something else.
 $ . ${FFBIDX_INSTALL_DIR}/share/ffbidx/setup-env.sh
 ```
 
-As a quick installation test you could do
+for Cmake and with meson, use
+
+```
+$ meson install
+$ . ${FFBIDX_INSTALL_DIR}/share/ffbidx/setup-env.sh
+```
+
+If you included the simple indexers and python interface in the build, a quick installation test would be
 
 ```
 $ refined_simple_data_indexer \
@@ -129,26 +150,8 @@ spack load ffbidx
 
 See: https://git.psi.ch/germann_e/spack-psi
 
-### Installation with Meson
-(This was tested with meson-1.4.0 and ninja-1.10.1)
-
-In the top code directory do the following after adapting these lines (especially the cpu-arch and gpu-arch) to your own needs:
-```
-$ FFBIDX_INSTALL_DIR=${HOME}/ffbidx
-$ CXX=g++-13 meson setup --reconfigure --buildtype=release --prefix=${FFBIDX_INSTALL_DIR} \
-  --libdir=lib -D install-simple-data-reader=enabled -D install-simple-data-files=enabled \
-  -D build-tests=enabled -D build-simple-indexers=enabled -D include-python-api=enabled \
-  -D python-installation=python3 -D default_library=both \
-  -D cpu-arch=native -D gpu-arch='arch=compute_89,code=sm_89' meson
-$ cd meson
-$ meson compile -v
-$ meson test
-$ meson install
-$ . ${FFBIDX_INSTALL_DIR}/share/ffbidx/setup-env.sh
-```
-Then do the qiuck installation test above.
+### Installation for Python using micromamba and conda-build
 
-### Installation for Python using micromamba and conda-build (experimental)
 ```
 $ micromamba install conda-build
 $ conda-build -c conda-forge conda/meta.yaml

indexer/README.md

@@ -91,4 +91,16 @@ Steer library behaviour with environment variables.
 The Meson build differs in certain points from the CMake build.
 * GPU and CPU architecture can be left undefined
 * Less control over rpaths and compiler options
+* Option cpu-arch=xxx sets the -march=xxx compiler flag
+* Option gpu-arch=xxx sets the --gencode=xxx nvcc flag
+* CXX and CXXFLAGS will be picked up in the build setup
+* NVCC_PREPEND_FLAGS and NVCC_APPEND_FLAGS can be used to set nvcc flags at compile time
 See the meson.options file for available options.
+
+### Using the library in your own code
+
+Depending on what is included in the build, the setup-env.sh script sets up a bash environment that enables
+* PKG_CONFIG_PATH for the Linux pkg-config mechanism ('fast_indexer' or 'fast_indexer_static') to pickup linking flags and include directories, e.g. in Cmake or meson projects
+* LD_LIBRARY_PATH for linking at runtime
+* CPLUS_INCLUDE_PATH and C_INCLUDE_PATH for picking up includes
+* PYTHONPATH for importing the ffbidx python package

meson.options

@@ -1,4 +1,6 @@
-# Compile options, can be set to 'ignore'
+# Compile options, can be set to 'ignore' to avoid setting the flags
+# cpu-arch=xxx sets the -march=xxx compiler flag
+# gpu-arch=xxx sets the --gencode=xxx nvcc flag
 option('cpu-arch', type: 'string', value: 'ignore')
 option('gpu-arch', type: 'string', value: 'ignore')
 option('all-warnings', type: 'feature', value: 'disabled')