Skip to content

Build dunedaq v2.10.1 in spack with docker

Jump to bottom
Pengfei Ding edited this page Apr 2, 2022 · 11 revisions

Required system packages

System packages found in this Dockerfile;

The docker image built with this file is: dunedaq/c8-minimal:spack

Steps to build and install dunedaq@dunedaq-v2.10.1

Prepare for the installation

  • Start the docker container Firstly, create a new directory on host machine, change into the new directory, and run the following docker command:
docker run --rm -it --net=host -v $PWD:/cvmfs/dunedaq-development.opensciencegrid.org dunedaq/c8-minimal:spack

Detailed steps

  • Build and install dunedaq@dunedaq-v2.10.1. Step-by-step descriptions can be found in the comments below. The modified YAML files can be found in the next section.
cd /cvmfs/dunedaq-development.opensciencegrid.org/
mkdir sandbox
cd sandbox/

# Only need the following if you are on NP04.
# export http_proxy=http://np04-web-proxy.cern.ch:3128; export https_proxy=http://np04-web-proxy.cern.ch:3128; export no_proxy=".cern.ch"

# Checkout daw-release repo, and generate spack repo for dunedaq-v2.10.1
git clone https://github.com/DUNE-DAQ/daq-release -b dingpf/spack-release
python3.8 daq-release/scripts/spack/make-release-repo.py -t daq-release/spack-repos/release-repo-template -i daq-release/configs/dunedaq-v2.10.1/dunedaq-v2.10.1.yaml -u -o releases/dunedaq-v2.10.1

# install spack v0.17.1
mkdir -p tools/spack
cd tools/spack/
wget https://github.com/spack/spack/archive/refs/tags/v0.17.1.tar.gz
tar xvf v0.17.1.tar.gz
rm -f v0.17.1.tar.gz
cd ../../

# copy over the dune-externals repo
cp -pr daq-release/spack-repos/externals spack-externals

# setup spack
. tools/spack/spack-0.17.1/share/spack/setup-env.sh

# find system compiler
spack compiler find

# add two repos: dune-externals, dunedaq-v2.10.1
spack repo add spack-externals
spack repo add release/dunedaq-v2.10.1/spack-repo
# modify default repos.yaml to include two added repos.
# details can be found in the next section.
cat ~/.spack/repos.yaml >>$SPACK_ROOT/etc/spack/defaults/repos.yaml ; vi $SPACK_ROOT/etc/spack/defaults/repos.yaml

# modify installation path to remove arch name
# details can be found in the next section.
vi $SPACK_ROOT/etc/spack/defaults/config.yaml

# install gcc@8.2.0 and modify default compilers.yaml
spack install gcc@8.2.0 +binutils
spack load gcc@8.2.0
spack compiler find
# modify default compilers.yaml to 1) remove system compiler; 2) leave gcc@8.2.0 as the only compiler available
# details can be found in the next section.
cp /root/.spack/linux/compilers.yaml $SPACK_ROOT/etc/spack/defaults/linux; vi $SPACK_ROOT/etc/spack/defaults/linux/compilers.yaml

# create spack env for dunedaq-v2.10.1; env name cannot contain ".", use "-" instead
spack env create dunedaq-v2-10-1

# Activate spack env, and add the umbrella package "dunedaq@dunedaq-v2.10.1"
spack env activate dunedaq-v2-10-1 -p
spack add dunedaq@dunedaq-v2.10.1%gcc@8.2.0

# run concretizer to ensure no conflicts between packages, do the installation.
spack concretize
spack install

Modified files

  • $SPACK_ROOT/etc/spack/defaults/repos.yaml, the file will look like: (:boom: make sure built-in repo came at the last)
repos:
  - /cvmfs/dunedaq-development.opensciencegrid.org/sandbox/release/dunedaq-v2.10.1/spack-repo
  - /cvmfs/dunedaq-development.opensciencegrid.org/sandbox/spack-externals
  - $spack/var/spack/repos/builtin
  • $SPACK_ROOT/etc/spack/defaults/config.yaml, remove ${ARCHITECTURE}/ in the config.install_tree.projections.all field.
config:
  # This is the path to the root of the Spack install tree.
  # You can use $spack here to refer to the root of the spack instance.
  install_tree:
    root: $spack/opt/spack
    projections:
      all: "${COMPILERNAME}-${COMPILERVER}/${PACKAGE}-${VERSION}-${HASH}"
      #all: "${ARCHITECTURE}/${COMPILERNAME}-${COMPILERVER}/${PACKAGE}-${VERSION}-${HASH}"
  • $SPACK_ROOT/etc/spack/defaults/linux/compilers.yaml, the file will look like:
compilers:
- compiler:
    spec: gcc@8.5.0
    paths:
      cc: /usr/bin/gcc
      cxx: /usr/bin/g++
      f77: /usr/bin/gfortran
      fc: /usr/bin/gfortran
    flags: {}
    operating_system: centos8
    target: x86_64
    modules: []
    environment: {}
    extra_rpaths: []
- compiler:
    spec: gcc@8.2.0
    paths:
      cc: /cvmfs/dunedaq-development.opensciencegrid.org/sandbox/spack-installation/spack-0.17.1/opt/spack/linux-centos8-broadwell/gcc-8.5.0/gcc-8.2.0-argfipwyx6scglauduwlu666npsmy2zu/bin/gcc
      cxx: /cvmfs/dunedaq-development.opensciencegrid.org/sandbox/spack-installation/spack-0.17.1/opt/spack/linux-centos8-broadwell/gcc-8.5.0/gcc-8.2.0-argfipwyx6scglauduwlu666npsmy2zu/bin/g++
      f77: /cvmfs/dunedaq-development.opensciencegrid.org/sandbox/spack-installation/spack-0.17.1/opt/spack/linux-centos8-broadwell/gcc-8.5.0/gcc-8.2.0-argfipwyx6scglauduwlu666npsmy2zu/bin/gfortran
      fc: /cvmfs/dunedaq-development.opensciencegrid.org/sandbox/spack-installation/spack-0.17.1/opt/spack/linux-centos8-broadwell/gcc-8.5.0/gcc-8.2.0-argfipwyx6scglauduwlu666npsmy2zu/bin/gfortran
    flags: {}
    operating_system: centos8
    target: x86_64
    modules: []
    environment: {}
    extra_rpaths: []
Clone this wiki locally