diff --git a/.github/workflows/ci-dsl.yaml b/.github/workflows/ci-dsl.yaml index 94367c4bd..f750869b0 100644 --- a/.github/workflows/ci-dsl.yaml +++ b/.github/workflows/ci-dsl.yaml @@ -35,6 +35,10 @@ jobs: WRITE_JOB_LOG=1 java -jar tools/jobdsl.jar *.dsl find logs/* -exec sort {} -o {} \; popd + - name: Checks for existing scripts in docker/ + run: | + cd jenkins-scripts/docker + ./sanity_checks.bash - name: Checks for DSL Code run: | cd jenkins-scripts/dsl diff --git a/bloom/ros_gazebo_pkgs/README.md b/bloom/ros_gazebo_pkgs/README.md deleted file mode 100644 index 43af4d34e..000000000 --- a/bloom/ros_gazebo_pkgs/README.md +++ /dev/null @@ -1,239 +0,0 @@ -# Scripts - -## Release new version of Gazebo unofficial wrappers - -### Quick commands - -1. bloom in the osrf/ repository -```bash - $ BLOOM_RELEASE_REPO_BASE=https://github.com/osrf/ bloom-release --no-pull-request --rosdistro --track gazeboX_ros[2]_pkgs - - Example - $ BLOOM_RELEASE_REPO_BASE=https://github.com/osrf/ bloom-release --no-pull-request --rosdistro foxy --track foxy gazebo11_ros2_pkgs -``` - -1. Trigger Jenkins jobs with ros_gazebo_pkgs-release.py.bash -```bash - $ ros_gazebo_pkgs-release. 'other arguments used in release.py'* - - Example: - $ ros_gazebo_pkgs-release.py.bash 3.4.4 https://github.com/osrf/gazebo11_ros2_pkgs-release foxy xxx -r 1 --dry-run -``` - -### Background - -Each ROS release defines one version of Gazebo supported officially through -all the ROS packages. The different combinations of ROS <-> Gazebo can be -found in the [REP-3](http://www.ros.org/reps/rep-0003.html). Some examples: - - * ROS Kinetic: Gazebo 7 - * ROS Lunar: Gazebo 7 - * ROS Melodic: Gazebo 9 - -Some use cases require the use of alternative combinations of ROS and Gazebo -versions. The `gazebo_ros_pkgs` code is usually prepared to be compatible with -different versions of Gazebo, especially the latest ones. - -To release a modified version of `gazebo_ros_pkgs` which supports a different -major version of gazebo, before running bloom some actions need to be taken: - - 1. Make the system resolve the Ubuntu package `gazebo` as `gazeboX`. - 2. Fork and modify the official [gbp -release repo](https://github.com/ros-gbp/gazebo_ros_pkgs-release). - 3. Run bloom on the modified release repository. - 4. Custom infrastructure to create .deb packages. - -We'll go over these steps in more detail below. - -### Upstream versions released using this tutorial - -The `gbp -release repository` hosts the latest version released by the maintainers -of `gazebo_ros_pkgs`. When using these instructions to release a new custom version -the version of `gazebo_ros_pkgs` released will be the latest one existing in the -official `gbp -release repository`. The version would be the same but the release number -will be one above since bloom needs to be run on top of the modified `-release repository`. - -### Create a new gazeboX_{ros,ros2}_pkgs repository - -For a new official wrappers the notation used below correspond to: -`gazeboX_rosY_pkgs` (`X` is major version in Gazebo, `Y` is 1 or 2 for ROS) - - 1. Create new repo under github.com/osrf/ organization named: - - ROS1 https://github.com/osrf/gazeboX_ros_pkgs-release - - ROS2 https://github.com/osrf/gazeboX_ros2_pkgs-release - - 1. Run initial_info_for_new_release_repo.bash - It will import all branches and tags from official gbp into the new repo - - Usage: *$ initial_info_for_new_release_repo X rosY* - - 1. Clone the new repo, go to the directory and run rename-gazebo-ros-pkgs.bash - - Usage: *$ rename-gazebo-ros-pkgs.bash X * - -### Make the system resolve gazeboX - -To have `gazebo` packages be resolved to `gazeboX` in the local -machine where bloom-release will be run, `rosdep` should be configured -accordingly. - -The repository `https://github.com/osrf/osrf-rosdep/` hosts different sets -of custom rosdep keys that can be deployed in a system in order to override -the default configuration of resolving gazebo packages. - -At `/etc/ros/rosdep/sources.list.d` there should be a default file named -`20-default.list`. To change the default keys by a custom version of gazebo -packages, the desired `00-gazeboX.list` from the previously indicated repository -needs to be download to the same directory. - -Update the rosdep cache: - - $ rosdep update - -To double check: run rosdep resolve, it should resolve to right major versions: - - $ rosdep resolve gazebo - $ rosdep resolve gazebo_ros - - -### Run Bloom on the custom -release repository - -The point of this last step is to generate the ubuntu packaging metadata from bloom templates since they were -modified. The metadata will be uploaded to branches in the custom fork created before. - -`bloom-release` needs to be executed once per each desired `ROS_DISTRO` to generate metadata from. The name -`gazeboX_ros_pkgs` is just a stub to avoid use the official name: - - bloom-release --track ${ROS_DISTRO} --ros-distro ${ROS_DISTRO} gazeboX_ros_pkgs --edit - -Bloom will stop at some points asking for the user input: - - * `Release repository url [press enter to abort]:` the URL of the forked github repo needs to be set here - * `Repository Name:` `gazeboX_ros_pkgs` (being `X` the custom gazebo major version) - * The other options can be set to default just by pressing `enter` - -Bloom must be stopped after generating and pushing the new tags and branches. - -#### Example Releasing Gazebo8 for Kinetic - - $ bloom-release --track kinetic --ros-distro kinetic gazeboX_ros_pkgs --edit - - Specified repository 'gazeboX_ros_pkgs' is not in the distribution file located at 'https://raw.github.com/ros/rosdistro/master/kinetic/distribution.yaml' - Did you mean one of these: 'gazebo_ros_pkgs'? - Could not determine release repository url for repository 'gazeboX_ros_pkgs' of distro 'kinetic' - You can continue the release process by manually specifying the location of the RELEASE repository. - To be clear this is the url of the RELEASE repository not the upstream repository. - For release repositories on github, you should provide the `https://` url which should end in `.git`. - Release repository url [press enter to abort]: git@github.com:osrf/gazebo8_ros_pkgs-release.git - The release repository url you provided is not a `https://` address. - Would you like to enter the address again? - Continue [Y/n]? n - Very well, the address 'git@github.com:osrf/gazebo8_ros_pkgs-release.git' will be used as is. - ==> Fetching 'gazeboX_ros_pkgs' repository from 'git@github.com:osrf/gazebo8_ros_pkgs-release.git' - Cloning into '/tmp/tmpRGYh76'... - remote: Counting objects: 16388, done. - remote: Total 16388 (delta 0), reused 0 (delta 0) - Receiving objects: 100% (16388/16388), 4.02 MiB | 954.00 KiB/s, done. - Resolving deltas: 100% (4232/4232), done. - Checking connectivity... done. - Track 'kinetic' exists, editing... - Repository Name: - upstream - Default value, leave this as upstream if you are unsure - - Name of the repository (used in the archive name) - ['gazebo4_ros_pkgs']: - Upstream Repository URI: - - Any valid URI. This variable can be templated, for example an svn url - can be templated as such: "https://svn.foo.com/foo/tags/foo-:{version}" - where the :{version} token will be replaced with the version for this release. - ['https://github.com/ros-simulation/gazebo_ros_pkgs.git']: - Upstream VCS Type: - svn - Upstream URI is a svn repository - git - Upstream URI is a git repository - hg - Upstream URI is a hg repository - tar - Upstream URI is a tarball - ['git']: - Version: - :{ask} - This means that the user will be prompted for the version each release. - This also means that the upstream devel will be ignored. - :{auto} - This means the version will be guessed from the devel branch. - This means that the devel branch must be set, the devel branch must exist, - and there must be a valid package.xml in the upstream devel branch. - - This will be the version used. - It must be updated for each new upstream version. - [':{auto}']: - Release Tag: - :{none} - For svn and tar only you can set the release tag to :{none}, so that - it is ignored. For svn this means no revision number is used. - :{ask} - This means the user will be prompted for the release tag on each release. - :{version} - This means that the release tag will match the :{version} tag. - This can be further templated, for example: "foo-:{version}" or "v:{version}" - - This can describe any vcs reference. For git that means {tag, branch, hash}, - for hg that means {tag, branch, hash}, for svn that means a revision number. - For tar this value doubles as the sub directory (if the repository is - in foo/ of the tar ball, putting foo here will cause the contents of - foo/ to be imported to upstream instead of foo itself). - [':{version}']: - Upstream Devel Branch: - - Branch in upstream repository on which to search for the version. - This is used only when version is set to ':{auto}'. - ['kinetic-devel']: - ROS Distro: - - This can be any valid ROS distro, e.g. groovy, hydro - ['kinetic']: - Patches Directory: - :{none} - Use this if you want to disable overlaying of files. - - This can be any valid relative path in the bloom branch. The contents - of this folder will be overlaid onto the upstream branch after each - import-upstream. Additionally, any package.xml files found in the - overlay will have the :{version} string replaced with the current - version being released. - [None]: - Release Repository Push URL: - :{none} - This indicates that the default release url should be used. - - (optional) Used when pushing to remote release repositories. This is only - needed when the release uri which is in the rosdistro file is not writable. - This is useful, for example, when a releaser would like to use a ssh url - to push rather than a https:// url. - [None]: - Saving 'kinetic' track. - - ... (all bloom output while generating) ... - - <== Pushed tags successfully - ==> Generating pull request to distro file located at 'https://raw.github.com/ros/rosdistro/master/kinetic/distribution.yaml' - Would you like to add documentation information for this repository? [Y/n]? - - - -### Custom infrastructure to create .deb packages - -All the previous steps are designed to generate the appropriate Ubuntu metadata -inside the `forked -prerelease repository`. The metadata is being hosted in -git tags following the schema: - - - release/// - debian/ros--__ - -The `debian/...` tag contains the upstream code together with the `debian/` metadata repository ready to be -build using `debbuild` or any other debian generation tool. - -For reference, the OSRF buildfarm is using [this script](https://github.com/gazebo-tooling/release-tools/blob/master/jenkins-scripts/docker/lib/debbuild-bloom-base.bash) - diff --git a/bloom/ros_gazebo_pkgs/initial_info_for_new_release_repo.bash b/bloom/ros_gazebo_pkgs/initial_info_for_new_release_repo.bash deleted file mode 100755 index 9b8bcea69..000000000 --- a/bloom/ros_gazebo_pkgs/initial_info_for_new_release_repo.bash +++ /dev/null @@ -1,64 +0,0 @@ -#!/bin/bash -e -# -# This script will populate an empty gazeboX_rosY_pkgs-release repository with -# information from current official -release repositories -# - -if [[ ${#} -lt 2 ]]; then - echo "Usage: ${0} [ros1|ros2]" - exit -1 -fi - -GZ_MAJOR=${1} -ROS_VERSION=${2} - -SOURCE_REPO="https://github.com/${ROS_VERSION}-gbp/gazebo_ros_pkgs-release" -DEST_REPO="https://github.com/osrf/gazebo${GZ_MAJOR}_${ROS_VERSION}_pkgs-release" -TEST_DIR=$(mktemp -d) - -safety_check() -{ - RAW_URL="${DEST_REPO/github.com/raw.githubusercontent.com}/master/tracks.yaml" - - if wget -q "${RAW_URL}"; then - echo "!! Looks like the ${DEST_REPO} exists!" - echo "!! this tool is only for new repos" - exit -1 - fi -} - -clone_all_remote_branches() -{ - # source: https://coderwall.com/p/0ypmka - for branch in `git branch -a | grep remotes | grep -v HEAD | grep -v master `; do - git branch --track ${branch#remotes/origin/} $branch - done -} - -echo "Summary: " -echo " - The source repo to use would be: ${SOURCE_REPO}" -echo " - The destination repo would be : ${DEST_REPO}" - -pushd ${TEST_DIR} -safety_check - -git clone "${DEST_REPO}" dest_repo -git clone "${SOURCE_REPO}" source_repo -cp source_repo/* dest_repo/ -cd dest_repo -git add * -git commit -m "Import metadata from gazebo_ros_pkgs repository" -git remote rename origin gazebo${GZ_MAJOR} -git remote add origin ${SOURCE_REPO} -git fetch origin -clone_all_remote_branches -git remote remove origin -git remote rename gazebo${GZ_MAJOR} origin -# 7. Push all the local branch and tags to the new repo -git push origin --all -git push origin --tags - -echo " ** READY" -echo " remember to run: " -echo " - rename-gazebo-ros-pkgs.bash ${GZ_MAJOR} " -popd 2> /dev/null diff --git a/bloom/ros_gazebo_pkgs/rename-gazebo-ros-pkgs.bash b/bloom/ros_gazebo_pkgs/rename-gazebo-ros-pkgs.bash deleted file mode 100755 index 352ed3a45..000000000 --- a/bloom/ros_gazebo_pkgs/rename-gazebo-ros-pkgs.bash +++ /dev/null @@ -1,52 +0,0 @@ -#!/bin/bash -# -# This script should be run on a fork of gazebo_ros_pkgs-release repository -# and will modify control.em bloom templates for: -# -# - Rename the Package name to $(Package)X (X being major version) -# - Define a conflict on current official versiona dn 2 previous major versions $(Package) -# - -if [[ ${#} -lt 2 ]]; then - echo "Usage: ${0} " - exit -1 -fi - -MAJOR_VERSION=${1} -ROS_DISTROS=${*:2} - -for i in $(seq 1 2); do - conflict_ver=$((MAJOR_VERSION - i)) - CONFLICTS="${CONFLICTS}, @(Package.replace('gazebo-','gazebo${conflict_ver}-'))" -done - -PKGS="gazebo_dev gazebo_plugins gazebo_msgs gazebo_ros gazebo_ros_control gazebo_ros_pkgs" - -for pkg in ${PKGS}; do - for distro in ${ROS_DISTROS}; do - echo " - Processing $pkg in $distro" - if ! git checkout "debian/$distro/$pkg"; then - if [[ ${pkg} == "gazebo_ros_control" ]]; then - echo " [??] gazebo_ros_control not found in the repository" - echo " [??] assuming not avilable in this platform" - continue - fi - echo "The branch debian/$distro/$pkg was not found in the repo" - echo "Did you forget to run git fetch?" - exit 1 - fi - if grep 'Package.replace' debian/control.em; then - echo " + skip ${pkg} for ${distro}: seems to have changes in place" - continue - fi - # Modify package name - sed -i -e "s/Package: @(Package)/Package: @(Package.replace('gazebo-','gazebo${MAJOR_VERSION}-'))/" debian/control.em - sed -i -e "s/Source: @(Package)/Source: @(Package.replace('gazebo-','gazebo${MAJOR_VERSION}-'))/" debian/control.em - sed -i -e "s/@(Package)/@(Package.replace('gazebo-','gazebo${MAJOR_VERSION}-'))/" debian/changelog.em - git commit debian/control.em debian/changelog.em -m "Patch name to release ${MAJOR_VERSION} version" - # Include conflict with same package (not current) - sed -i -e "/^Depends/aConflicts: @(Package) ${CONFLICTS}" debian/control.em - git commit debian/control.em -m "Set up a conflict with official ROS packages and the two previous gazebo versions" - git push origin "debian/$distro/$pkg" - done -done diff --git a/bloom/ros_gazebo_pkgs/ros_gazebo_pkgs-release.py.bash b/bloom/ros_gazebo_pkgs/ros_gazebo_pkgs-release.py.bash deleted file mode 100755 index 721954fb1..000000000 --- a/bloom/ros_gazebo_pkgs/ros_gazebo_pkgs-release.py.bash +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/bash -# Launch all the suite of ros_gazebo_pkgs: -# Usage: ros_gazebo_pkgs -# - -if [[ $# -lt 2 ]]; then - echo "ros_gazebo_pkgs-release 'other arguments used in release.py'" - exit 1 -fi - -if [[ ${1%-*} != "${1}" ]]; then - echo "Version should not contain a revision number. Use -r argument" - exit 1 -fi - -for p in gazebo-dev gazebo-msgs gazebo-plugins gazebo-ros gazebo-ros-control gazebo-ros-pkgs; do - ../release-bloom.py ${p} $(for i in $@; do echo -n "$i "; done) -done diff --git a/conda/envs/legacy/pixi.lock b/conda/envs/legacy/pixi.lock index 852ac8c96..985701f36 100644 --- a/conda/envs/legacy/pixi.lock +++ b/conda/envs/legacy/pixi.lock @@ -509,6 +509,7 @@ environments: - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libvorbis-1.3.7-h01db608_0.tar.bz2 - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libvpx-1.13.1-h2f0025b_0.conda - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libwebp-base-1.5.0-h0886dbf_0.conda + - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libwebsockets-4.3.3-h8945da3_0.conda - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libxcb-1.15-h2a766a3_0.conda - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libxcrypt-4.4.36-h31becfc_1.conda - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libxkbcommon-1.7.0-h2555907_0.conda @@ -748,6 +749,7 @@ environments: - conda: https://conda.anaconda.org/conda-forge/win-64/libuv-1.50.0-h2466b09_0.conda - conda: https://conda.anaconda.org/conda-forge/win-64/libvorbis-1.3.7-h0e60522_0.tar.bz2 - conda: https://conda.anaconda.org/conda-forge/win-64/libwebp-base-1.5.0-h3b0e114_0.conda + - conda: https://conda.anaconda.org/conda-forge/win-64/libwebsockets-4.3.3-heb0366b_0.conda - conda: https://conda.anaconda.org/conda-forge/win-64/libxml2-2.12.7-h283a6d9_1.conda - conda: https://conda.anaconda.org/conda-forge/win-64/libzip-1.10.1-h1d365fa_3.conda - conda: https://conda.anaconda.org/conda-forge/win-64/libzlib-1.2.13-h2466b09_6.conda @@ -8860,6 +8862,23 @@ packages: license_family: BSD size: 429973 timestamp: 1734777489810 +- kind: conda + name: libwebsockets + version: 4.3.3 + build: h8945da3_0 + subdir: linux-aarch64 + url: https://conda.anaconda.org/conda-forge/linux-aarch64/libwebsockets-4.3.3-h8945da3_0.conda + sha256: 25a18da74fbc048307f5cb5cce5addeaa9caa3c4f94d3ef1ae11fad2279c6095 + md5: 2446142d49dadc928caa6e4efb7f2e58 + depends: + - libgcc-ng >=12 + - libstdcxx-ng >=12 + - libuv >=1.46.0,<2.0a0 + - openssl >=3.1.4,<4.0a0 + license: MIT + license_family: MIT + size: 395813 + timestamp: 1700469095651 - kind: conda name: libwebsockets version: 4.3.3 @@ -8878,6 +8897,24 @@ packages: license_family: MIT size: 385161 timestamp: 1700469113171 +- kind: conda + name: libwebsockets + version: 4.3.3 + build: heb0366b_0 + subdir: win-64 + url: https://conda.anaconda.org/conda-forge/win-64/libwebsockets-4.3.3-heb0366b_0.conda + sha256: cae70d40d705d69b3e11254681f4692001ef79e692ff61b21475cc861bf50746 + md5: 35a441c21943345aa0b5a247206dbaed + depends: + - libuv >=1.48.0,<2.0a0 + - openssl >=3.2.1,<4.0a0 + - ucrt >=10.0.20348.0 + - vc >=14.2,<15 + - vc14_runtime >=14.29.30139 + license: MIT + license_family: MIT + size: 354282 + timestamp: 1712216657465 - kind: conda name: libxcb version: '1.15' diff --git a/conda/envs/legacy/pixi.toml b/conda/envs/legacy/pixi.toml index 7bcf18572..62e3a8675 100644 --- a/conda/envs/legacy/pixi.toml +++ b/conda/envs/legacy/pixi.toml @@ -36,6 +36,7 @@ tinyxml2 = "*" yaml = "*" zeromq = "4.3.4" cli11 = "2.1.2.*" +libwebsockets = ">=4.3.3,<5" [target.win-64.dependencies] dlfcn-win32 = "*" diff --git a/conda/envs/legacy_ogre23/pixi.lock b/conda/envs/legacy_ogre23/pixi.lock index 0e537762b..c8eb13781 100644 --- a/conda/envs/legacy_ogre23/pixi.lock +++ b/conda/envs/legacy_ogre23/pixi.lock @@ -507,6 +507,7 @@ environments: - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libvorbis-1.3.7-h01db608_0.tar.bz2 - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libvpx-1.13.1-h2f0025b_0.conda - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libwebp-base-1.5.0-h0886dbf_0.conda + - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libwebsockets-4.3.3-h8945da3_0.conda - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libxcb-1.15-h2a766a3_0.conda - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libxcrypt-4.4.36-h31becfc_1.conda - conda: https://conda.anaconda.org/conda-forge/linux-aarch64/libxkbcommon-1.7.0-h2555907_0.conda @@ -745,6 +746,7 @@ environments: - conda: https://conda.anaconda.org/conda-forge/win-64/libuv-1.50.0-h2466b09_0.conda - conda: https://conda.anaconda.org/conda-forge/win-64/libvorbis-1.3.7-h0e60522_0.tar.bz2 - conda: https://conda.anaconda.org/conda-forge/win-64/libwebp-base-1.5.0-h3b0e114_0.conda + - conda: https://conda.anaconda.org/conda-forge/win-64/libwebsockets-4.3.3-heb0366b_0.conda - conda: https://conda.anaconda.org/conda-forge/win-64/libxml2-2.12.7-h283a6d9_1.conda - conda: https://conda.anaconda.org/conda-forge/win-64/libzip-1.10.1-h1d365fa_3.conda - conda: https://conda.anaconda.org/conda-forge/win-64/libzlib-1.2.13-h2466b09_6.conda @@ -8812,6 +8814,23 @@ packages: license_family: BSD size: 429973 timestamp: 1734777489810 +- kind: conda + name: libwebsockets + version: 4.3.3 + build: h8945da3_0 + subdir: linux-aarch64 + url: https://conda.anaconda.org/conda-forge/linux-aarch64/libwebsockets-4.3.3-h8945da3_0.conda + sha256: 25a18da74fbc048307f5cb5cce5addeaa9caa3c4f94d3ef1ae11fad2279c6095 + md5: 2446142d49dadc928caa6e4efb7f2e58 + depends: + - libgcc-ng >=12 + - libstdcxx-ng >=12 + - libuv >=1.46.0,<2.0a0 + - openssl >=3.1.4,<4.0a0 + license: MIT + license_family: MIT + size: 395813 + timestamp: 1700469095651 - kind: conda name: libwebsockets version: 4.3.3 @@ -8830,6 +8849,24 @@ packages: license_family: MIT size: 385161 timestamp: 1700469113171 +- kind: conda + name: libwebsockets + version: 4.3.3 + build: heb0366b_0 + subdir: win-64 + url: https://conda.anaconda.org/conda-forge/win-64/libwebsockets-4.3.3-heb0366b_0.conda + sha256: cae70d40d705d69b3e11254681f4692001ef79e692ff61b21475cc861bf50746 + md5: 35a441c21943345aa0b5a247206dbaed + depends: + - libuv >=1.48.0,<2.0a0 + - openssl >=3.2.1,<4.0a0 + - ucrt >=10.0.20348.0 + - vc >=14.2,<15 + - vc14_runtime >=14.29.30139 + license: MIT + license_family: MIT + size: 354282 + timestamp: 1712216657465 - kind: conda name: libxcb version: '1.15' diff --git a/conda/envs/legacy_ogre23/pixi.toml b/conda/envs/legacy_ogre23/pixi.toml index 8112e900a..7d247babd 100644 --- a/conda/envs/legacy_ogre23/pixi.toml +++ b/conda/envs/legacy_ogre23/pixi.toml @@ -35,6 +35,7 @@ ruby = "3.2.*" tinyxml2 = "*" yaml = "*" zeromq = "4.3.4" +libwebsockets = ">=4.3.3,<5" [target.win-64.dependencies] dlfcn-win32 = "*" diff --git a/jenkins-scripts/docker/gazebo_ros_pkgs-compilation.bash b/jenkins-scripts/docker/gazebo_ros_pkgs-compilation.bash deleted file mode 100644 index 03e1b0f29..000000000 --- a/jenkins-scripts/docker/gazebo_ros_pkgs-compilation.bash +++ /dev/null @@ -1,29 +0,0 @@ -#!/bin/bash -x - -# Knowing Script dir beware of symlink -[[ -L ${0} ]] && SCRIPT_DIR=$(readlink ${0}) || SCRIPT_DIR=${0} -SCRIPT_DIR="${SCRIPT_DIR%/*}" - -# Handle github-pullrequest jenkins plugin if present -if [[ -n ${ghprbTargetBranch} ]]; then - if [[ ${ghprbTargetBranch} != 'ros2' ]]; then - export ROS_DISTRO="${ghprbTargetBranch/-devel}" - fi -fi - -if [[ -z ${ARCH} ]]; then - echo "ARCH variable not set!" - exit 1 -fi - -if [[ -z ${DISTRO} ]]; then - echo "DISTRO variable not set!" - exit 1 -fi - -if [[ -z ${ROS_DISTRO} ]]; then - echo "ROS_DISTRO variable not set!" - exit 1 -fi - -. ${SCRIPT_DIR}/lib/gazebo_ros_pkgs-base.bash diff --git a/jenkins-scripts/docker/gazebo_ros_pkgs-compilation_regression.bash b/jenkins-scripts/docker/gazebo_ros_pkgs-compilation_regression.bash deleted file mode 100644 index 450556e18..000000000 --- a/jenkins-scripts/docker/gazebo_ros_pkgs-compilation_regression.bash +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/bash -x - -# Knowing Script dir beware of symlink -[[ -L ${0} ]] && SCRIPT_DIR=$(readlink ${0}) || SCRIPT_DIR=${0} -SCRIPT_DIR="${SCRIPT_DIR%/*}" - -# Handle github-pullrequest jenkins plugin if present -if [[ -n ${ghprbTargetBranch} ]]; then - export ROS_DISTRO="${ghprbTargetBranch/-devel}" -fi - -if [[ -z ${ARCH} ]]; then - echo "ARCH variable not set!" - exit 1 -fi - -if [[ -z ${DISTRO} ]]; then - echo "DISTRO variable not set!" - exit 1 -fi - -if [[ -z ${ROS_DISTRO} ]]; then - echo "ROS_DISTRO variable not set!" - exit 1 -fi - -ROS_WS_PREBUILD_HOOK=""" -git clone https://github.com/tu-darmstadt-ros-pkg/hector_gazebo -b ${ROS_DISTRO}-devel -""" -. ${SCRIPT_DIR}/lib/gazebo_ros_pkgs-base.bash diff --git a/jenkins-scripts/docker/gazebo_ros_pkgs-release-testing.bash b/jenkins-scripts/docker/gazebo_ros_pkgs-release-testing.bash deleted file mode 100644 index 50177ac39..000000000 --- a/jenkins-scripts/docker/gazebo_ros_pkgs-release-testing.bash +++ /dev/null @@ -1,8 +0,0 @@ -#!/bin/bash -x - -# Knowing Script dir beware of symlink -[[ -L ${0} ]] && SCRIPT_DIR=$(readlink ${0}) || SCRIPT_DIR=${0} -SCRIPT_DIR="${SCRIPT_DIR%/*}" -export GPU_SUPPORT_NEEDED=true - -. ${SCRIPT_DIR}/lib/gazebo_ros_pkgs-check-release.bash diff --git a/jenkins-scripts/docker/lib/_sdformat_version_hook.bash b/jenkins-scripts/docker/lib/_sdformat_version_hook.bash new file mode 100644 index 000000000..40c061ef3 --- /dev/null +++ b/jenkins-scripts/docker/lib/_sdformat_version_hook.bash @@ -0,0 +1,10 @@ +# Identify SDFORMAT_MAJOR_VERSION to help with dependency resolution +SDFORMAT_MAJOR_VERSION=$(\ + python3 ${SCRIPT_DIR}/../tools/detect_cmake_major_version.py \ + ${WORKSPACE}/sdformat/CMakeLists.txt) + +# Check SDFORMAT version is integer +if ! [[ ${SDFORMAT_MAJOR_VERSION} =~ ^-?[0-9]+$ ]]; then + echo "Error! SDFORMAT_MAJOR_VERSION is not an integer, check the detection" + exit -1 +fi diff --git a/jenkins-scripts/docker/lib/debbuild-bloom-base.bash b/jenkins-scripts/docker/lib/debbuild-bloom-base.bash index 3bd74ca0c..11b4296d3 100644 --- a/jenkins-scripts/docker/lib/debbuild-bloom-base.bash +++ b/jenkins-scripts/docker/lib/debbuild-bloom-base.bash @@ -96,13 +96,6 @@ PKGS=\`find .. -name '*.deb' || true\` FOUND_PKG=0 for pkg in \${PKGS}; do echo "found \$pkg" - # Check for problems in gazebo_ros_pkgs unofficial wrappers to avoid - # uploads with the same name than official ROS packages - # ros-DISTRO-gazebo-* instead of ros-DISTRO-gazeboX-* - if [[ \${pkg} != \${pkg/-gazebo-} ]]; then - echo "Detected official ROS names in gazebo_ros_pkgs" - exit -1 - fi # Check for correctly generated packages size > 3Kb [[ \$(find \$pkg -size +3k) ]] || echo "WARNING: empty package?" cp \${pkg} $WORKSPACE/pkgs diff --git a/jenkins-scripts/docker/sanity_checks.bash b/jenkins-scripts/docker/sanity_checks.bash new file mode 100755 index 000000000..891e314ce --- /dev/null +++ b/jenkins-scripts/docker/sanity_checks.bash @@ -0,0 +1,22 @@ +#!/bin/bash -e + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# +# Check for existing scripts. Lines: +# 1. Find all .bash files excluding *_checks.bash +# 2. Search for occurrences of ${SCRIPT_DIR}/ in the files +# 3. Exclude lines containing '//' (comments) +# 4. Extract the paths containing ${SCRIPT_DIR}/ +# 5. Remove spurious " +# 6. Sort and remove duplicates to get clean output +for f in $(find -type f -name '*.bash' ! -path '*_checks.bash' -exec grep -Eh '\${SCRIPT_DIR}/' {} \; | grep -v '//' | \ + grep -Eh -o '\${SCRIPT_DIR}/.*' | awk '{print $1}' | \ + sed 's/"//g' | \ + sort | uniq); do + f=$(eval echo $f) + if ! test -f ${f}; then + echo "${f} script not found in the repository" + exit 1 + fi +done diff --git a/jenkins-scripts/docker/sdformat-compilation.bash b/jenkins-scripts/docker/sdformat-compilation.bash new file mode 100644 index 000000000..6f7b91895 --- /dev/null +++ b/jenkins-scripts/docker/sdformat-compilation.bash @@ -0,0 +1,30 @@ +#!/bin/bash -x + +# Knowing Script dir beware of symlink +[[ -L ${0} ]] && SCRIPT_DIR=$(readlink ${0}) || SCRIPT_DIR=${0} +SCRIPT_DIR="${SCRIPT_DIR%/*}" + +if [[ -z ${ARCH} ]]; then + echo "ARCH variable not set!" + exit 1 +fi + +if [[ -z ${DISTRO} ]]; then + echo "DISTRO variable not set!" + exit 1 +fi + +. "${SCRIPT_DIR}/lib/_sdformat_version_hook.bash" + +export BUILDING_SOFTWARE_DIRECTORY="${BUILDING_SOFTWARE_DIRECTORY:-sdformat}" + +if [[ ${SDFORMAT_MAJOR_VERSION} -ge 8 ]]; then + export NEED_C17_COMPILER=true +fi + +export GZDEV_PROJECT_NAME="sdformat${SDFORMAT_MAJOR_VERSION}" + +# master and major branches compilations +export BUILDING_PKG_DEPENDENCIES_VAR_NAME="SDFORMAT_BASE_DEPENDENCIES" + +. "${SCRIPT_DIR}/lib/generic-building-base.bash" diff --git a/jenkins-scripts/lib/colcon-default-devel-windows.bat b/jenkins-scripts/lib/colcon-default-devel-windows.bat index aeaba2d3f..b0d425132 100644 --- a/jenkins-scripts/lib/colcon-default-devel-windows.bat +++ b/jenkins-scripts/lib/colcon-default-devel-windows.bat @@ -20,8 +20,9 @@ set win_lib=%SCRIPT_DIR%\lib\windows_library.bat set EXPORT_TEST_RESULT_PATH=%WORKSPACE%\build\test_results set LOCAL_WS=%WORKSPACE%\ws -set LOCAL_WS_SOFTWARE_DIR=%LOCAL_WS%\%VCS_DIRECTORY% -set LOCAL_WS_BUILD=%WORKSPACE%\build +set LOCAL_WS_BUILD=%LOCAL_WS%\build +set LOCAL_WS_SRC=%LOCAL_WS%\src +set LOCAL_WS_SOFTWARE_DIR=%LOCAL_WS_SRC%\%VCS_DIRECTORY% :: default values @if "%BUILD_TYPE%" == "" set BUILD_TYPE=Release @@ -133,28 +134,28 @@ if defined USE_PIXI ( echo # BEGIN SECTION: setup workspace if not defined KEEP_WORKSPACE ( - IF exist %LOCAL_WS_BUILD% ( + IF exist %LOCAL_WS% ( echo # BEGIN SECTION: preclean workspace - rmdir /s /q %LOCAL_WS_BUILD% || goto :error + rmdir /s /q %LOCAL_WS% || goto :error echo # END SECTION ) ) -mkdir %LOCAL_WS% || echo "Workspace already exists!" -cd %LOCAL_WS% +mkdir %LOCAL_WS% +mkdir %LOCAL_WS_SRC% echo # END SECTION echo # BEGIN SECTION: get open robotics deps (%GAZEBODISTRO_FILE%) sources into the workspace -if exist %LOCAL_WS_SOFTWARE_DIR% ( rmdir /q /s %LOCAL_WS_SOFTWARE_DIR% ) -call %win_lib% :get_source_from_gazebodistro %GAZEBODISTRO_FILE% %LOCAL_WS% || goto :error +call %win_lib% :get_source_from_gazebodistro %GAZEBODISTRO_FILE% %LOCAL_WS_SRC% || goto :error echo # END SECTION :: this step is important since overwrite the gazebodistro file echo # BEGIN SECTION: move %VCS_DIRECTORY% source to workspace -if exist %LOCAL_WS_SOFTWARE_DIR% ( rmdir /q /s %LOCAL_WS_SOFTWARE_DIR% ) +if exist %LOCAL_WS_SOFTWARE_DIR% ( rmdir /q /s %LOCAL_WS_SOFTWARE_DIR% ) xcopy %WORKSPACE%\%VCS_DIRECTORY% %LOCAL_WS_SOFTWARE_DIR% /s /e /i > xcopy_vcs_directory.log || goto :error echo # END SECTION echo # BEGIN SECTION: packages in workspace +cd %LOCAL_WS% call %win_lib% :list_workspace_pkgs || goto :error echo # END SECTION @@ -163,13 +164,13 @@ echo # BEGIN SECTION: Update package !COLCON_PACKAGE! from gz to ignition echo Packages in workspace: colcon list --names-only -colcon list --names-only | find "!COLCON_PACKAGE!" +colcon list --names-only | find "!COLCON_PACKAGE!" > nul 2>&1 if errorlevel 1 ( :: REQUIRED for Gazebo Fortress set COLCON_PACKAGE=!COLCON_PACKAGE:gz=ignition! set COLCON_PACKAGE=!COLCON_PACKAGE:sim=gazebo! ) -colcon list --names-only | find "!COLCON_PACKAGE!" +colcon list --names-only | find "!COLCON_PACKAGE!" > nul 2>&1 if errorlevel 1 ( echo Failed to find package !COLCON_PACKAGE! in workspace. goto :error @@ -183,7 +184,7 @@ call %win_lib% :build_workspace !COLCON_PACKAGE! !COLCON_PACKAGE_EXTRA_CMAKE_ARG echo # END SECTION if "%ENABLE_TESTS%" == "TRUE" ( - set TEST_RESULT_PATH=%WORKSPACE%\ws\build\!COLCON_PACKAGE!\test_results + set TEST_RESULT_PATH=%LOCAL_WS_BUILD%\!COLCON_PACKAGE!\test_results echo # BEGIN SECTION: running tests for !COLCON_PACKAGE! call %win_lib% :tests_in_workspace !COLCON_PACKAGE! diff --git a/jenkins-scripts/sdformat-default-devel-windows-amd64.bat b/jenkins-scripts/sdformat-default-devel-windows-amd64.bat new file mode 100644 index 000000000..22587f442 --- /dev/null +++ b/jenkins-scripts/sdformat-default-devel-windows-amd64.bat @@ -0,0 +1,22 @@ +@echo on + +set SCRIPT_DIR=%~dp0 + +if not defined VCS_DIRECTORY set VCS_DIRECTORY=sdformat +set PLATFORM_TO_BUILD=x86_amd64 +set IGN_CLEAN_WORKSPACE=true + +set DEPEN_PKGS="libxml2 tinyxml2" + +for /f %%i in ('python "%SCRIPT_DIR%\tools\detect_cmake_major_version.py" "%WORKSPACE%\%VCS_DIRECTORY%\CMakeLists.txt"') do set SDFORMAT_MAJOR_VERSION=%%i + +if %SDFORMAT_MAJOR_VERSION% GEQ 10 ( + set COLCON_PACKAGE=sdformat + set COLCON_AUTO_MAJOR_VERSION=true + call "%SCRIPT_DIR%\lib\colcon-default-devel-windows.bat" +) else if %SDFORMAT_MAJOR_VERSION% GEQ 6 ( + call "%SCRIPT_DIR%/lib/generic-default-devel-windows.bat" +) else ( + echo "Ancient version of sdformat is not supported %SDFORMAT_MAJOR_VERSION%" + exit 1 +)