DCOR uses virtualbox images created with Vagrant for testing. This repository contains the recipe for creating the dcor-test vagrant boxes. The default test user is "peterpan:peterpan".
There is a docker recipe for CKAN and I could probably also have adapted that to DCOR, but that looked like a lot of code to maintain. The upside of virtualbox images is that you can take your time to set it up (which is very convenient if you use vagrant) and then just distribute the final box image for testing.
By default, the Vagrantfile gives access to the DCOR web interface (http://127.0.0.1:8888)
as well as the SOLR search interface (http://127.0.0.1:8983)
Let's use the ckanext-dcor_schemas repository as an example.
If you clone into it, you will have the files
Vagrantfile
(the recipe for setting up the box) and
vagrant-run-tests.sh
(that installs the current working tree into the CKAN environment and executes the tests).
When you run vagrant up in this directory, vagrant uses Vagrantfile to setup the virtualbox
image (download and rsync the current working directory of the host to /testing on the guest).
To run the tests, simply run:
vagrant ssh -- sudo bash /testing/vagrant-run-tests.sh
This is only necessary if you want to create an image from scratch (from an existing ubuntu box). If you only have to do something incrementally, you can just base your box on the existing dcor-test images.
Install prerequisites:
apt install virtualbox vagrant
Download and setup the base image:
cd from_scratch vagrant up
Login to the virual machine:
vagrant ssh
At this point, install CKAN/DCOR according to the description at https://dc.readthedocs.io/en/latest/https://dc.readthedocs.io/en/latest/sec_self_hosting/index.html. The default admin user and password for MinIO is minio-vagrant.
Also, make sure that the following packages are installed in the ckan environment:
pip install --upgrade codecov coverage pytest-ckan # required by ckan test suite: pip install --upgrade pytest_factoryboy pip install --upgrade pytest-rerunfailures
When job is finished then create the box:
# cleanup apt apt autoremove apt clean # cleanup pip source /usr/lib/ckan/default/bin/activate pip cache purge # At this point it makes sense to restart. # zero-out swap swapoff /dev/sda2 dd if=/dev/zero of=/dev/sda2 bs=1M mkswap /dev/sda2 # and edit `/etc/fstab` as well as `/etc/initramfs-tools/conf.d/resume` with new UUID # zero-out the empty space on the boot and root partitions mount -o remount,ro /dev/sda1 zerofree -v /dev/sda1 # This requires you to stop all services via systemctl systemctl stop minio nginx postgresql rsyslog solr supervisor systemctl stop syslog.socket rsyslog.service systemd-journald # Try `lsof / | awk '$4 ~ /[0-9].*w/'` if this does not work: systemctl stop systemd-journald systemd-journald.socket && mount -o remount,ro /dev/sda3 zerofree -v /dev/sda3 # logout and create the package vagrant halt vagrant package --output dcor-test_0.6.0.box
If apt/CKAN/DCOR packages only require updates, then it makes sense to base a new version of an image on an existing image:
cd from_previous vagrant destroy -f vagrant up vagrant ssh
Once finished with the DCOR update, follow the box creation steps above.