Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/collect versions #61

Merged
merged 10 commits into from
Jan 23, 2020
Merged

Feature/collect versions #61

merged 10 commits into from
Jan 23, 2020

Conversation

rsuchecki
Copy link
Collaborator

GitHub actions set-up closes #43
although NF caching does not seem to work in actions with singularity (minor)

Collecting tool versions versions from within containers #45 #18

@rsuchecki
introducing version parsing from mappers
d6d248a
@rsuchecki
version calls added for biokanga and hisat2 conf
3649859
@rsuchecki
overwrite declared versions with parsed, warn or abort?
4470fe2
@rsuchecki
added version calls for several mappers
a3e0dbb
@rsuchecki
added filter exluding mappers lacking versionCall definition
bb252f6
@rsuchecki
Implemented GitHub actions workflow run tests
ad72a3d 
  * Docker
  * Singularity
  * Actions caching of
    * Test data
    * Singularity images
    * NF work/ and ./.nextflow/cache
  * Per mapper/mapmode test runs (docker)
  * Multiple mapper/mapmode test run (singularity)
  * Leaner container images
  * Updated exec profiles

Details:

commit 2c5a806
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 23 11:06:08 2020 +1030

    simg size

commit 66cb0dd
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 23 11:03:01 2020 +1030

    keep NF/work caches separate per mapmode 4 independent parallel exec

commit 01fc789
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 23 11:00:27 2020 +1030

    cleanup

commit 283afe2
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 23 10:52:14 2020 +1030

    missing quotes around mappers regex

commit 389da55
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 23 10:40:12 2020 +1030

    singularity test all-in one

commit 2b58954
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 23 10:30:34 2020 +1030

    -redundancy

commit 4d3cf8f
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 23 10:21:06 2020 +1030

    re-worked singularity caching

commit a13b306
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 23 10:09:24 2020 +1030

    seaprated data cache from work cache which callows to keep docker/singularity work caches separate

commit c65c389
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 17:07:57 2020 +1030

    cache-in

commit b012dcf
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 16:51:42 2020 +1030

    cache-er

commit a4bb2ca
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 16:45:39 2020 +1030

    cache-ish

commit df39af7
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 16:44:07 2020 +1030

    simplified restore

commit a2346e5
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 16:35:00 2020 +1030

    cache-keys

commit 393ab7e
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 16:15:52 2020 +1030

    cache key changes

commit a6c3282
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 16:03:08 2020 +1030

    cache conditional2

commit 07694f5
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 16:02:22 2020 +1030

    cache conditional2

commit b9cfb45
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 16:00:18 2020 +1030

    cache conditional2

commit 811533a
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 15:47:37 2020 +1030

    cache conditional1

commit d643974
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 15:40:29 2020 +1030

    cache conditional

commit e9f8b50
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 15:26:55 2020 +1030

    cache conditional

commit ffe1520
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 15:14:38 2020 +1030

    cache conditional

commit cc77479
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 15:09:57 2020 +1030

    cache conditional

commit 258fbcf
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 15:08:43 2020 +1030

    cache conditional

commit aa93c37
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 15:07:45 2020 +1030

    cache conditional

commit f17f311
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 15:07:27 2020 +1030

    cache conditional

commit 26542e6
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:53:59 2020 +1030

    cache conditional

commit be93a9d
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:50:01 2020 +1030

    cache conditional

commit feceeba
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:40:42 2020 +1030

    cache conditional

commit ad77c77
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:31:02 2020 +1030

    data

commit 0ad06bb
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:29:02 2020 +1030

    data

commit fbf2c86
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:28:19 2020 +1030

    data

commit f714d73
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:27:43 2020 +1030

    data

commit e8e31e7
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:18:31 2020 +1030

    cache

commit d7643ba
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:16:44 2020 +1030

    cache

commit ad65601
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:13:30 2020 +1030

    cache

commit 9f5f114
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:11:44 2020 +1030

    cache

commit 390cf71
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:09:52 2020 +1030

    cache

commit 080a09b
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:07:42 2020 +1030

    cache

commit 3964336
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:05:58 2020 +1030

    cache

commit 6484441
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 14:04:00 2020 +1030

    cache

commit 66ed6da
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 13:51:06 2020 +1030

    indent?

commit 1813403
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 13:49:04 2020 +1030

    Using NF install action

commit 533a18c
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 12:55:05 2020 +1030

    typos?

commit 3f153b9
Merge: 02c9b3c ee8a048
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Wed Jan 22 12:53:46 2020 +1030

    Merge branch 'feature/ci' of github.com:csiro-crop-informatics/repset into feature/ci

commit 02c9b3c
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Tue Jan 21 16:36:36 2020 +1030

    new matrix

commit ee8a048
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Tue Jan 21 16:36:36 2020 +1030

    artefact sharing4

commit 06ae782
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Tue Jan 21 16:34:47 2020 +1030

    artefact sharing3

commit 82b038b
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Tue Jan 21 16:31:15 2020 +1030

    artefact sharing2

commit 9903211
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Tue Jan 21 16:29:53 2020 +1030

    artefact sharing

commit a715b97
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Tue Jan 21 16:07:07 2020 +1030

    CI refinements cont

commit 3cce358
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Tue Jan 21 15:47:18 2020 +1030

    CI refinements

commit 1ac1f03
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Mon Jan 20 13:48:12 2020 +1030

    try report rendering @ CI

commit 580f828
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Mon Jan 20 13:29:40 2020 +1030

    Updated report rendering isntructions outside the pipeline

commit d5f3c92
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 16:50:05 2020 +1030

    added graph-easy rendering

commit 8368e62
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 16:39:49 2020 +1030

    typo

commit aeefa69
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 16:34:20 2020 +1030

    typo

commit d637ead
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 16:32:31 2020 +1030

    singularity containered pulls taylored to CI

commit f67dfab
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 16:07:07 2020 +1030

    not pulling s images, more sysinfo

commit 69c5241
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:43:27 2020 +1030

    typo

commit 4fa616f
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:32:32 2020 +1030

    temp: skipping report rendering in CI

commit 5a18092
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:31:55 2020 +1030

    added sysinfo

commit adc6223
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:08:16 2020 +1030

    updated picocli version

commit 0d472c6
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:07:49 2020 +1030

    added minimal eukaryotic dataset

commit 2904e6a
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:04:50 2020 +1030

    catch all "groovy" containing labels

commit 4058604
Merge: 90caaca d933b7c
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:03:39 2020 +1030

    Merge branch 'docker/samtools/1.10' into docker/groovy-samtools/3.0jre-1.9-alpine

commit 90caaca
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:03:20 2020 +1030

    fixed label

commit d933b7c
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 15:02:20 2020 +1030

    container spec

commit 9870ff5
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 14:45:08 2020 +1030

    biocontaiers mirror

commit 879f6ce
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 13:21:10 2020 +1030

    dropped image sha - could that be triggering internal server error?

commit 58691d1
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Fri Jan 17 12:58:05 2020 +1030

    Samtools and bash added to alpine groovy 3.0 jre

commit ce8211d
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 16 15:57:50 2020 +1030

    trigger build from quay

commit d8fc877
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 16 15:54:00 2020 +1030

    fix compatibility issues due to busybox sed on biocontainers rnftools image

commit 92a59bc
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 16 15:53:08 2020 +1030

    downgrade for leaner container image

commit 6785c05
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 16 15:52:42 2020 +1030

    tested quay rnftools containers

commit 9277418
Author: Rad Suchecki <r.suchecki@gmail.com>
Date:   Thu Jan 16 11:27:36 2020 +1030

    singularity and docker conf for GH actions CI
@rsuchecki
bug fix renderReport caching, typos, formatting
b5a658f
@rsuchecki
-redundant
ba4cbef
@rsuchecki
streamlined results and meta JSON handling
b3ff9c5
@rsuchecki
supress DAG warning
c54ccfe
@rsuchecki rsuchecki merged commit 25aa350 into master Jan 23, 2020
@rsuchecki rsuchecki deleted the feature/collectVersions branch July 22, 2020 05:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add CI
1 participant
@rsuchecki