Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linter CI part 1 #52

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
117b02a
Linter CI part 1
yaswant Jan 13, 2025
0804359
fix codeowners
yaswant Jan 13, 2025
9d358e3
fix indent
yaswant Jan 13, 2025
4745280
fix indent again
yaswant Jan 13, 2025
0d6cc66
block some unwanted checks
yaswant Jan 13, 2025
55a0e93
falsify fail-fast strategy
yaswant Jan 13, 2025
3187c07
fix Perl lexical warnings
yaswant Jan 13, 2025
c35d2fa
fix Perl warning
yaswant Jan 13, 2025
8a2b172
update README
yaswant Jan 13, 2025
eafaa31
add percriticrc
yaswant Jan 13, 2025
5c492a3
add codeql badge
yaswant Jan 13, 2025
f31ba59
add codeql badge
yaswant Jan 13, 2025
74ec869
cancel in progress runs with PR updates
yaswant Jan 13, 2025
99ff5f5
Update .github/linters/.ruff.toml
yaswant Jan 24, 2025
d5198c9
Update .github/linters/.ruff.toml
yaswant Jan 24, 2025
ad64634
Update .github/linters/.ruff.toml
yaswant Jan 24, 2025
1fd8cc3
Update .github/linters/.ruff.toml
yaswant Jan 24, 2025
9fc38b6
turn off python ruff
james-bruten-mo Jan 24, 2025
e0c4a3a
Merge remote-tracking branch 'origin/main' into feature/ci_linter
james-bruten-mo Jan 24, 2025
5373809
apply black/isort, populate black/isort conf files
james-bruten-mo Jan 24, 2025
14fd008
run isort
james-bruten-mo Jan 24, 2025
f79a711
missed file
james-bruten-mo Jan 24, 2025
a528fda
update black config
james-bruten-mo Jan 24, 2025
4cd0dcc
partial fix shellcheck issues
yaswant Jan 24, 2025
cdee2a9
tidy
yaswant Jan 25, 2025
1978237
black check config try
yaswant Jan 25, 2025
3772d24
opaque black
yaswant Jan 25, 2025
2648d8a
black default
yaswant Jan 25, 2025
3882fb3
refactor run kgo script with POSIX compliance
yaswant Jan 27, 2025
3ddd0d1
response to @james-bruten-mo comments
yaswant Jan 27, 2025
db24407
revert changes to meto_run_kgi_script
yaswant Jan 30, 2025
93d04f1
revert printf changes and add shellcheck ignore
james-bruten-mo Jan 31, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Default owners of the repository
* @metoffice/ssdteam

# Add component owners as appropriate
5 changes: 5 additions & 0 deletions .github/linters/.perlcriticrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Report slightly less severe violations (severity >= 4)
severity = 4
# Verbosity format
# filename:line:column [severity policy] message near 'string of source code that caused the violation'
verbose = %f:%l:%c [%s %p] %m near '%r'\n
Empty file added .github/linters/.python-black
Empty file.
20 changes: 20 additions & 0 deletions .github/linters/.python-lint
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[MASTER]
# Use multiple processes to speed up Pylint.
jobs=2

[MESSAGES CONTROL]
# C0114: Missing module docstring (missing-module-docstring)
# C0116: Missing function or method docstring (missing-function-docstring)
# C0103: Constant name doesn't conform to UPPER_CASE naming style (invalid-name)
# C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
# E0401: Unable to import module (import-error)
# W0611: Unused import sphinx_rtd_theme (unused-import)
# W0622: Redefining built-in 'copyright' (redefined-builtin)
# R0801: Similar lines in 2 files
disable=C0114,C0116,C0103,C0209,E0401,W0611,W0622,R0801,

[SIMILARITIES]
# Minimum lines number of a similarity.
min-similarity-lines=4
ignore-comments=yes
ignore-docstrings=yes
20 changes: 20 additions & 0 deletions .github/linters/.ruff.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
cache-dir = "/tmp/.ruff_cache"
line-length = 80

[lint]
# C0116: Missing function or method docstring (missing-function-docstring)
# C0103: Constant name doesn't conform to UPPER_CASE naming style (invalid-name)
# C0209: Formatting a regular string which could be an f-string (consider-using-f-string)
# W0622: Redefining built-in 'copyright' (redefined-builtin)
# R0801: Similar lines in 2 files
ignore = ["C0116","C0103","C0209","W0622","R0801",]
yaswant marked this conversation as resolved.
Show resolved Hide resolved

# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

[format]
# Like Black, indent with spaces, rather than tabs.
indent-style = "space"
yaswant marked this conversation as resolved.
Show resolved Hide resolved

# Like Black, respect magic trailing commas.
skip-magic-trailing-comma = false
1 change: 1 addition & 0 deletions .github/linters/.shellcheckrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
source-path=SCRIPTDIR
32 changes: 32 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Description

## Summary

_Briefly describe the feature being introduced._

## Changes

_List the major changes made in this pull request._

## Dependency

_List dependent changes. Can use build-group logic here._

## Impact

_Discuss any potential impacts this feature may have on existing functionalities._

## Issues addressed

Resolves

_List issue(s) related to this PR._

## Coordinated merge

_Specify any coordinated merges here._


## Checklist

- [ ] I have performed a self-review of my own changes
57 changes: 57 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
# Lint/analyse code with Super-Linter.
name: Checks

on:
pull_request:
branches: [main]
push:
branches: [main, 'releases/**']

concurrency:
group: ${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

permissions: read-all

jobs:
check:
name: Lint
runs-on: ubuntu-latest
strategy:
max-parallel: 4
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]

permissions:
contents: read
packages: read
statuses: write

steps:
- name: Checkout current
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Super-Linter
uses: super-linter/super-linter/slim@v7.2.1
env:
FILTER_REGEX_EXCLUDE: (.*[.]conf.py|pull_request_template.md|/linters/)
IGNORE_GITIGNORED_FILES: true
VALIDATE_BASH_EXEC: false
VALIDATE_JAVASCRIPT_STANDARD: false
VALIDATE_JSCPD: false
VALIDATE_MARKDOWN_PRETTIER: false
VALIDATE_PYTHON_FLAKE8: false
VALIDATE_PYTHON_ISORT: false
VALIDATE_PYTHON_MYPY: false
VALIDATE_PYTHON_PYINK: false
VALIDATE_PYTHON_PYLINT: false
VALIDATE_SHELL_SHFMT: false
VALIDATE_YAML_PRETTIER: false
VALIDATE_YAML: false
# To report GitHub Actions status checks
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
...
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
*.sublime-workspace
*.code-workspace
.vscode
__pycache__/

*.pyc
*~
14 changes: 9 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# SimSys Scripts
# Simulation Systems Scripts

This repository contains support scripts that are common across the many simulation and modelling codes owned by the Met Office.
Particularily those owned and maintained by the SSD team.
[![Checks](https://github.com/MetOffice/SimSys_Scripts/actions/workflows/lint.yml/badge.svg)](https://github.com/MetOffice/SimSys_Scripts/actions/workflows/lint.yml)
[![CodeQL](https://github.com/MetOffice/SimSys_Scripts/actions/workflows/github-code-scanning/codeql/badge.svg)](https://github.com/MetOffice/SimSys_Scripts/actions/workflows/github-code-scanning/codeql)

Also contains a copy of script_updater.sh which is intended to live in the fcm repositories to pull from
this repo.
This repository contains support scripts that are common across the many
simulation and modelling codes owned by the Met Office. Particularly those
owned and maintained by the Simulation Systems and Deployment (SSD) team.

Also contains a copy of `script_updater.sh` which is intended to live in the
fcm repositories to pull from this repository.
10 changes: 5 additions & 5 deletions script_umdp3_checker/bin/umdp3_check.pl
Original file line number Diff line number Diff line change
Expand Up @@ -1010,13 +1010,13 @@ sub run_checks {
}

# read in data from file to $data, then
my $io_array = new IO::ScalarArray \@file_lines;
my $io_array = IO::ScalarArray->new(\@file_lines);
my $mimetype = mimetype($io_array);

# if we can't detect a mime type, try some tricks to aid detection
if ( $mimetype =~ /text\/plain/sxm ) {
my @mime_file_lines = grep !/^\s*\#/sxm, @file_lines;
$io_array = new IO::ScalarArray \@mime_file_lines;
$io_array = IO::ScalarArray->new(\@mime_file_lines);
$mimetype = mimetype($io_array);
}

Expand Down Expand Up @@ -1133,12 +1133,12 @@ sub run_checks {
$filename .= ".";
}
$filename = $log_cylc . "." . $filename . "report";
my $fileres = open( FH, '>', $filename );
my $fileres = open( my $fh, '>', $filename );
if ( !defined $fileres ) {
die "ERR: $filename\n";
}
print FH $failure_text;
close(FH);
print $fh $failure_text;
close($fh);
}
}
$message = '';
Expand Down
Loading