Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs workflolw #998

Merged
merged 47 commits into from
Nov 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
fead1f4
test
stavros-k Nov 22, 2024
8ed913d
clen
stavros-k Nov 22, 2024
f5ef339
test more complex
stavros-k Nov 22, 2024
6747c13
fix
stavros-k Nov 22, 2024
71904a5
one line?
stavros-k Nov 22, 2024
e80e8ff
test
stavros-k Nov 22, 2024
aa67c2b
br
stavros-k Nov 22, 2024
897e3eb
list
stavros-k Nov 22, 2024
95f4e02
test more
stavros-k Nov 22, 2024
cde2dfc
omre?
stavros-k Nov 22, 2024
f7f795d
cleaner
stavros-k Nov 22, 2024
4c90157
hmm
stavros-k Nov 22, 2024
2b6d739
back
stavros-k Nov 22, 2024
0e97538
hmm
stavros-k Nov 22, 2024
70b524b
no json
stavros-k Nov 22, 2024
55a58b7
fmt
stavros-k Nov 22, 2024
61c531a
speratate
stavros-k Nov 22, 2024
b5d44b2
cleaner
stavros-k Nov 22, 2024
a4f2005
test
stavros-k Nov 22, 2024
9fdb682
fix
stavros-k Nov 22, 2024
38c6335
aha
stavros-k Nov 22, 2024
1ae4fb0
remove
stavros-k Nov 22, 2024
32ef0fa
ok
stavros-k Nov 22, 2024
bb6acd9
okay
stavros-k Nov 22, 2024
544728a
test
stavros-k Nov 22, 2024
f47fe20
reduce noise
stavros-k Nov 22, 2024
d45f181
reduce nosie
stavros-k Nov 22, 2024
b12fd5c
separate
stavros-k Nov 22, 2024
3820e59
fmt
stavros-k Nov 22, 2024
e665d21
cleaner
stavros-k Nov 22, 2024
76e0e3d
new lines
stavros-k Nov 22, 2024
bd5c2b1
slim down
stavros-k Nov 22, 2024
ceaba15
sort
stavros-k Nov 22, 2024
a1f393b
remove import
stavros-k Nov 22, 2024
06548f8
fmt
stavros-k Nov 22, 2024
c44b2e0
app
stavros-k Nov 22, 2024
1defc72
tag docs-team
stavros-k Nov 22, 2024
4379ce2
quote
stavros-k Nov 22, 2024
275a26e
revert app changes
stavros-k Nov 22, 2024
5b2e7b5
skip if empty
stavros-k Nov 22, 2024
697aeda
skip better
stavros-k Nov 22, 2024
bf85256
test
stavros-k Nov 22, 2024
dada067
clean
stavros-k Nov 22, 2024
798b9b5
lets not block the rest of the pipeline
stavros-k Nov 22, 2024
0786e20
Merge branch 'master' into docs-workflolw
stavros-k Nov 25, 2024
68e7e35
use files
stavros-k Nov 25, 2024
96b376c
dont fail if its empty
stavros-k Nov 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions .github/scripts/message.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import sys
import json


def get_files_from_file(file: str):
with open(file, "r") as f:
json_files = f.read()

try:
return json.loads(json_files.replace("\\", ""))
except json.JSONDecodeError:
print(f"Failed to decode JSON from {file}", file=sys.stderr)
exit(1)


trains_to_check = ["test", "stable", "enterprise"]
account_to_notify = ["@truenas/docs-team"]


def process(changed_files=[], added_files=[]):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should never use mutable args as default args

changes = {}

for file in changed_files:
if not file.startswith("ix-dev/"):
continue
train = file.split("/")[1]
if train not in trains_to_check:
continue

app = file.split("/")[2]
if file.startswith(f"ix-dev/{train}/{app}/templates/library/base_"):
continue

if train not in changes:
changes[train] = {"apps": {}}

if app not in changes[train]["apps"]:
changes[train]["apps"][app] = {"areas": set([]), "added": set([]), "modified": set([])}

trimmed_file_name = file.replace(f"ix-dev/{train}/{app}/", "")
if file in added_files:
changes[train]["apps"][app]["added"].add(trimmed_file_name)
else:
changes[train]["apps"][app]["modified"].add(trimmed_file_name)

if file.endswith("questions.yaml"):
changes[train]["apps"][app]["areas"].add("ui")
elif file.endswith("app.yaml"):
changes[train]["apps"][app]["areas"].add("metadata")
elif file.endswith("docker-compose.yaml"):
changes[train]["apps"][app]["areas"].add("template")
elif file.endswith("ix_values.yaml"):
changes[train]["apps"][app]["areas"].add("static_config")

return generate_message(changes)


def generate_message(changes):
message = ""
for train in sorted(changes):
message += f"## `{train.title()}`\n\n"
for app in sorted(changes[train]["apps"]):
message += f"### `{app.title()}`\n"
if len(changes[train]["apps"][app]["areas"]) > 0:
fmt_areas = [f"`{a}`" for a in changes[train]["apps"][app]["areas"]]
message += f"Affected areas: {', '.join(fmt_areas)}\n"
if len(changes[train]["apps"][app]["added"]) > 0:
message += "Added files:\n"
for file in sorted(changes[train]["apps"][app]["added"]):
message += f"- `{file}`\n"
message += "\n"
if len(changes[train]["apps"][app]["modified"]) > 0:
message += "Modified files:\n"
for file in sorted(changes[train]["apps"][app]["modified"]):
message += f"- `{file}`\n"
message += "\n"
message += "\n---\n\n"

if message != "":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could do

if message:

message += "Notifying the following about changes to the trains:\n"
message += ", ".join(account_to_notify)
return message


ALL_CHANGED_FILE = ".github/outputs/all_changed_files.json"
ADDED_FILE = ".github/outputs/added_files.json"


def main():
changed_files = get_files_from_file(ALL_CHANGED_FILE)
added_files = get_files_from_file(ADDED_FILE)

print(process(changed_files, added_files))


if __name__ == "__main__":
main()
19 changes: 17 additions & 2 deletions .github/workflows/app-test-suite.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,28 @@ jobs:

- name: Matrix Output
id: changed-apps
# env:
# CHANGED_FILES: ${{ steps.changed-files-json.outputs.all_changed_files }}
run: |
out=$(python3 .github/scripts/changed_apps.py)
echo "changed-apps=${out}" >> $GITHUB_OUTPUT
echo "change-count=$(echo "${out}" | jq -r '.include | length')" >> $GITHUB_OUTPUT

- name: Message Generation
id: message
run: |
python3 .github/scripts/message.py > pr-comment.txt
if [ "$(cat pr-comment.txt)" != "" ]; then
echo "message=true" >> $GITHUB_OUTPUT
else
echo "message=false" >> $GITHUB_OUTPUT
fi
- name: Comment PR
uses: thollander/actions-comment-pull-request@v3
if: steps.message.outputs.message == 'true'
continue-on-error: true
with:
comment-tag: notify-teams
file-path: pr-comment.txt

run-apps:
name: Run Docker Compose Render/Install
needs: changed-files
Expand Down