Skip to content

‐ 1.2.1 Workflow ‐ syncdata

Jump to bottom
bsc7080gbc edited this page Dec 27, 2024 · 1 revision

Workflow: syncdata

The syncdata workflow is designed to maintain and synchronize the data.json file between the updates/data.json file and the root-level data.json in the repository. It performs validation, updates, and commits changes, while also handling errors by notifying maintainers through GitHub Issues. Below is a step-by-step explanation of its functionality.

Trigger Points

  1. On Push to updates/data.json:
    • Automatically activates whenever changes are pushed to the updates/data.json file.
  2. Manual Trigger:
    • Can be manually triggered using the workflow_dispatch option.

Permissions

  • Contents: Write: Allows modification of repository files.
  • Issues: Write: Required to create GitHub Issues if the workflow encounters an error.

Job: syncdata

Environment

  • Runs on ubuntu-latest to ensure a consistent and reliable build environment.

Steps Overview

  1. Checkout the Repository:

    • The workflow retrieves the repository's content using the actions/checkout@v3 action.
    • Configures Git to fetch all history (fetch-depth: 0) for robust processing.

  2. Set up Git Configuration:

    • Sets the Git user identity to Ziggy[bot].
    • Configures the repository's remote URL using a GitHub personal access token stored in the MYSHELF secret.

  3. Validate JSON Files:

    • Uses the jq utility to check if updates/data.json is valid JSON.
    • Fails the workflow and logs an error (syncdata-error-log.txt) if validation fails.

  4. Update the Root data.json File:

    • Copies updates/data.json to the root as data.json.
    • Logs the action or an error if the updates/data.json file is not found.

  5. Commit and Push Changes:

    • Adds the updated data.json to Git.
    • Commits the change with a timestamped message.
    • Pushes the update to the main branch.
    • Logs an error if any Git operations fail, such as adding, committing, or pushing files.

  6. Notify via GitHub Issues on Failure:

    • If any step fails, the workflow creates a GitHub Issue using the peter-evans/create-issue-from-file@v5 action.
    • The issue includes the error details logged in syncdata-error-log.txt and is labeled as a bug.

Detailed Step Descriptions

Step 1: Checkout Repository

The workflow retrieves the repository's latest state, ensuring it includes all history for accurate synchronization. The actions/checkout@v3 action is configured with:

  • A GitHub token for secure authentication.
  • Fetch depth of 0 for complete repository data.

Step 2: Set up Git Configuration

The bot is configured with:

  • User name: Ziggy[bot]
  • Email: ziggy@myshelf.ai The remote origin URL is set to include the MYSHELF secret token for authentication, allowing seamless commits and pushes.

Step 3: Validate JSON Files

Using jq, the workflow ensures updates/data.json is valid JSON. If invalid:

  • An error log (syncdata-error-log.txt) is created.
  • The workflow exits with a failure.

Step 4: Update the Root data.json File

The file at updates/data.json is copied to the root as data.json. This ensures the root file remains up to date. If updates/data.json is missing:

  • The error is logged, and the workflow exits with failure.

Step 5: Commit and Push Changes

The workflow:

  • Adds data.json to Git.
  • Commits with a timestamped message, e.g., "Sync data.json to main branch on 2024-12-26T10:00:00Z."
  • Pushes changes to the main branch. Errors during these operations are logged and halt the workflow.

Step 6: Notify via GitHub Issues on Failure

If any failure occurs, the workflow:

  • Creates a GitHub Issue titled "Data Sync Failure".
  • Attaches the error log as the issue's content.
  • Labels the issue with bug for easy tracking.

Key Benefits

  1. Automation:

    • Ensures data.json is always synchronized with updates from updates/data.json.

  2. Error Resilience:

    • Validates JSON to prevent propagation of invalid data.
    • Notifies maintainers promptly in case of errors.

  3. Transparency:

    • Logs every action, making troubleshooting straightforward.

Usage

  • Push updates to updates/data.json or trigger the workflow manually via GitHub Actions.
  • Monitor logs and GitHub Issues for error notifications or validation feedback.

This workflow ensures the integrity and synchronization of your data files while providing robust error handling and notifications.

Clone this wiki locally