Test Script for Causality Check in Baseline Attention Methods

[JinA0218]

This PR introduces a test script designed to validate whether baseline attention mechanisms satisfy the causality condition.

Purpose
The primary goal of this script is to ensure that baseline attention mechanisms maintain the causality condition. By injecting a specific canary value into the input and comparing it with normal inputs, the script checks for any violations in the outputs of context layers and attention probabilities.

Key Features

• Causality Validation:

  • Tests various baseline attention mechanisms (e.g., Performer, standard OPTAttention) for compliance with the causality condition.
  • Compares outputs of normal inputs and canary-injected inputs for discrepancies.

• Baseline Method Configuration:

  • The baseline attention method can be specified using the --baseline argument, enabling flexibility for testing various mechanisms.

• Canary Injection:

  • Injects a high "canary" value (300000) at a specific index in the input sequence to validate robustness and adherence to the causality condition.

• Error Logging:

  • Uses a logarithmic error metric to compute discrepancies in the context layer and attention probabilities. Logs any identified causality violations for further analysis.

• Flexible Input Configurations:

  • Supports adjustable parameters like batch size (N), number of attention heads (H), sequence length (T_DST), and hidden dimension size (HID) through the script.

How to Use
To run the script, use the following command:

python src/main/tests/test_baseline_opt_causality.py --baseline <baseline_method> [--canary]

• --baseline: Specify the baseline attention method to test (default: performer).
• --canary: Enable canary injection to test the robustness of the attention mechanism.