Test script for Causality Check in OPT model with SEA attention

[JinA0218]

This PR introduces a test script for verifying the causality condition in the OPT model using SEA attention. Inspired by the concept of a stack canary, this test injects random inputs sequentially and checks whether the causality condition is violated in the presence of SEA attention. The results confirm that the OPT model with SEA attention satisfies the causality condition effectively.

Purpose
The primary goal of this script is to ensure that SEA attention mechanisms maintain the causality condition when applied to the Causal Models. By injecting a specific canary value into the input and comparing it with normal inputs, the script checks for any violations in the outputs of context layers and attention probabilities.

Key Features

• generate random hidden states and computes attention outputs using SEA attention.
• check for any discrepancies in causality by comparing outputs with and without injected random inputs.
• handle edge cases and boundary conditions (e.g., canary injection) to ensure robustness.
• Errors (if any) are logged for debugging

Supported File:
src/main/tests/test_perlin_opt_causal.py: Implements the causality check for the OPT model with SEA attention.

How to Run the Test
python src/main/tests/test_perlin_opt_causality.py --canary