Skip to content

Running A2A test suite on JS example -- missing features #253

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
robert-at-pretension-io opened this issue Apr 20, 2025 · 0 comments
Open

Comments

@robert-at-pretension-io

elliot@Lil-Guy:~/Projects/a2a-test-suite$ ./start_server_and_test_client.sh http://localhost:41241

🔗 Testing against provided server URL: http://localhost:41241

(Using default test authentication: Header='Authorization', Value='Bearer test-token')
Note: Authentication might need adjustment for external servers.

===============================
🧪 Testing Client Functions against http://localhost:41241

--> [Test 1] Running: Get Agent Card...
Command: timeout 15s RUSTFLAGS="-A warnings" cargo run --quiet -- client get-agent-card --url "http://localhost:41241"
Getting agent card from: http://localhost:41241
{
"capabilities": {
"pushNotifications": false,
"stateTransitionHistory": true,
"streaming": true
},
"defaultInputModes": [
"text"
],
"defaultOutputModes": [
"text",
"file"
],
"description": "An agent that generates code based on natural language instructions and streams file outputs.",
"name": "Coder Agent",
"provider": {
"organization": "A2A Samples"
},
"skills": [
{
"description": "Generates code snippets or complete files based on user requests, streaming the results.",
"examples": [
"Write a python function to calculate fibonacci numbers.",
"Create an HTML file with a basic button that alerts 'Hello!' when clicked.",
"Generate a TypeScript class for a user profile with name and email properties.",
"Refactor this Java code to be more efficient.",
"Write unit tests for the following Go function."
],
"id": "code_generation",
"name": "Code Generation",
"tags": [
"code",
"development",
"programming"
]
}
],
"url": "http://localhost:41241",
"version": "0.0.1"
}
✅ [Test 1] Success: Get Agent Card
--> [Test 2] Running: Send Task...
timeout: failed to run command ‘RUSTFLAGS=-A warnings’: No such file or directory
⚠️ [Test 2] Unsupported or not working (Exit Code 127): Send Task
⚠️ Skipping tests dependent on successful task creation.

===============================
🌊 Testing Streaming

--> [Test 3] Running: Start Streaming Task...
Command: timeout 5s RUSTFLAGS="-A warnings" cargo run --quiet -- client stream-task --url "http://localhost:41241" --message "This is a streaming task" & echo $! > stream_pid.txt
✅ [Test 3] Success: Start Streaming Task
Streaming client started (PID 1350763). Letting it run for a few seconds...
Streaming task to: http://localhost:41241
Connected to streaming endpoint. Waiting for updates...
Status update: working (2025-04-20 20:59:57.079 UTC)
Error in stream: Failed to parse final task: data did not match any variant of untagged enum Part
--> Stopping streaming client (PID 1350763)...
⚠️ Skipping Push Notification tests as TASK_ID was not set.

===============================
📦 Testing Task Batching

--> [Test 4] Running: Create Task Batch...
timeout: failed to run command ‘RUSTFLAGS=-A warnings’: No such file or directory
⚠️ [Test 4] Unsupported or not working (Exit Code 127): Create Task Batch
⚠️ Skipping tests dependent on successful batch creation.

===============================
🛠️ Testing Agent Skills (Unofficial)

⏭️ Skipping Agent Skills tests (Use --run-unofficial to include)

===============================
🚨 Testing Error Handling

--> [Test 5] Running: Error handling for non-existent task...
✅ [Test 5] Success: Got expected error for non-existent task
--> [Test 6] Running: Error handling for non-existent skill...
✅ [Test 6] Success: Got expected error for non-existent skill
--> [Test 7] Running: Error handling for non-existent batch...
✅ [Test 7] Success: Got expected error for non-existent batch

===============================
📁 Testing File Operations

--> Creating test file at /tmp/tmp.KzTqcUknC4/test_file.txt...
--> [Test 8] Running: Create Task for File Ops...
timeout: failed to run command ‘RUSTFLAGS=-A warnings’: No such file or directory
⚠️ [Test 8] Unsupported or not working (Exit Code 127): Create Task for File Ops
⚠️ Skipping Upload/List/Download File tests as FILE_TASK_ID was not set.
--> [Test 9] Running: Send Task With File Attachment...
Command: timeout 15s RUSTFLAGS="-A warnings" cargo run --quiet -- client send-task-with-file --url "http://localhost:41241" --message "Task with file attachment" --file-path "/tmp/tmp.KzTqcUknC4/test_file.txt"
Sending task with file attachment /tmp/tmp.KzTqcUknC4/test_file.txt to http://localhost:41241
Error sending task with file: JSON-RPC error: An unknown error occurred. (code: -32603)
⚠️ [Test 9] Unsupported or not working (Exit Code 1): Send Task With File Attachment
--> Creating test JSON data at /tmp/tmp.KzTqcUknC4/test_data.json...
--> [Test 10] Running: Send Task With Data...
timeout: failed to run command ‘RUSTFLAGS=-A warnings’: No such file or directory
⚠️ [Test 10] Unsupported or not working (Exit Code 127): Send Task With Data
⚠️ Skipping Get Task Details with Data test as DATA_TASK_ID was not set.
--> Cleaning up temporary directory /tmp/tmp.KzTqcUknC4...

===============================
🔑 Testing Authentication Validation (Unofficial)

⏭️ Skipping test: Validate Authentication (Use --run-unofficial to include)

===============================
⏱️ Testing Configurable Delays (Unofficial)

⏭️ Skipping Configurable Delays tests (Use --run-unofficial to include)

===============================
🌊 Testing Dynamic Streaming Content (Unofficial)

⏭️ Skipping Dynamic Streaming Content tests (Use --run-unofficial to include)

======================================================
📊 Test Summary

Total Tests Attempted: 11
Successful Tests: 5
Unsupported/Not Working: 5
(Unofficial tests were skipped. Use --run-unofficial to include them)

Some tests were unsupported or not working!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant