Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: increase snapshot sampling intervals. #71

Merged
merged 2 commits into from
Jan 27, 2025
Merged

fix: increase snapshot sampling intervals. #71

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f9ac620
fix: increase snapshot sampling intervals.
asafkorem Jan 27, 2025
3e934fc
feat: ask pilot to wait if it detects active loaders or animations.
asafkorem Jan 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/utils/PilotPromptCreator.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ export class PilotPromptCreator {
` Please generate a one-line string that precisely describes the next action the user should take to move closer to their goal,`,
` and another string (which can be greater than one line) which describes your thoughts while creating the step.`,
` If you think that the goal has been reached return a one word action 'success'`,
` If there are any active loaders, spinners, animations, or partially rendered content in the screen, return "wait 3000" as the action.`,
"2. Creating comprehensive UX, Accessibility, and Internationalization reports that include a review, findings, and a score.",
"",
"Please adhere to the instructions below to provide detailed and helpful responses.",
Expand Down Expand Up @@ -329,6 +330,7 @@ The 'Profile' icon may not be properly adapted for different locales.
): string[] {
const steps = [
"Analyze the provided goal, view hierarchy, and previous steps to understand the user's progress and available actions.",
"Check if there are any active loaders, spinners, animations, or partially rendered content in the view hierarchy - if so, the next action should be to wait for 3 seconds ('wait 3000').",
`Consider the elements present in the view hierarchy${isSnapshotImageAttached ? " and the snapshot image" : ""} to determine possible next actions.`,
"Determine the optimal next action the user should take to move closer to their goal.",
"Ensure the action is directly related to available elements in the view hierarchy.",
Expand Down
6 changes: 3 additions & 3 deletions src/utils/SnapshotManager.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ import { TestingFrameworkDriver } from "@/types";
import { SnapshotComparator } from "@/utils/SnapshotComparator";
import crypto from "crypto";

const DEFAULT_POLL_INTERVAL = 100; // ms
const DEFAULT_TIMEOUT = 3000; // ms
const DEFAULT_STABILITY_THRESHOLD = 0.1;
const DEFAULT_POLL_INTERVAL = 500; // ms
const DEFAULT_TIMEOUT = 5000; // ms
const DEFAULT_STABILITY_THRESHOLD = 0.05;

export class SnapshotManager {
constructor(
Expand Down
66 changes: 36 additions & 30 deletions src/utils/__snapshots__/PilotPromptCreator.test.ts.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ You are an AI assistant tasked with:
Please generate a one-line string that precisely describes the next action the user should take to move closer to their goal,
and another string (which can be greater than one line) which describes your thoughts while creating the step.
If you think that the goal has been reached return a one word action 'success'
If there are any active loaders, spinners, animations, or partially rendered content in the screen, return "wait 3000" as the action.
2. Creating comprehensive UX, Accessibility, and Internationalization reports that include a review, findings, and a score.

Please adhere to the instructions below to provide detailed and helpful responses.
Expand Down Expand Up @@ -40,16 +41,17 @@ Your tasks are as follows:
### Please follow these steps carefully:

1. Analyze the provided goal, view hierarchy, and previous steps to understand the user's progress and available actions.
2. Consider the elements present in the view hierarchy and the snapshot image to determine possible next actions.
3. Determine the optimal next action the user should take to move closer to their goal.
4. Ensure the action is directly related to available elements in the view hierarchy.
5. Make sure to create a unique screen name enclosed with <SCREENDESCRIPTION> blocks according to the snapshot and view.
6. Generate a one-line string that precisely describes this next action, enclosed within \`<ACTION>\` tags.
7. Provide a detailed description of your thought process enclosed within \`<THOUGHTS>\` tags.
8. If the goal is achieved, add a \`<SUMMARY>\` block inside the \`<THOUGHTS>\` section, summarizing the overall flow and findings. Also, provide comprehensive overall UX, Accessibility, and Internationalization reviews with total scores, given all the screens seen in previous steps, inside the respective review sections.
9. For each applicable review section (\`UX\`, \`Accessibility\`, \`Internationalization\`), create a comprehensive report enclosed within corresponding tags (e.g., \`<UX>\`, \`<ACCESSIBILITY>\`, \`<INTERNATIONALIZATION>\`), including a summary, findings, and a score out of 10.
10. Ensure each section is clearly labeled and formatted as shown in the examples.
11. If you cannot determine the next action due to ambiguity or missing information, throw an informative error explaining the problem in one sentence.
2. Check if there are any active loaders, spinners, animations, or partially rendered content in the view hierarchy - if so, the next action should be to wait for 3 seconds ('wait 3000').
3. Consider the elements present in the view hierarchy and the snapshot image to determine possible next actions.
4. Determine the optimal next action the user should take to move closer to their goal.
5. Ensure the action is directly related to available elements in the view hierarchy.
6. Make sure to create a unique screen name enclosed with <SCREENDESCRIPTION> blocks according to the snapshot and view.
7. Generate a one-line string that precisely describes this next action, enclosed within \`<ACTION>\` tags.
8. Provide a detailed description of your thought process enclosed within \`<THOUGHTS>\` tags.
9. If the goal is achieved, add a \`<SUMMARY>\` block inside the \`<THOUGHTS>\` section, summarizing the overall flow and findings. Also, provide comprehensive overall UX, Accessibility, and Internationalization reviews with total scores, given all the screens seen in previous steps, inside the respective review sections.
10. For each applicable review section (\`UX\`, \`Accessibility\`, \`Internationalization\`), create a comprehensive report enclosed within corresponding tags (e.g., \`<UX>\`, \`<ACCESSIBILITY>\`, \`<INTERNATIONALIZATION>\`), including a summary, findings, and a score out of 10.
11. Ensure each section is clearly labeled and formatted as shown in the examples.
12. If you cannot determine the next action due to ambiguity or missing information, throw an informative error explaining the problem in one sentence.

### Verify the Prompt

Expand Down Expand Up @@ -225,6 +227,7 @@ You are an AI assistant tasked with:
Please generate a one-line string that precisely describes the next action the user should take to move closer to their goal,
and another string (which can be greater than one line) which describes your thoughts while creating the step.
If you think that the goal has been reached return a one word action 'success'
If there are any active loaders, spinners, animations, or partially rendered content in the screen, return "wait 3000" as the action.
2. Creating comprehensive UX, Accessibility, and Internationalization reports that include a review, findings, and a score.

Please adhere to the instructions below to provide detailed and helpful responses.
Expand Down Expand Up @@ -256,16 +259,17 @@ Your tasks are as follows:
### Please follow these steps carefully:

1. Analyze the provided goal, view hierarchy, and previous steps to understand the user's progress and available actions.
2. Consider the elements present in the view hierarchy to determine possible next actions.
3. Determine the optimal next action the user should take to move closer to their goal.
4. Ensure the action is directly related to available elements in the view hierarchy.
5. Make sure to create a unique screen name enclosed with <SCREENDESCRIPTION> blocks according to the snapshot and view.
6. Generate a one-line string that precisely describes this next action, enclosed within \`<ACTION>\` tags.
7. Provide a detailed description of your thought process enclosed within \`<THOUGHTS>\` tags.
8. If the goal is achieved, add a \`<SUMMARY>\` block inside the \`<THOUGHTS>\` section, summarizing the overall flow and findings. Also, provide comprehensive overall UX, Accessibility, and Internationalization reviews with total scores, given all the screens seen in previous steps, inside the respective review sections.
9. For each applicable review section (\`UX\`, \`Accessibility\`, \`Internationalization\`), create a comprehensive report enclosed within corresponding tags (e.g., \`<UX>\`, \`<ACCESSIBILITY>\`, \`<INTERNATIONALIZATION>\`), including a summary, findings, and a score out of 10.
10. Ensure each section is clearly labeled and formatted as shown in the examples.
11. If you cannot determine the next action due to ambiguity or missing information, throw an informative error explaining the problem in one sentence.
2. Check if there are any active loaders, spinners, animations, or partially rendered content in the view hierarchy - if so, the next action should be to wait for 3 seconds ('wait 3000').
3. Consider the elements present in the view hierarchy to determine possible next actions.
4. Determine the optimal next action the user should take to move closer to their goal.
5. Ensure the action is directly related to available elements in the view hierarchy.
6. Make sure to create a unique screen name enclosed with <SCREENDESCRIPTION> blocks according to the snapshot and view.
7. Generate a one-line string that precisely describes this next action, enclosed within \`<ACTION>\` tags.
8. Provide a detailed description of your thought process enclosed within \`<THOUGHTS>\` tags.
9. If the goal is achieved, add a \`<SUMMARY>\` block inside the \`<THOUGHTS>\` section, summarizing the overall flow and findings. Also, provide comprehensive overall UX, Accessibility, and Internationalization reviews with total scores, given all the screens seen in previous steps, inside the respective review sections.
10. For each applicable review section (\`UX\`, \`Accessibility\`, \`Internationalization\`), create a comprehensive report enclosed within corresponding tags (e.g., \`<UX>\`, \`<ACCESSIBILITY>\`, \`<INTERNATIONALIZATION>\`), including a summary, findings, and a score out of 10.
11. Ensure each section is clearly labeled and formatted as shown in the examples.
12. If you cannot determine the next action due to ambiguity or missing information, throw an informative error explaining the problem in one sentence.

### Verify the Prompt

Expand Down Expand Up @@ -441,6 +445,7 @@ You are an AI assistant tasked with:
Please generate a one-line string that precisely describes the next action the user should take to move closer to their goal,
and another string (which can be greater than one line) which describes your thoughts while creating the step.
If you think that the goal has been reached return a one word action 'success'
If there are any active loaders, spinners, animations, or partially rendered content in the screen, return "wait 3000" as the action.
2. Creating comprehensive UX, Accessibility, and Internationalization reports that include a review, findings, and a score.

Please adhere to the instructions below to provide detailed and helpful responses.
Expand Down Expand Up @@ -483,16 +488,17 @@ Your tasks are as follows:
### Please follow these steps carefully:

1. Analyze the provided goal, view hierarchy, and previous steps to understand the user's progress and available actions.
2. Consider the elements present in the view hierarchy to determine possible next actions.
3. Determine the optimal next action the user should take to move closer to their goal.
4. Ensure the action is directly related to available elements in the view hierarchy.
5. Make sure to create a unique screen name enclosed with <SCREENDESCRIPTION> blocks according to the snapshot and view.
6. Generate a one-line string that precisely describes this next action, enclosed within \`<ACTION>\` tags.
7. Provide a detailed description of your thought process enclosed within \`<THOUGHTS>\` tags.
8. If the goal is achieved, add a \`<SUMMARY>\` block inside the \`<THOUGHTS>\` section, summarizing the overall flow and findings. Also, provide comprehensive overall UX, Accessibility, and Internationalization reviews with total scores, given all the screens seen in previous steps, inside the respective review sections.
9. For each applicable review section (\`UX\`, \`Accessibility\`, \`Internationalization\`), create a comprehensive report enclosed within corresponding tags (e.g., \`<UX>\`, \`<ACCESSIBILITY>\`, \`<INTERNATIONALIZATION>\`), including a summary, findings, and a score out of 10.
10. Ensure each section is clearly labeled and formatted as shown in the examples.
11. If you cannot determine the next action due to ambiguity or missing information, throw an informative error explaining the problem in one sentence.
2. Check if there are any active loaders, spinners, animations, or partially rendered content in the view hierarchy - if so, the next action should be to wait for 3 seconds ('wait 3000').
3. Consider the elements present in the view hierarchy to determine possible next actions.
4. Determine the optimal next action the user should take to move closer to their goal.
5. Ensure the action is directly related to available elements in the view hierarchy.
6. Make sure to create a unique screen name enclosed with <SCREENDESCRIPTION> blocks according to the snapshot and view.
7. Generate a one-line string that precisely describes this next action, enclosed within \`<ACTION>\` tags.
8. Provide a detailed description of your thought process enclosed within \`<THOUGHTS>\` tags.
9. If the goal is achieved, add a \`<SUMMARY>\` block inside the \`<THOUGHTS>\` section, summarizing the overall flow and findings. Also, provide comprehensive overall UX, Accessibility, and Internationalization reviews with total scores, given all the screens seen in previous steps, inside the respective review sections.
10. For each applicable review section (\`UX\`, \`Accessibility\`, \`Internationalization\`), create a comprehensive report enclosed within corresponding tags (e.g., \`<UX>\`, \`<ACCESSIBILITY>\`, \`<INTERNATIONALIZATION>\`), including a summary, findings, and a score out of 10.
11. Ensure each section is clearly labeled and formatted as shown in the examples.
12. If you cannot determine the next action due to ambiguity or missing information, throw an informative error explaining the problem in one sentence.

### Verify the Prompt

Expand Down
Loading