feat: add 2 new testcases
This commit is contained in:
parent
a8837f9d76
commit
dab2941630
49
autoqa/tested/models/model-hub.txt
Normal file
49
autoqa/tested/models/model-hub.txt
Normal file
@ -0,0 +1,49 @@
|
|||||||
|
prompt = """
|
||||||
|
|
||||||
|
You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
```\nThought: ...
|
||||||
|
Action: ...\n```
|
||||||
|
|
||||||
|
## Action Space
|
||||||
|
|
||||||
|
click(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
|
||||||
|
hotkey(key='')
|
||||||
|
type(content='') #If you want to submit your input, use \"\\
|
||||||
|
\" at the end of `content`.
|
||||||
|
scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
|
||||||
|
wait() #Sleep for 5s and take a screenshot to check for any changes.
|
||||||
|
finished()
|
||||||
|
call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
|
||||||
|
|
||||||
|
## Note
|
||||||
|
- Use Chinese in `Thought` part.
|
||||||
|
- Summarize your next action (with its target element) in one sentence in `Thought` part.
|
||||||
|
|
||||||
|
## User Instruction
|
||||||
|
|
||||||
|
You are going to verify that **Hugging Face (HF) models are listed properly** in the app.
|
||||||
|
|
||||||
|
Steps:
|
||||||
|
1. If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
|
||||||
|
2. Open the **Hub** page (e.g., from the left sidebar or main navigation) by clicking **Hub**.
|
||||||
|
3. On the **Hub** page, see list of the models appear
|
||||||
|
4. Verification rule:
|
||||||
|
- If you can see **at least one model item** in the Hub list (i.e., the list is not empty), consider the check **passed**.
|
||||||
|
|
||||||
|
CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
|
||||||
|
- You MUST respond in English only, not any other language.
|
||||||
|
- You MUST return ONLY the JSON format below, nothing else.
|
||||||
|
- Do NOT add any explanations, thoughts, or additional text.
|
||||||
|
|
||||||
|
If models are visible in the Hub (list not empty), return: {"result": True}.
|
||||||
|
Otherwise, return: {"result": False}.
|
||||||
|
|
||||||
|
IMPORTANT:
|
||||||
|
- Your response must be ONLY the JSON above.
|
||||||
|
- Do NOT add any other text before or after the JSON.
|
||||||
|
"""
|
||||||
51
autoqa/tested/settings/app-log.txt
Normal file
51
autoqa/tested/settings/app-log.txt
Normal file
@ -0,0 +1,51 @@
|
|||||||
|
prompt = """
|
||||||
|
|
||||||
|
You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
```\nThought: ...
|
||||||
|
Action: ...\n```
|
||||||
|
|
||||||
|
## Action Space
|
||||||
|
|
||||||
|
click(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
|
||||||
|
hotkey(key='')
|
||||||
|
type(content='') #If you want to submit your input, use \"\\
|
||||||
|
\" at the end of `content`.
|
||||||
|
scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
|
||||||
|
wait() #Sleep for 5s and take a screenshot to check for any changes.
|
||||||
|
finished()
|
||||||
|
call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
|
||||||
|
|
||||||
|
## Note
|
||||||
|
- Use Chinese in `Thought` part.
|
||||||
|
- Summarize your next action (with its target element) in one sentence in `Thought` part.
|
||||||
|
|
||||||
|
## User Instruction
|
||||||
|
|
||||||
|
You are going to verify that the **Open Logs** button works correctly in Settings.
|
||||||
|
|
||||||
|
Steps:
|
||||||
|
1. If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
|
||||||
|
2. In the bottom-left menu, click **Settings**.
|
||||||
|
3. In the left sidebar, click on **General**.
|
||||||
|
4. Scroll down until you see the **Data folder** section.
|
||||||
|
5. In the **Data folder** section, click the **Open Logs** button.
|
||||||
|
6. Verification rule:
|
||||||
|
- If an **App Logs - Jan** window is displayed **and it contains some logs**, consider the check **passed**.
|
||||||
|
|
||||||
|
CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
|
||||||
|
- You MUST respond in English only, not any other language.
|
||||||
|
- You MUST return ONLY the JSON format below, nothing else.
|
||||||
|
- Do NOT add any explanations, thoughts, or additional text.
|
||||||
|
|
||||||
|
If the App Logs window appears, return: {"result": True}.
|
||||||
|
Otherwise, return: {"result": False}.
|
||||||
|
|
||||||
|
IMPORTANT:
|
||||||
|
- Your response must be ONLY the JSON above.
|
||||||
|
- Do NOT add any other text before or after the JSON.
|
||||||
|
"""
|
||||||
@ -29,13 +29,13 @@ Before testing, set-up the following in the old version to make sure that we can
|
|||||||
- [x] Custom Provider Set-up
|
- [x] Custom Provider Set-up
|
||||||
|
|
||||||
#### In `Hub`:
|
#### In `Hub`:
|
||||||
- [x] Can see model from HF listed properly 🔥
|
- [x] Can see model from HF listed properly ✅
|
||||||
- [x] Downloaded model will show `Use` instead of `Download` ✅
|
- [x] Downloaded model will show `Use` instead of `Download` ✅
|
||||||
- [x] Toggling on `Downloaded` on the right corner show the correct list of downloaded models 🔥🔥
|
- [x] Toggling on `Downloaded` on the right corner show the correct list of downloaded models 🔥🔥
|
||||||
|
|
||||||
#### In `Settings -> General`:
|
#### In `Settings -> General`:
|
||||||
- [x] Ensure the `App Data` path is the same ✅
|
- [x] Ensure the `App Data` path is the same ✅
|
||||||
- [x] Click Open Logs, App Log will show 🔥
|
- [x] Click Open Logs, App Log will show ✅
|
||||||
|
|
||||||
#### In `Settings -> Model Providers`:
|
#### In `Settings -> Model Providers`:
|
||||||
- [x] Llama.cpp still listed downloaded models and user can chat with the models 🔥🔥🔥
|
- [x] Llama.cpp still listed downloaded models and user can chat with the models 🔥🔥🔥
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user