feat: add 2 new testcases
This commit is contained in:
parent
a8837f9d76
commit
dab2941630
49
autoqa/tested/models/model-hub.txt
Normal file
49
autoqa/tested/models/model-hub.txt
Normal file
@ -0,0 +1,49 @@
|
||||
prompt = """
|
||||
|
||||
You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.
|
||||
|
||||
## Output Format
|
||||
```\nThought: ...
|
||||
Action: ...\n```
|
||||
|
||||
## Action Space
|
||||
|
||||
click(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
|
||||
hotkey(key='')
|
||||
type(content='') #If you want to submit your input, use \"\\
|
||||
\" at the end of `content`.
|
||||
scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
|
||||
wait() #Sleep for 5s and take a screenshot to check for any changes.
|
||||
finished()
|
||||
call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
|
||||
|
||||
## Note
|
||||
- Use Chinese in `Thought` part.
|
||||
- Summarize your next action (with its target element) in one sentence in `Thought` part.
|
||||
|
||||
## User Instruction
|
||||
|
||||
You are going to verify that **Hugging Face (HF) models are listed properly** in the app.
|
||||
|
||||
Steps:
|
||||
1. If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
|
||||
2. Open the **Hub** page (e.g., from the left sidebar or main navigation) by clicking **Hub**.
|
||||
3. On the **Hub** page, see list of the models appear
|
||||
4. Verification rule:
|
||||
- If you can see **at least one model item** in the Hub list (i.e., the list is not empty), consider the check **passed**.
|
||||
|
||||
CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
|
||||
- You MUST respond in English only, not any other language.
|
||||
- You MUST return ONLY the JSON format below, nothing else.
|
||||
- Do NOT add any explanations, thoughts, or additional text.
|
||||
|
||||
If models are visible in the Hub (list not empty), return: {"result": True}.
|
||||
Otherwise, return: {"result": False}.
|
||||
|
||||
IMPORTANT:
|
||||
- Your response must be ONLY the JSON above.
|
||||
- Do NOT add any other text before or after the JSON.
|
||||
"""
|
||||
51
autoqa/tested/settings/app-log.txt
Normal file
51
autoqa/tested/settings/app-log.txt
Normal file
@ -0,0 +1,51 @@
|
||||
prompt = """
|
||||
|
||||
You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.
|
||||
|
||||
## Output Format
|
||||
```\nThought: ...
|
||||
Action: ...\n```
|
||||
|
||||
## Action Space
|
||||
|
||||
click(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
|
||||
hotkey(key='')
|
||||
type(content='') #If you want to submit your input, use \"\\
|
||||
\" at the end of `content`.
|
||||
scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
|
||||
wait() #Sleep for 5s and take a screenshot to check for any changes.
|
||||
finished()
|
||||
call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
|
||||
|
||||
## Note
|
||||
- Use Chinese in `Thought` part.
|
||||
- Summarize your next action (with its target element) in one sentence in `Thought` part.
|
||||
|
||||
## User Instruction
|
||||
|
||||
You are going to verify that the **Open Logs** button works correctly in Settings.
|
||||
|
||||
Steps:
|
||||
1. If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
|
||||
2. In the bottom-left menu, click **Settings**.
|
||||
3. In the left sidebar, click on **General**.
|
||||
4. Scroll down until you see the **Data folder** section.
|
||||
5. In the **Data folder** section, click the **Open Logs** button.
|
||||
6. Verification rule:
|
||||
- If an **App Logs - Jan** window is displayed **and it contains some logs**, consider the check **passed**.
|
||||
|
||||
CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
|
||||
- You MUST respond in English only, not any other language.
|
||||
- You MUST return ONLY the JSON format below, nothing else.
|
||||
- Do NOT add any explanations, thoughts, or additional text.
|
||||
|
||||
If the App Logs window appears, return: {"result": True}.
|
||||
Otherwise, return: {"result": False}.
|
||||
|
||||
IMPORTANT:
|
||||
- Your response must be ONLY the JSON above.
|
||||
- Do NOT add any other text before or after the JSON.
|
||||
"""
|
||||
@ -29,13 +29,13 @@ Before testing, set-up the following in the old version to make sure that we can
|
||||
- [x] Custom Provider Set-up
|
||||
|
||||
#### In `Hub`:
|
||||
- [x] Can see model from HF listed properly 🔥
|
||||
- [x] Can see model from HF listed properly ✅
|
||||
- [x] Downloaded model will show `Use` instead of `Download` ✅
|
||||
- [x] Toggling on `Downloaded` on the right corner show the correct list of downloaded models 🔥🔥
|
||||
|
||||
#### In `Settings -> General`:
|
||||
- [x] Ensure the `App Data` path is the same ✅
|
||||
- [x] Click Open Logs, App Log will show 🔥
|
||||
- [x] Click Open Logs, App Log will show ✅
|
||||
|
||||
#### In `Settings -> Model Providers`:
|
||||
- [x] Llama.cpp still listed downloaded models and user can chat with the models 🔥🔥🔥
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user