feat: add local api server log testcase and download model with url testcase
This commit is contained in:
parent
dab2941630
commit
82eb76a564
56
autoqa/tested/models/download-model-with-url.txt
Normal file
56
autoqa/tested/models/download-model-with-url.txt
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
prompt = """
|
||||||
|
|
||||||
|
You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
```\nThought: ...
|
||||||
|
Action: ...\n```
|
||||||
|
|
||||||
|
## Action Space
|
||||||
|
|
||||||
|
click(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
|
||||||
|
hotkey(key='')
|
||||||
|
type(content='') #If you want to submit your input, use \"\\
|
||||||
|
\" at the end of `content`.
|
||||||
|
scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
|
||||||
|
wait() #Sleep for 5s and take a screenshot to check for any changes.
|
||||||
|
finished()
|
||||||
|
call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
|
||||||
|
|
||||||
|
## Note
|
||||||
|
- Use Chinese in `Thought` part.
|
||||||
|
- Summarize your next action (with its target element) in `Thought` part.
|
||||||
|
|
||||||
|
## User Instruction
|
||||||
|
|
||||||
|
You are going to verify that a **Hugging Face model can be added from its URL and downloaded successfully**.
|
||||||
|
|
||||||
|
Steps:
|
||||||
|
1. If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
|
||||||
|
2. In the bottom-left menu, click **Hub**.
|
||||||
|
3. In the Hub search bar, paste the model URL:
|
||||||
|
`https://huggingface.co/Menlo/Lucy-gguf/tree/main`
|
||||||
|
4. Press **Enter** to search.
|
||||||
|
5. In the results, click **Download** for the model.
|
||||||
|
6. Wait until the **Download** button changes to **Use**.
|
||||||
|
- This indicates the model has been fully downloaded.
|
||||||
|
|
||||||
|
Verification rule:
|
||||||
|
- Consider the check **passed** if the button changes to **Use** after the download completes.
|
||||||
|
- If the button does not change to **Use**, the check **fails**.
|
||||||
|
|
||||||
|
CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
|
||||||
|
- You MUST respond in English only, not any other language.
|
||||||
|
- You MUST return ONLY the JSON format below, nothing else.
|
||||||
|
- Do NOT add any explanations, thoughts, or additional text.
|
||||||
|
|
||||||
|
If the model downloads successfully and the button changes to "Use", return: {"result": True}.
|
||||||
|
Otherwise, return: {"result": False}.
|
||||||
|
|
||||||
|
IMPORTANT:
|
||||||
|
- Your response must be ONLY the JSON above.
|
||||||
|
- Do NOT add any other text before or after the JSON.
|
||||||
|
"""
|
||||||
52
autoqa/tested/settings/local-api-server-logs.txt
Normal file
52
autoqa/tested/settings/local-api-server-logs.txt
Normal file
@ -0,0 +1,52 @@
|
|||||||
|
prompt = """
|
||||||
|
|
||||||
|
You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
```\nThought: ...
|
||||||
|
Action: ...\n```
|
||||||
|
|
||||||
|
## Action Space
|
||||||
|
|
||||||
|
click(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||||
|
drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
|
||||||
|
hotkey(key='')
|
||||||
|
type(content='') #If you want to submit your input, use \"\\
|
||||||
|
\" at the end of `content`.
|
||||||
|
scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
|
||||||
|
wait() #Sleep for 5s and take a screenshot to check for any changes.
|
||||||
|
finished()
|
||||||
|
call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
|
||||||
|
|
||||||
|
## Note
|
||||||
|
- Use Chinese in `Thought` part.
|
||||||
|
- Summarize your next action (with its target element) in `Thought` part.
|
||||||
|
|
||||||
|
## User Instruction
|
||||||
|
|
||||||
|
You are going to verify that the **Local API Server** can start and logs are displayed correctly.
|
||||||
|
|
||||||
|
Steps:
|
||||||
|
1. If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
|
||||||
|
2. In the bottom-left menu, click **Settings**.
|
||||||
|
3. In the left sidebar, click on **Local API Server**.
|
||||||
|
4. Under **Server Configuration**, type `"1234"` into the **API Key** field **if the field is empty**, otherwise ignore this step.
|
||||||
|
5. Click the **Start Server** button in the Local API Server section.
|
||||||
|
6. In the **Server Logs** area, click **Open Logs**.
|
||||||
|
7. Verification rule:
|
||||||
|
- If the logs window shows entries such as `INFO Jan API server started on http://127.0.0.1:1337` (or similar log lines confirming the server started), consider the check **passed**.
|
||||||
|
|
||||||
|
CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
|
||||||
|
- You MUST respond in English only, not any other language.
|
||||||
|
- You MUST return ONLY the JSON format below, nothing else.
|
||||||
|
- Do NOT add any explanations, thoughts, or additional text.
|
||||||
|
|
||||||
|
If the server logs window shows that the server started (e.g., contains an INFO startup log), return: {"result": True}.
|
||||||
|
Otherwise, return: {"result": False}.
|
||||||
|
|
||||||
|
IMPORTANT:
|
||||||
|
- Your response must be ONLY the JSON above.
|
||||||
|
- Do NOT add any other text before or after the JSON.
|
||||||
|
"""
|
||||||
@ -162,7 +162,7 @@ Ensure that the following section information show up for hardware
|
|||||||
- [x] User can `Start Server` and chat with the default endpoint
|
- [x] User can `Start Server` and chat with the default endpoint
|
||||||
- [x] User should see the correct model name at `v1/models`
|
- [x] User should see the correct model name at `v1/models`
|
||||||
- [x] User should be able to chat with it at `v1/chat/completions`
|
- [x] User should be able to chat with it at `v1/chat/completions`
|
||||||
- [x] `Open Logs` show the correct query log send to the server and return from the server
|
- [x] `Open Logs` show the correct query log send to the server and return from the server ✅
|
||||||
- [x] Make sure that changing all the parameter in `Server Configuration` is reflected when `Start Server`
|
- [x] Make sure that changing all the parameter in `Server Configuration` is reflected when `Start Server`
|
||||||
|
|
||||||
#### In `HTTPS Proxy`:
|
#### In `HTTPS Proxy`:
|
||||||
@ -171,7 +171,7 @@ Ensure that the following section information show up for hardware
|
|||||||
## C. Hub
|
## C. Hub
|
||||||
- [x] User can click `Download` to download a model ✅
|
- [x] User can click `Download` to download a model ✅
|
||||||
- [x] User can cancel a model in the middle of downloading 🔥🔥🔥
|
- [x] User can cancel a model in the middle of downloading 🔥🔥🔥
|
||||||
- [x] User can add a Hugging Face model detail to the list by pasting a model name / model url into the search bar and press enter 🔥
|
- [x] User can add a Hugging Face model detail to the list by pasting a model name / model url into the search bar and press enter ✅
|
||||||
- [x] Clicking on a listing will open up the model card information within Jan and render the HTML properly
|
- [x] Clicking on a listing will open up the model card information within Jan and render the HTML properly
|
||||||
- [x] Clicking download work on the `Show variants` section 🔥🔥🔥
|
- [x] Clicking download work on the `Show variants` section 🔥🔥🔥
|
||||||
- [x] Clicking download work inside the Model card HTML 🔥🔥🔥
|
- [x] Clicking download work inside the Model card HTML 🔥🔥🔥
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user