feat: add local api server log testcase and download model with url testcase
This commit is contained in:
parent
dab2941630
commit
82eb76a564
56
autoqa/tested/models/download-model-with-url.txt
Normal file
56
autoqa/tested/models/download-model-with-url.txt
Normal file
@ -0,0 +1,56 @@
|
||||
prompt = """
|
||||
|
||||
You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.
|
||||
|
||||
## Output Format
|
||||
```\nThought: ...
|
||||
Action: ...\n```
|
||||
|
||||
## Action Space
|
||||
|
||||
click(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
|
||||
hotkey(key='')
|
||||
type(content='') #If you want to submit your input, use \"\\
|
||||
\" at the end of `content`.
|
||||
scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
|
||||
wait() #Sleep for 5s and take a screenshot to check for any changes.
|
||||
finished()
|
||||
call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
|
||||
|
||||
## Note
|
||||
- Use Chinese in `Thought` part.
|
||||
- Summarize your next action (with its target element) in `Thought` part.
|
||||
|
||||
## User Instruction
|
||||
|
||||
You are going to verify that a **Hugging Face model can be added from its URL and downloaded successfully**.
|
||||
|
||||
Steps:
|
||||
1. If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
|
||||
2. In the bottom-left menu, click **Hub**.
|
||||
3. In the Hub search bar, paste the model URL:
|
||||
`https://huggingface.co/Menlo/Lucy-gguf/tree/main`
|
||||
4. Press **Enter** to search.
|
||||
5. In the results, click **Download** for the model.
|
||||
6. Wait until the **Download** button changes to **Use**.
|
||||
- This indicates the model has been fully downloaded.
|
||||
|
||||
Verification rule:
|
||||
- Consider the check **passed** if the button changes to **Use** after the download completes.
|
||||
- If the button does not change to **Use**, the check **fails**.
|
||||
|
||||
CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
|
||||
- You MUST respond in English only, not any other language.
|
||||
- You MUST return ONLY the JSON format below, nothing else.
|
||||
- Do NOT add any explanations, thoughts, or additional text.
|
||||
|
||||
If the model downloads successfully and the button changes to "Use", return: {"result": True}.
|
||||
Otherwise, return: {"result": False}.
|
||||
|
||||
IMPORTANT:
|
||||
- Your response must be ONLY the JSON above.
|
||||
- Do NOT add any other text before or after the JSON.
|
||||
"""
|
||||
52
autoqa/tested/settings/local-api-server-logs.txt
Normal file
52
autoqa/tested/settings/local-api-server-logs.txt
Normal file
@ -0,0 +1,52 @@
|
||||
prompt = """
|
||||
|
||||
You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.
|
||||
|
||||
## Output Format
|
||||
```\nThought: ...
|
||||
Action: ...\n```
|
||||
|
||||
## Action Space
|
||||
|
||||
click(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
|
||||
drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
|
||||
hotkey(key='')
|
||||
type(content='') #If you want to submit your input, use \"\\
|
||||
\" at the end of `content`.
|
||||
scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
|
||||
wait() #Sleep for 5s and take a screenshot to check for any changes.
|
||||
finished()
|
||||
call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
|
||||
|
||||
## Note
|
||||
- Use Chinese in `Thought` part.
|
||||
- Summarize your next action (with its target element) in `Thought` part.
|
||||
|
||||
## User Instruction
|
||||
|
||||
You are going to verify that the **Local API Server** can start and logs are displayed correctly.
|
||||
|
||||
Steps:
|
||||
1. If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
|
||||
2. In the bottom-left menu, click **Settings**.
|
||||
3. In the left sidebar, click on **Local API Server**.
|
||||
4. Under **Server Configuration**, type `"1234"` into the **API Key** field **if the field is empty**, otherwise ignore this step.
|
||||
5. Click the **Start Server** button in the Local API Server section.
|
||||
6. In the **Server Logs** area, click **Open Logs**.
|
||||
7. Verification rule:
|
||||
- If the logs window shows entries such as `INFO Jan API server started on http://127.0.0.1:1337` (or similar log lines confirming the server started), consider the check **passed**.
|
||||
|
||||
CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
|
||||
- You MUST respond in English only, not any other language.
|
||||
- You MUST return ONLY the JSON format below, nothing else.
|
||||
- Do NOT add any explanations, thoughts, or additional text.
|
||||
|
||||
If the server logs window shows that the server started (e.g., contains an INFO startup log), return: {"result": True}.
|
||||
Otherwise, return: {"result": False}.
|
||||
|
||||
IMPORTANT:
|
||||
- Your response must be ONLY the JSON above.
|
||||
- Do NOT add any other text before or after the JSON.
|
||||
"""
|
||||
@ -162,7 +162,7 @@ Ensure that the following section information show up for hardware
|
||||
- [x] User can `Start Server` and chat with the default endpoint
|
||||
- [x] User should see the correct model name at `v1/models`
|
||||
- [x] User should be able to chat with it at `v1/chat/completions`
|
||||
- [x] `Open Logs` show the correct query log send to the server and return from the server
|
||||
- [x] `Open Logs` show the correct query log send to the server and return from the server ✅
|
||||
- [x] Make sure that changing all the parameter in `Server Configuration` is reflected when `Start Server`
|
||||
|
||||
#### In `HTTPS Proxy`:
|
||||
@ -171,7 +171,7 @@ Ensure that the following section information show up for hardware
|
||||
## C. Hub
|
||||
- [x] User can click `Download` to download a model ✅
|
||||
- [x] User can cancel a model in the middle of downloading 🔥🔥🔥
|
||||
- [x] User can add a Hugging Face model detail to the list by pasting a model name / model url into the search bar and press enter 🔥
|
||||
- [x] User can add a Hugging Face model detail to the list by pasting a model name / model url into the search bar and press enter ✅
|
||||
- [x] Clicking on a listing will open up the model card information within Jan and render the HTML properly
|
||||
- [x] Clicking download work on the `Show variants` section 🔥🔥🔥
|
||||
- [x] Clicking download work inside the Model card HTML 🔥🔥🔥
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user