From af5afc00f6a95643d781701e1b7ff13bb9135451 Mon Sep 17 00:00:00 2001 From: Minh141120 Date: Wed, 3 Sep 2025 13:45:38 +0700 Subject: [PATCH] test: add migration modify local api server test case --- .../settings/setup-local-api-server.txt | 76 +++++++++++++++++++ .../verify-local-api-server-persistence.txt | 76 +++++++++++++++++++ autoqa/windows-qa-checklist.md | 2 +- 3 files changed, 153 insertions(+), 1 deletion(-) create mode 100644 autoqa/tests/migration/settings/setup-local-api-server.txt create mode 100644 autoqa/tests/migration/settings/verify-local-api-server-persistence.txt diff --git a/autoqa/tests/migration/settings/setup-local-api-server.txt b/autoqa/tests/migration/settings/setup-local-api-server.txt new file mode 100644 index 000000000..1a34a06b2 --- /dev/null +++ b/autoqa/tests/migration/settings/setup-local-api-server.txt @@ -0,0 +1,76 @@ +prompt = """ +You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task. + +## Output Format +```\nThought: ... +Action: ...\n``` + +## Action Space + +click(start_box='<|box_start|>(x1,y1)<|box_end|>') +left_double(start_box='<|box_start|>(x1,y1)<|box_end|>') +right_single(start_box='<|box_start|>(x1,y1)<|box_end|>') +drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>') +hotkey(key='') +type(content='') #If you want to submit your input, use \"\ +\" at the end of `content`. +scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left') +wait() #Sleep for 5s and take a screenshot to check for any changes. +finished() +call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help. + + +## Note +- Use Chinese in `Thought` part. +- Summarize your next action (with its target element) in one sentence in `Thought` part. + +## User Instruction + +You are setting up local API server settings in the OLD version of the Jan application. + +PHASE: SETUP LOCAL API SERVER (OLD VERSION) + +Step-by-step instructions: + +1. Open the Jan application (OLD version). + - If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing. + - If **New Version Available** popup appears on app launch (older versions), click **Remind Me Later** to dismiss it before continuing. + +2. Navigate to Local API Server settings: + - In the bottom-left menu, click **Settings**. + - In the left sidebar, click **Local API Server**. + +3. Configure the server settings: + - Under **Server Configuration**, find the **Server Port** field. + - If the port is not already `1338`, change it to `1338`. + - Find the **API Prefix** field. + - If the prefix is not already `/v2`, change it to `/v2`. + - Find the **API Key** field (look for placeholder "Enter API Key"). + - If the API Key field appears empty, enter `1234` as the API key value (this field is required for server startup). + - Verify the Server Port and API Prefix values are set to their target values. + +4. Test server functionality: + - Click the **Start Server** button. + - In the **Server Logs** area, click **Open Logs**. + - Verify the server starts successfully and shows startup logs. + +5. Return result: + - If the Server Port is set to `1338` AND the API Prefix is set to `/v2` AND the server starts successfully, return: + {"result": True, "phase": "setup_complete"} + - If there are any issues with configuration or server startup, return: + {"result": False, "phase": "setup_failed"} + +CRITICAL INSTRUCTIONS FOR FINAL RESPONSE: +- You MUST respond in English only, not any other language +- You MUST return ONLY the JSON format below, nothing else +- Do NOT add any explanations, thoughts, or additional text + + - If the configuration is successful and server works, return: + {"result": True, "phase": "setup_complete"} + - If there are any issues, return: + {"result": False, "phase": "setup_failed"} + +IMPORTANT: +- Your response must be ONLY the JSON above +- Do NOT add any other text before or after the JSON +""" \ No newline at end of file diff --git a/autoqa/tests/migration/settings/verify-local-api-server-persistence.txt b/autoqa/tests/migration/settings/verify-local-api-server-persistence.txt new file mode 100644 index 000000000..5ed00497e --- /dev/null +++ b/autoqa/tests/migration/settings/verify-local-api-server-persistence.txt @@ -0,0 +1,76 @@ +prompt = """ +You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task. + +## Output Format +```\nThought: ... +Action: ...\n``` + +## Action Space + +click(start_box='<|box_start|>(x1,y1)<|box_end|>') +left_double(start_box='<|box_start|>(x1,y1)<|box_end|>') +right_single(start_box='<|box_start|>(x1,y1)<|box_end|>') +drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>') +hotkey(key='') +type(content='') #If you want to submit your input, use \"\ +\" at the end of `content`. +scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left') +wait() #Sleep for 5s and take a screenshot to check for any changes. +finished() +call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help. + + +## Note +- Use Chinese in `Thought` part. +- Summarize your next action (with its target element) in one sentence in `Thought` part. + +## User Instruction + +You are verifying that local API server settings persist after upgrading to the NEW version of the Jan application. + +PHASE: VERIFY LOCAL API SERVER PERSISTENCE (NEW VERSION) + +Step-by-step instructions: + +1. Open the Jan application (NEW version). + - If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing. + - If **New Version Available** popup appears on app launch (older versions), click **Remind Me Later** to dismiss it before continuing. + +2. Navigate to Local API Server settings: + - In the bottom-left menu, click **Settings**. + - In the left sidebar, click **Local API Server**. + +3. Verify the server settings persist: + - Under **Server Configuration**, check the **Server Port** field. + - Verify it still contains the value `1338` that was set in the old version. + - Check the **API Prefix** field. + - Verify it still contains the value `/v2` that was set in the old version. + - Check the **API Key** field (look for placeholder "Enter API Key"). + - If the API Key field appears empty, enter `1234` as the API key value (this field is required for server startup). + - If the Server Port is not `1338` OR the API Prefix is not `/v2`, the verification fails. + +4. Test server functionality: + - Click the **Start Server** button. + - In the **Server Logs** area, click **Open Logs**. + - Verify the server starts successfully and shows startup logs. + +5. Return result: + - If the Server Port remains `1338` AND the API Prefix remains `/v2` AND the server starts successfully, return: + {"result": True, "phase": "verification_complete"} + - If any of the settings have reverted OR the server fails to start, return: + {"result": False, "phase": "verification_failed"} + +CRITICAL INSTRUCTIONS FOR FINAL RESPONSE: +- You MUST respond in English only, not any other language +- You MUST return ONLY the JSON format below, nothing else +- Do NOT add any explanations, thoughts, or additional text + + - If the settings persist and server works correctly, return: + {"result": True, "phase": "verification_complete"} + - If there are any issues, return: + {"result": False, "phase": "verification_failed"} + +IMPORTANT: +- Your response must be ONLY the JSON above +- Do NOT add any other text before or after the JSON +""" \ No newline at end of file diff --git a/autoqa/windows-qa-checklist.md b/autoqa/windows-qa-checklist.md index bdce38ba3..3ff28ac48 100644 --- a/autoqa/windows-qa-checklist.md +++ b/autoqa/windows-qa-checklist.md @@ -9,7 +9,7 @@ Before testing, set-up the following in the old version to make sure that we can - [x] Ensure there are 2 model downloaded 🔥🔥 - [x] Ensure there are 2 import on local provider (llama.cpp) - [x] Modify MCP servers list and add some ENV value to MCP servers -- [x] Modify Local API Server 🔥 +- [x] Modify Local API Server ✅ - [x] HTTPS proxy config value 🔥 - [x] Add 2 custom assistants to Jan 🔥🔥 - [x] Create a new chat with the custom assistant 🔥🔥🔥