test: add testcase change theme/appearance

autoqa/tests/migration/appearance/setup-appearance.txt

@@ -0,0 +1,69 @@
+prompt = """
+You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task. 
+
+## Output Format
+```
+Thought: ...
+Action: ...
+```
+
+## Action Space
+
+click(start_box='<|box_start|>(x1,y1)<|box_end|>')
+left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
+right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
+drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
+hotkey(key='')
+type(content='') #If you want to submit your input, use "\
+" at the end of `content`.
+scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
+wait() #Sleep for 5s and take a screenshot to check for any changes.
+finished()
+call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
+
+
+## Note
+- Use Chinese in `Thought` part.
+- Summarize your next action (with its target element) in one sentence in `Thought` part.
+
+## User Instruction
+
+You are changing the Appearance theme in the OLD version of the Jan application.
+
+PHASE: SETUP APPEARANCE THEME (OLD VERSION)
+
+Step-by-step instructions:
+
+1. Open the Jan application (OLD version).
+   - If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
+   - If **New Version Available** popup appears on app launch (older versions), click **Remind Me Later** to dismiss it before continuing.
+
+2. Navigate to Appearance settings:
+   - In the bottom-left menu, click **Settings**.
+   - In the left sidebar, click **Appearance**.
+
+3. Change the theme:
+   - In the **Theme** field, click the current theme menu (e.g., **System**, **Light**, or **Dark**).
+   - Select **Dark** (only change if it is not already Dark).
+
+4. Return result:
+   - If the Theme is set to **Dark**, return:
+     {"result": True, "phase": "setup_complete"}
+   - If the Theme cannot be set to **Dark**, return:
+     {"result": False, "phase": "setup_failed"}
+
+CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
+- You MUST respond in English only, not any other language
+- You MUST return ONLY the JSON format below, nothing else
+- Do NOT add any explanations, thoughts, or additional text
+
+   - If the configuration is successful, return:
+     {"result": True, "phase": "setup_complete"}
+   - If there are any issues, return:
+     {"result": False, "phase": "setup_failed"}
+
+IMPORTANT: 
+- Your response must be ONLY the JSON above
+- Do NOT add any other text before or after the JSON
+"""
+

autoqa/tests/migration/appearance/verify-appearance-persistence.txt

@@ -0,0 +1,69 @@
+prompt = """
+You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task. 
+
+## Output Format
+```
+Thought: ...
+Action: ...
+```
+
+## Action Space
+
+click(start_box='<|box_start|>(x1,y1)<|box_end|>')
+left_double(start_box='<|box_start|>(x1,y1)<|box_end|>')
+right_single(start_box='<|box_start|>(x1,y1)<|box_end|>')
+drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>')
+hotkey(key='')
+type(content='') #If you want to submit your input, use "\
+" at the end of `content`.
+scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left')
+wait() #Sleep for 5s and take a screenshot to check for any changes.
+finished()
+call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help.
+
+
+## Note
+- Use Chinese in `Thought` part.
+- Summarize your next action (with its target element) in one sentence in `Thought` part.
+
+## User Instruction
+
+You are verifying that the Appearance theme persists after upgrading to the NEW version of the Jan application.
+
+PHASE: VERIFY APPEARANCE THEME PERSISTENCE (NEW VERSION)
+
+Step-by-step instructions:
+
+1. Open the Jan application (NEW version).
+   - If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing.
+   - If **New Version Available** popup appears on app launch (older versions), click **Remind Me Later** to dismiss it before continuing.
+
+2. Navigate to Appearance settings:
+   - In the bottom-left menu, click **Settings**.
+   - In the left sidebar, click **Appearance**.
+
+3. Verify theme persistence:
+   - Check the **Theme** field menu and read the current theme value.
+   - Confirm the theme is **Dark**.
+
+4. Return result:
+   - If the current theme is **Dark**, return:
+     {"result": True, "phase": "verification_complete"}
+   - Otherwise, return:
+     {"result": False, "phase": "verification_failed"}
+
+CRITICAL INSTRUCTIONS FOR FINAL RESPONSE:
+- You MUST respond in English only, not any other language
+- You MUST return ONLY the JSON format below, nothing else
+- Do NOT add any explanations, thoughts, or additional text
+
+   - If the settings persist and are correct, return:
+     {"result": True, "phase": "verification_complete"}
+   - If there are any issues, return:
+     {"result": False, "phase": "verification_failed"}
+
+IMPORTANT: 
+- Your response must be ONLY the JSON above
+- Do NOT add any other text before or after the JSON
+"""
+

autoqa/windows-qa-checklist.md

@@ -3,7 +3,7 @@
 ## A. Initial update / migration Data check
 
 Before testing, set-up the following in the old version to make sure that we can see the data is properly migrated:
-- [x] Changing appearance / theme to something that is obviously different from default set-up 🔥
+- [x] Changing appearance / theme to something that is obviously different from default set-up ✅
 - [x] Ensure there are a few chat threads 🔥🔥🔥
 - [x] Ensure there are a few favourites / star threads 🔥🔥🔥
 - [x] Ensure there are 2 model downloaded 🔥🔥