diff --git a/autoqa/tested/models/disable-selected-providers.txt b/autoqa/tested/models/disable-selected-providers.txt new file mode 100644 index 000000000..024423e47 --- /dev/null +++ b/autoqa/tested/models/disable-selected-providers.txt @@ -0,0 +1,78 @@ +prompt = """ +You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task. + +## Output Format +``` +Thought: ... +Action: ... +``` + +## Action Space + +click(start_box='<|box_start|>(x1,y1)<|box_end|>') +left_double(start_box='<|box_start|>(x1,y1)<|box_end|>') +right_single(start_box='<|box_start|>(x1,y1)<|box_end|>') +drag(start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x3,y3)<|box_end|>') +hotkey(key='') +type(content='') #If you want to submit your input, use "\ +" at the end of `content`. +scroll(start_box='<|box_start|>(x1,y1)<|box_end|>', direction='down or up or right or left') +wait() #Sleep for 5s and take a screenshot to check for any changes. +finished() +call_user() # Submit the task and call the user when the task is unsolvable, or when you need the user's help. + + +## Note +- Use Chinese in `Thought` part. +- Summarize your next action (with its target element) in one sentence in `Thought` part. + +## User Instruction + +You are disabling specific Model Providers in the Jan application to verify they are hidden from the sidebar list. + +TARGET PROVIDERS TO DISABLE: Llama.cpp, OpenAI, Anthropic + +Step-by-step instructions: + +1. Open the Jan application. + - If a dialog appears in the bottom-right corner titled **"Help Us Improve Jan"**, click **Deny** to dismiss it before continuing. + - If **New Version Available** popup appears on app launch (older versions), click **Remind Me Later** to dismiss it before continuing. + +2. Navigate to Model Providers settings: + - In the bottom-left menu, click **Settings**. + - In the left sidebar, click **Model Providers**. + +3. Disable selected providers: + - For each of the following providers on the right panel (Llama.cpp, OpenAI, Anthropic): + - Locate the provider row. + - On the right side of the row, find the enable/disable chip/toggle (enabled by default). + - If it is enabled, click the chip/toggle to disable it. + - Immediately call `wait()` once to allow the UI to update and the next screenshot to be captured for the model. + - If it is already disabled, leave it as is (no click). Still call `wait()` once to confirm state in the next screenshot. + +4. Verify providers are hidden in the sidebar list: + - Call `wait()` once more to ensure the sidebar reflects the latest state. + - Check the left sidebar list under **Model Providers**. + - Confirm that the entries for **Llama.cpp**, **OpenAI**, and **Anthropic** are not visible in the sidebar after disabling. + +5. Return result: + - If all three providers (Llama.cpp, OpenAI, Anthropic) are disabled and not visible in the sidebar list, return: + {"result": True} + - If any of the three providers is still visible in the sidebar list after being disabled, return: + {"result": False} + +CRITICAL INSTRUCTIONS FOR FINAL RESPONSE: +- You MUST respond in English only, not any other language +- You MUST return ONLY the JSON format below, nothing else +- Do NOT add any explanations, thoughts, or additional text + + - On success, return: + {"result": True} + - On failure, return: + {"result": False} + +IMPORTANT: +- Your response must be ONLY the JSON above +- Do NOT add any other text before or after the JSON +""" + diff --git a/autoqa/windows-qa-checklist.md b/autoqa/windows-qa-checklist.md index a62f61030..0b1d5be77 100644 --- a/autoqa/windows-qa-checklist.md +++ b/autoqa/windows-qa-checklist.md @@ -15,7 +15,7 @@ Before testing, set-up the following in the old version to make sure that we can - [x] Create a new chat with the custom assistant 🔥🔥🔥 - [x] Change the `App Data` to some other folder - [x] Create a Custom Provider 🔥🔥 (Not verified yet) -- [x] Disabled some model providers 🔥🔥🔥 +- [x] Disabled some model providers ✅ (Intermittently work) #### Validate that the update does not corrupt existing user data or settings (before and after update show the same information): - [x] Threads - [x] Previously used model and assistants is shown correctly