Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #5597 from menloresearch/docs/update-explanation

Browse Source 
docs: update explanation and model parameter sections
This commit is contained in:
Daniel Ching 2025-07-01 11:58:45 +08:00 committed by GitHub
commit e274d04d9b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 119 additions and 119 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/src/pages/docs/_assets/llama.cpp-01-updated.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 512 KiB

166
docs/src/pages/docs/data-folder.mdx
View File

@ -56,36 +56,37 @@ cd ~/.config/Jan/data # Default install
<Callout type="info"> <Callout type="info">
Root directory: `~/jan` Root directory: `~/jan`
</Callout> </Callout>
```sh ```sh
/assistants /assistants/
/jan /jan/
assistant.json assistant.json
/extensions /engines/
/llama.cpp/
/extensions/
extensions.json extensions.json
/@janhq /@janhq/
/extension_A /assistant-extension/
package.json /conversational-extension/
/logs /download-extension/
/app.txt /engine-management-extension/
/models /hardware-management-extension/
/model_A /inference-cortex-extension/
model.yaml /model-extension/
/files/
/logs/
app.log
/models/
/huggingface.co/
/Model_Provider_A/
/Model_A
model_A.gguf
model_A.yaml model_A.yaml
/settings /threads/
settings.json /thread_A/
/@janhq
/extension_A_Settings
settings.json
/themes
/dark-dimmed
/joi-dark
/joi-light
/night-blue
/threads
/jan_thread_A
messages.jsonl messages.jsonl
thread.json thread.json
messages.jsonl
``` ```
### `assistants/` ### `assistants/`
@ -93,14 +94,28 @@ Where AI personalities live. The default one (`/assistants/jan/`):
```json ```json
{ {
"avatar": "", "avatar": "👋",
"id": "jan", "id": "jan",
"object": "assistant", "object": "assistant",
"created_at": 1715132389207, "created_at": 1750945742.536,
"name": "Jan", "name": "Jan",
"description": "A default assistant that can use all downloaded models", "description": "Jan is a helpful AI assistant that can use tools and help complete tasks for its users.",
"model": "*", "model": "*",
"instructions": "" "instructions": "You have access to a set of tools to help you answer the user’s question. You can use only one tool per message, and you’ll receive the result of that tool in the user’s next response. To complete a task, use tools step by step—each step should be guided by the outcome of the previous one.\nTool Usage Rules:\n1. Always provide the correct values as arguments when using tools. Do not pass variable names—use actual values instead.\n2. You may perform multiple tool steps to complete a task.\n3. Avoid repeating a tool call with exactly the same parameters to prevent infinite loops.",
"tools": [
{
"type": "retrieval",
"enabled": false,
"useTimeWeightedRetriever": false,
"settings": {
"top_k": 2,
"chunk_size": 1024,
"chunk_overlap": 64,
"retrieval_template": "Use the following pieces of context to answer the question at the end.\n----------------\nCONTEXT: {CONTEXT}\n----------------\nQUESTION: {QUESTION}\n----------------\nHelpful Answer:"
}
}
],
"file_ids": []
} }
``` ```
@ -140,75 +155,47 @@ Debugging headquarters (`/logs/app.txt`):
The silicon brain collection. Each model has its own `model.json`. The silicon brain collection. Each model has its own `model.json`.
<Callout type="info"> <Callout type="info">
Full parameters: [here](/docs/models/model-parameters) Full parameters: [here](/docs/model-parameters)
</Callout> </Callout>
### `settings/`
Control panel. Extension settings in `/settings/@janhq/`:
| Parameter | Description |
|----------------|----------------------------------------------------|
| key | Setting identifier |
| title | Display name |
| description | Setting explanation |
| controllerType | UI component type |
| controllerProps| Component properties |
| extensionName | Parent extension link |
GPU settings (`settings.json`):
| Parameter | Description |
|----------------------|--------------------------------------------|
| notify | Notification status |
| run_mode | Operating mode |
| nvidia_driver.exist | NVIDIA driver presence |
| nvidia_driver.version| Driver version |
| cuda.exist | CUDA availability |
| cuda.version | CUDA version |
| gpus[0].id | GPU identifier |
| gpus[0].vram | GPU memory (MB) |
| gpus[0].name | GPU model |
| gpus[0].arch | GPU architecture |
| gpu_highest_vram | Most capable GPU |
| gpus_in_use | Active GPUs |
| is_initial | First run flag |
| vulkan | Vulkan support |
### `themes/`
Visual wardrobe. Each theme's `theme.json`:
| Parameter | Description |
|------------------|-------------------------------------------|
| id | Theme identifier |
| displayName | UI name |
| reduceTransparent| Transparency control |
| nativeTheme | OS theme sync |
| variables | Component settings |
### `threads/` ### `threads/`
Chat archive. Each thread (`/threads/jan_unixstamp/`) contains: Chat archive. Each thread (`/threads/jan_unixstamp/`) contains:
- `messages.jsonl`: - `messages.jsonl`:
```json ```json
{ {
"id":"01J6Y6FH8PFTHQB5PNJTHEN27C", "completed_at": 0,
"thread_id":"jan_1725437954", "content": [
"type":"Thread",
"role":"assistant",
"content":
[
{ {
"type": "text",
"text": { "text": {
"value": "Hello! Is there something I can help you with or would you like to chat?", "annotations": [],
"annotations": [] "value": "Hello! I can help you with various tasks. I can search for information on the internet, including news, videos, images, shopping, and more. I can also scrape webpages to extract specific information. Let me know what you need!"
} },
"type": "text"
} }
], ],
"created_at": 1751012639307,
"id": "01JYR7S0JB5ZBGMJV52KWMW5VW",
"metadata": {
"assistant": {
"avatar": "👋",
"id": "jan",
"instructions": "You have access to a set of tools to help you answer the user's question. You can use only one tool per message, and you'll receive the result of that tool in the user's next response. To complete a task, use tools step by step—each step should be guided by the outcome of the previous one.\nTool Usage Rules:\n1. Always provide the correct values as arguments when using tools. Do not pass variable names—use actual values instead.\n2. You may perform multiple tool steps to complete a task.\n3. Avoid repeating a tool call with exactly the same parameters to prevent infinite loops.",
"name": "Jan",
"parameters": ""
},
"tokenSpeed": {
"lastTimestamp": 1751012637097,
"message": "01JYR7S0GW5M9PSHMRE7T8VQJM",
"tokenCount": 49,
"tokenSpeed": 22.653721682847895
}
},
"object": "thread.message",
"role": "assistant",
"status": "ready", "status": "ready",
"created": 1725442802966, "thread_id": "8f2c9922-db49-4d1e-8620-279c05baf2d0",
"updated": 1725442802966, "type": "text"
"object": "thread.message"
} }
``` ```
@ -216,12 +203,17 @@ Chat archive. Each thread (`/threads/jan_unixstamp/`) contains:
| Parameter | Description | | Parameter | Description |
|------------|------------------------------------------------| |------------|------------------------------------------------|
| assistants | Assistant configuration clone |
| created | Creation timestamp |
| id | Thread identifier | | id | Thread identifier |
| metadata | Additional thread data |
| model | Active model settings |
| object | OpenAI compatibility marker | | object | OpenAI compatibility marker |
| title | Thread name | | title | Thread name |
| assistants | Assistant configuration clone | | updated | Updated timestamp |
| model | Active model settings |
| metadata | Additional thread data |
## Delete Jan Data ## Delete Jan Data
Uninstall guides: [Mac](/docs/desktop/mac#step-2-clean-up-data-optional), Uninstall guides: [Mac](/docs/desktop/mac#step-2-clean-up-data-optional),

3
docs/src/pages/docs/llama-cpp.mdx
View File

@ -33,7 +33,7 @@ import { Settings, EllipsisVertical, Plus, FolderOpen, Pencil } from 'lucide-rea
Jan uses **llama.cpp** for running local AI models. You can find its settings in **Settings** (<Settings width={16} height={16} style={{display:"inline"}}/>) > **Local Engine** > **llama.cpp**: Jan uses **llama.cpp** for running local AI models. You can find its settings in **Settings** (<Settings width={16} height={16} style={{display:"inline"}}/>) > **Local Engine** > **llama.cpp**:
<br/> <br/>
![llama.cpp](./_assets/llama.cpp-01.png) ![llama.cpp](./_assets/llama.cpp-01-updated.png)
<br/> <br/>
These settings are for advanced users, you would want to check these settings when: These settings are for advanced users, you would want to check these settings when:
@ -151,6 +151,7 @@ For detailed hardware compatibility, please visit our guide for [Mac](/docs/desk
| **Caching** | - Enable to store recent prompts and responses<br></br>- Improves response time for repeated prompts | Enabled | | **Caching** | - Enable to store recent prompts and responses<br></br>- Improves response time for repeated prompts | Enabled |
| **KV Cache Type** | - KV cache implementation type; controls memory usage and precision trade-off<br></br>- Options:<br></br>• f16 (most stable)<br></br>• q8_0 (balanced)<br></br>• q4_0 (lowest memory) | f16 | | **KV Cache Type** | - KV cache implementation type; controls memory usage and precision trade-off<br></br>- Options:<br></br>• f16 (most stable)<br></br>• q8_0 (balanced)<br></br>• q4_0 (lowest memory) | f16 |
| **mmap** | - Enables memory-mapped model loading<br></br>- Reduces memory usage<br></br>- Recommended for large models | Enabled | | **mmap** | - Enables memory-mapped model loading<br></br>- Reduces memory usage<br></br>- Recommended for large models | Enabled |
| **Context Shift** | - Automatically shifts the context window when the model is unable to process the entire prompt<br/> - Ensures that the most relevant information is always included <br/> - Recommended for long conversations and multiple tool calls | Disabled |
## Best Practices ## Best Practices

2
docs/src/pages/docs/model-parameters.mdx
View File

@ -38,8 +38,6 @@ These settings are available in the model settings modal:
| **Repeat Last N** | Number of tokens to consider for repeat penalty. | | **Repeat Last N** | Number of tokens to consider for repeat penalty. |
| **Repeat Penalty** | Penalize repeating token sequences. | | **Repeat Penalty** | Penalize repeating token sequences. |
| **Presence Penalty**| Penalize alpha presence (encourages new topics). | | **Presence Penalty**| Penalize alpha presence (encourages new topics). |
| **Max Tokens** | Maximum length of the model's response. |
| **Stop Sequences** | Tokens or phrases that will end the model's response. |
| **Frequency Penalty** | Reduces word repetition. | | **Frequency Penalty** | Reduces word repetition. |
<br/> <br/>

47
docs/src/pages/docs/server-examples/continue-dev.mdx
View File

@ -36,11 +36,15 @@ Follow this [guide](https://continue.dev/docs/quickstart) to install the Continu
To set up Continue for use with Jan's Local Server, you must activate the Jan API Server with your chosen model. To set up Continue for use with Jan's Local Server, you must activate the Jan API Server with your chosen model.
1. Press the `<>` button. Jan will take you to the **Local API Server** section. 1. Press the `⚙️ Settings` button.
2. Setup the server, which includes the **IP Port**, **Cross-Origin-Resource-Sharing (CORS)** and **Verbose Server Logs**. 2. Locate `Local API Server`.
3. Press the **Start Server** button 3. Setup the server, which includes the **IP Port**, **Cross-Origin-Resource-Sharing (CORS)** and **Verbose Server Logs**.
4. Include your user-defined API Key.
5. Press the **Start Server** button
### Step 3: Configure Continue to Use Jan's Local Server ### Step 3: Configure Continue to Use Jan's Local Server
@ -64,30 +68,35 @@ To set up Continue for use with Jan's Local Server, you must activate the Jan AP
</Tabs.Tab> </Tabs.Tab>
</Tabs> </Tabs>
```json title="~/.continue/config.json" ```yaml title="~/.continue/config.yaml"
{ name: Local Assistant
"models": [ version: 1.0.0
{ schema: v1
"title": "Jan", models:
"provider": "openai", - name: Jan
"model": "mistral-ins-7b-q4", provider: openai
"apiKey": "EMPTY", model: #MODEL_NAME (e.g. qwen3:0.6b)
"apiBase": "http://localhost:1337/v1" apiKey: #YOUR_USER_DEFINED_API_KEY_HERE (e.g. hello)
} apiBase: http://localhost:1337/v1
] context:
} - provider: code
- provider: docs
- provider: diff
- provider: terminal
- provider: problems
- provider: folder
- provider: codebase
``` ```
2. Ensure the file has the following configurations: 2. Ensure the file has the following configurations:
- Ensure `openai` is selected as the `provider`. - Ensure `openai` is selected as the `provider`.
- Match the `model` with the one enabled in the Jan API Server. - Match the `model` with the one enabled in the Jan API Server.
- Set `apiBase` to `http://localhost:1337`. - Set `apiBase` to `http://localhost:1337/v1`.
- Leave the `apiKey` field to `EMPTY`.
### Step 4: Ensure the Using Model Is Activated in Jan ### Step 4: Ensure the Using Model Is Activated in Jan
1. Navigate to `Settings` > `My Models`. 1. Navigate to `Settings` > `Model Providers`.
2. Click the **three dots (⋮)** button. 2. Under Llama.cpp, find the model that you would want to use.
3. Select the **Start Model** button to activate the model. 3. Select the **Start Model** button to activate the model.
</Steps> </Steps>