Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #1947 from janhq/docs

Browse Source 
sync current docs branch to dev branch
This commit is contained in:
Hieu 2024-02-07 10:59:04 +07:00 committed by GitHub
commit 97466c71f0
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 322 additions and 304 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
docs/docs/about/01-README.md
View File

@ -110,9 +110,10 @@ Adhering to Jan's privacy preserving philosophy, our analytics philosophy is to
#### What is tracked #### What is tracked
1. By default, Github tracks downloads and device metadata for all public Github repos. This helps us troubleshoot & ensure cross platform support. 1. By default, Github tracks downloads and device metadata for all public GitHub repositories. This helps us troubleshoot & ensure cross-platform support.
1. We use Posthog to track a single `app.opened` event without additional user metadata, in order to understand retention. 2. We use [Umami](https://umami.is/) to collect, analyze, and understand application data while maintaining visitor privacy and data ownership. We are using the Umami Cloud in Europe to ensure GDPR compliance. Please see [Umami Privacy Policy](https://umami.is/privacy) for more details.
1. Additionally, we plan to enable a `Settings` feature for users to turn off all tracking. 3. We use Umami to track a single `app.opened` event without additional user metadata, in order to understand retention. In addition, we track `app.event` to understand app version usage.
4. Additionally, we plan to enable a `Settings` feature for users to turn off all tracking.
#### Request for help #### Request for help

79
docs/docs/developer/01-overview/04-install-and-prerequisites.md Normal file
View File

@ -0,0 +1,79 @@
---
title: Installation and Prerequisites
slug: /developer/prereq
description: Guide to install and setup Jan for development.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
installation,
prerequisites,
developer setup,
]
---
## Requirements
### Hardware Requirements
Ensure your system meets the following specifications to guarantee a smooth development experience:
- [Hardware Requirements](../../guides/02-installation/06-hardware.md)
### System Requirements
Make sure your operating system meets the specific requirements for Jan development:
- [Windows](../../install/windows/#system-requirements)
- [MacOS](../../install/mac/#system-requirements)
- [Linux](../../install/linux/#system-requirements)
## Prerequisites
- [Node.js](https://nodejs.org/en/) (version 20.0.0 or higher)
- [yarn](https://yarnpkg.com/) (version 1.22.0 or higher)
- [make](https://www.gnu.org/software/make/) (version 3.81 or higher)
## Instructions
1. **Clone the Repository:**
```bash
git clone https://github.com/janhq/jan
cd jan
git checkout -b DESIRED_BRANCH
```
2. **Install Dependencies**
```bash
yarn install
```
3. **Run Development and Use Jan Desktop**
```bash
make dev
```
This command starts the development server and opens the Jan Desktop app.
## For Production Build
```bash
# Do steps 1 and 2 in the previous section
# Build the app
make build
```
This will build the app MacOS (M1/M2/M3) for production (with code signing already done) and place the result in `/electron/dist` folder.
## Troubleshooting
If you run into any issues due to a broken build, please check the [Stuck on a Broken Build](../../troubleshooting/stuck-on-broken-build) guide.

5
docs/docs/guides/02-installation/01-mac.md
View File

@ -12,11 +12,16 @@ keywords:
conversational AI, conversational AI,
no-subscription fee, no-subscription fee,
large language model, large language model,
installation guide,
] ]
--- ---
# Installing Jan on MacOS # Installing Jan on MacOS
## System Requirements
Ensure that your MacOS version is 13 or higher to run Jan.
## Installation ## Installation
Jan is available for download via our homepage, [https://jan.ai/](https://jan.ai/). Jan is available for download via our homepage, [https://jan.ai/](https://jan.ai/).

22
docs/docs/guides/02-installation/02-windows.md
View File

@ -12,11 +12,23 @@ keywords:
conversational AI, conversational AI,
no-subscription fee, no-subscription fee,
large language model, large language model,
installation guide,
] ]
--- ---
# Installing Jan on Windows # Installing Jan on Windows
## System Requirements
Ensure that your system meets the following requirements:
- Windows 10 or higher is required to run Jan.
To enable GPU support, you will need:
- NVIDIA GPU with CUDA Toolkit 11.7 or higher
- NVIDIA driver 470.63.01 or higher
## Installation ## Installation
Jan is available for download via our homepage, [https://jan.ai](https://jan.ai/). Jan is available for download via our homepage, [https://jan.ai](https://jan.ai/).
@ -59,13 +71,3 @@ To remove all user data associated with Jan, you can delete the `/jan` directory
cd C:\Users\%USERNAME%\AppData\Roaming cd C:\Users\%USERNAME%\AppData\Roaming
rmdir /S jan rmdir /S jan
``` ```
## Troubleshooting
### Microsoft Defender
**Error: "Microsoft Defender SmartScreen prevented an unrecognized app from starting"**
Windows Defender may display the above warning when running the Jan Installer, as a standard security measure.
To proceed, select the "More info" option and select the "Run Anyway" option to continue with the installation.

14
docs/docs/guides/02-installation/03-linux.md
View File

@ -12,11 +12,24 @@ keywords:
conversational AI, conversational AI,
no-subscription fee, no-subscription fee,
large language model, large language model,
installation guide,
] ]
--- ---
# Installing Jan on Linux # Installing Jan on Linux
## System Requirements
Ensure that your system meets the following requirements:
- glibc 2.27 or higher (check with `ldd --version`)
- gcc 11, g++ 11, cpp 11, or higher, refer to this [link](https://jan.ai/guides/troubleshooting/gpu-not-used/#specific-requirements-for-linux) for more information.
To enable GPU support, you will need:
- NVIDIA GPU with CUDA Toolkit 11.7 or higher
- NVIDIA driver 470.63.01 or higher
## Installation ## Installation
Jan is available for download via our homepage, [https://jan.ai](https://jan.ai/). Jan is available for download via our homepage, [https://jan.ai](https://jan.ai/).
@ -66,7 +79,6 @@ jan-linux-amd64-{version}.deb
# AppImage # AppImage
jan-linux-x86_64-{version}.AppImage jan-linux-x86_64-{version}.AppImage
``` ```
```
## Uninstall Jan ## Uninstall Jan

7
docs/docs/guides/04-using-models/03-integrate-with-remote-server.mdx
View File

@ -65,6 +65,13 @@ Navigate to the `~/jan/models` folder. Create a folder named `gpt-3.5-turbo-16k`
} }
``` ```
:::tip
- You can find the list of available models in the [OpenAI Platform](https://platform.openai.com/docs/models/overview).
- Please note that the `id` property need to match the model name in the list. For example, if you want to use the [GPT-4 Turbo](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo), you need to set the `id` property as `gpt-4-1106-preview`.
:::
### 2. Configure OpenAI API Keys ### 2. Configure OpenAI API Keys
You can find your API keys in the [OpenAI Platform](https://platform.openai.com/api-keys) and set the OpenAI API keys in `~/jan/engines/openai.json` file. You can find your API keys in the [OpenAI Platform](https://platform.openai.com/api-keys) and set the OpenAI API keys in `~/jan/engines/openai.json` file.

4
docs/docs/guides/08-troubleshooting/02-somethings-amiss.mdx
View File

@ -45,7 +45,9 @@ This may occur due to several reasons. Please follow these steps to resolve it:
5. If you are on Nvidia GPUs, please download [Cuda](https://developer.nvidia.com/cuda-downloads). 5. If you are on Nvidia GPUs, please download [Cuda](https://developer.nvidia.com/cuda-downloads).
6. When [checking app logs](https://jan.ai/troubleshooting/how-to-get-error-logs/), if you encounter the error log `Bind address failed at 127.0.0.1:3928`, it indicates that the port used by Nitro might already be in use. Use the following commands to check the port status: 6. If you're using Linux, please ensure that your system meets the following requirements gcc 11, g++ 11, cpp 11, or higher, refer to this [link](https://jan.ai/guides/troubleshooting/gpu-not-used/#specific-requirements-for-linux) for more information.
7. When [checking app logs](https://jan.ai/troubleshooting/how-to-get-error-logs/), if you encounter the error log `Bind address failed at 127.0.0.1:3928`, it indicates that the port used by Nitro might already be in use. Use the following commands to check the port status:
<Tabs groupId="operating-systems"> <Tabs groupId="operating-systems">
<TabItem value="mac" label="macOS"> <TabItem value="mac" label="macOS">

6
docs/docs/guides/08-troubleshooting/06-unexpected-token.mdx
View File

@ -17,4 +17,8 @@ keywords:
] ]
--- ---
1. You may receive an error response `Error occurred: Unexpected token '<', "<!DOCTYPE"...is not valid JSON`, when you start a chat with OpenAI models. Using a VPN may help fix the issue. You may receive an error response `Error occurred: Unexpected token '<', "<!DOCTYPE"...is not valid JSON`, when you start a chat with OpenAI models.
1. Check that you added an OpenAI API key. You can get an API key from OpenAI's [developer platform](https://platform.openai.com/). Alternatively, we recommend you download a local model from Jan Hub, which remains free to use and runs on your own computer!
2. Using a VPN may help fix the issue.

26
docs/docs/guides/08-troubleshooting/07-undefined-issue.mdx Normal file
View File

@ -0,0 +1,26 @@
---
title: Undefined Issue
slug: /troubleshooting/undefined-issue
description: Undefined issue troubleshooting guide.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
undefined issue,
]
---
You may encounter an "undefined" issue when using Jan. Here are some troubleshooting steps to help you resolve the issue.
1. Try wiping the Jan folder and reopening the Jan app and see if the issue persists.
2. If the issue persists, try to go `~/jan/extensions/@janhq/inference-nitro-extensions/dist/bin/<your-os>/nitro` and run the nitro manually and see if you get any error messages.
3. Resolve the error messages you get from the nitro and see if the issue persists.
4. Reopen the Jan app and see if the issue is resolved.
5. If the issue persists, please share with us the [app logs](https://jan.ai/troubleshooting/how-to-get-error-logs/) via [Jan Discord](https://discord.gg/mY69SZaMaC).

43
docs/openapi/jan.yaml
View File

@ -67,20 +67,31 @@ paths:
x-codeSamples: x-codeSamples:
- lang: cURL - lang: cURL
source: | source: |
curl http://localhost:1337/v1/chat/completions \ curl -X 'POST' \
-H "Content-Type: application/json" \ 'http://localhost:1337/v1/chat/completions' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{ -d '{
"model": "tinyllama-1.1b",
"messages": [ "messages": [
{ {
"role": "system", "content": "You are a helpful assistant.",
"content": "You are a helpful assistant." "role": "system"
}, },
{ {
"role": "user", "content": "Hello!",
"content": "Hello!" "role": "user"
} }
] ],
"model": "tinyllama-1.1b",
"stream": true,
"max_tokens": 2048,
"stop": [
"hello"
],
"frequency_penalty": 0,
"presence_penalty": 0,
"temperature": 0.7,
"top_p": 0.95
}' }'
/models: /models:
get: get:
@ -103,7 +114,9 @@ paths:
x-codeSamples: x-codeSamples:
- lang: cURL - lang: cURL
source: | source: |
curl http://localhost:1337/v1/models curl -X 'GET' \
'http://localhost:1337/v1/models' \
-H 'accept: application/json'
"/models/download/{model_id}": "/models/download/{model_id}":
get: get:
operationId: downloadModel operationId: downloadModel
@ -131,7 +144,9 @@ paths:
x-codeSamples: x-codeSamples:
- lang: cURL - lang: cURL
source: | source: |
curl -X POST http://localhost:1337/v1/models/download/{model_id} curl -X 'GET' \
'http://localhost:1337/v1/models/download/{model_id}' \
-H 'accept: application/json'
"/models/{model_id}": "/models/{model_id}":
get: get:
operationId: retrieveModel operationId: retrieveModel
@ -162,7 +177,9 @@ paths:
x-codeSamples: x-codeSamples:
- lang: cURL - lang: cURL
source: | source: |
curl http://localhost:1337/v1/models/{model_id} curl -X 'GET' \
'http://localhost:1337/v1/models/{model_id}' \
-H 'accept: application/json'
delete: delete:
operationId: deleteModel operationId: deleteModel
tags: tags:
@ -191,7 +208,9 @@ paths:
x-codeSamples: x-codeSamples:
- lang: cURL - lang: cURL
source: | source: |
curl -X DELETE http://localhost:1337/v1/models/{model_id} curl -X 'DELETE' \
'http://localhost:1337/v1/models/{model_id}' \
-H 'accept: application/json'
/threads: /threads:
post: post:
operationId: createThread operationId: createThread

2
docs/openapi/specs/assistants.yaml
View File

@ -316,4 +316,4 @@ components:
deleted: deleted:
type: boolean type: boolean
description: Indicates whether the assistant was successfully deleted. description: Indicates whether the assistant was successfully deleted.
example: true example: true

2
docs/openapi/specs/chat.yaml
View File

@ -188,4 +188,4 @@ components:
total_tokens: total_tokens:
type: integer type: integer
example: 533 example: 533
description: Total number of tokens used description: Total number of tokens used

47
docs/openapi/specs/messages.yaml
View File

@ -1,3 +1,4 @@
---
components: components:
schemas: schemas:
MessageObject: MessageObject:
@ -75,7 +76,7 @@ components:
example: msg_abc123 example: msg_abc123
object: object:
type: string type: string
description: "Type of the object, indicating it's a thread message." description: Type of the object, indicating it's a thread message.
default: thread.message default: thread.message
created_at: created_at:
type: integer type: integer
@ -88,7 +89,7 @@ components:
example: thread_abc123 example: thread_abc123
role: role:
type: string type: string
description: "Role of the sender, either 'user' or 'assistant'." description: Role of the sender, either 'user' or 'assistant'.
example: user example: user
content: content:
type: array type: array
@ -97,7 +98,7 @@ components:
properties: properties:
type: type:
type: string type: string
description: "Type of content, e.g., 'text'." description: Type of content, e.g., 'text'.
example: text example: text
text: text:
type: object type: object
@ -110,21 +111,21 @@ components:
type: array type: array
items: items:
type: string type: string
description: "Annotations for the text content, if any." description: Annotations for the text content, if any.
example: [] example: []
file_ids: file_ids:
type: array type: array
items: items:
type: string type: string
description: "Array of file IDs associated with the message, if any." description: Array of file IDs associated with the message, if any.
example: [] example: []
assistant_id: assistant_id:
type: string type: string
description: "Identifier of the assistant involved in the message, if applicable." description: Identifier of the assistant involved in the message, if applicable.
example: null example: null
run_id: run_id:
type: string type: string
description: "Run ID associated with the message, if applicable." description: Run ID associated with the message, if applicable.
example: null example: null
metadata: metadata:
type: object type: object
@ -139,7 +140,7 @@ components:
example: msg_abc123 example: msg_abc123
object: object:
type: string type: string
description: "Type of the object, indicating it's a thread message." description: Type of the object, indicating it's a thread message.
example: thread.message example: thread.message
created_at: created_at:
type: integer type: integer
@ -152,7 +153,7 @@ components:
example: thread_abc123 example: thread_abc123
role: role:
type: string type: string
description: "Role of the sender, either 'user' or 'assistant'." description: Role of the sender, either 'user' or 'assistant'.
example: user example: user
content: content:
type: array type: array
@ -161,7 +162,7 @@ components:
properties: properties:
type: type:
type: string type: string
description: "Type of content, e.g., 'text'." description: Type of content, e.g., 'text'.
example: text example: text
text: text:
type: object type: object
@ -174,21 +175,21 @@ components:
type: array type: array
items: items:
type: string type: string
description: "Annotations for the text content, if any." description: Annotations for the text content, if any.
example: [] example: []
file_ids: file_ids:
type: array type: array
items: items:
type: string type: string
description: "Array of file IDs associated with the message, if any." description: Array of file IDs associated with the message, if any.
example: [] example: []
assistant_id: assistant_id:
type: string type: string
description: "Identifier of the assistant involved in the message, if applicable." description: Identifier of the assistant involved in the message, if applicable.
example: null example: null
run_id: run_id:
type: string type: string
description: "Run ID associated with the message, if applicable." description: Run ID associated with the message, if applicable.
example: null example: null
metadata: metadata:
type: object type: object
@ -199,7 +200,7 @@ components:
properties: properties:
object: object:
type: string type: string
description: "Type of the object, indicating it's a list." description: Type of the object, indicating it's a list.
default: list default: list
data: data:
type: array type: array
@ -226,7 +227,7 @@ components:
example: msg_abc123 example: msg_abc123
object: object:
type: string type: string
description: "Type of the object, indicating it's a thread message." description: Type of the object, indicating it's a thread message.
example: thread.message example: thread.message
created_at: created_at:
type: integer type: integer
@ -239,7 +240,7 @@ components:
example: thread_abc123 example: thread_abc123
role: role:
type: string type: string
description: "Role of the sender, either 'user' or 'assistant'." description: Role of the sender, either 'user' or 'assistant'.
example: user example: user
content: content:
type: array type: array
@ -248,7 +249,7 @@ components:
properties: properties:
type: type:
type: string type: string
description: "Type of content, e.g., 'text'." description: Type of content, e.g., 'text'.
text: text:
type: object type: object
properties: properties:
@ -260,20 +261,20 @@ components:
type: array type: array
items: items:
type: string type: string
description: "Annotations for the text content, if any." description: Annotations for the text content, if any.
file_ids: file_ids:
type: array type: array
items: items:
type: string type: string
description: "Array of file IDs associated with the message, if any." description: Array of file IDs associated with the message, if any.
example: [] example: []
assistant_id: assistant_id:
type: string type: string
description: "Identifier of the assistant involved in the message, if applicable." description: Identifier of the assistant involved in the message, if applicable.
example: null example: null
run_id: run_id:
type: string type: string
description: "Run ID associated with the message, if applicable." description: Run ID associated with the message, if applicable.
example: null example: null
metadata: metadata:
type: object type: object
@ -309,4 +310,4 @@ components:
data: data:
type: array type: array
items: items:
$ref: "#/components/schemas/MessageFileObject" $ref: "#/components/schemas/MessageFileObject"

358
docs/openapi/specs/models.yaml
View File

@ -18,114 +18,82 @@ components:
Model: Model:
type: object type: object
properties: properties:
type: source_url:
type: string type: string
default: model format: uri
description: The type of the object. description: URL to the source of the model.
version: example: https://huggingface.co/janhq/trinity-v1.2-GGUF/resolve/main/trinity-v1.2.Q4_K_M.gguf
type: string
default: "1"
description: The version number of the model.
id: id:
type: string type: string
description: Unique identifier used in chat-completions model_name, matches description:
Unique identifier used in chat-completions model_name, matches
folder name. folder name.
example: zephyr-7b example: trinity-v1.2-7b
object:
type: string
example: model
name: name:
type: string type: string
description: Name of the model. description: Name of the model.
example: Zephyr 7B example: Trinity-v1.2 7B Q4
owned_by: version:
type: string type: string
description: Compatibility field for OpenAI. default: "1.0"
default: "" description: The version number of the model.
created:
type: integer
format: int64
description: Unix timestamp representing the creation time.
description: description:
type: string type: string
description: Description of the model. description: Description of the model.
state: example:
type: string Trinity is an experimental model merge using the Slerp method.
enum: Recommended for daily assistance purposes.
- null
- downloading
- ready
- starting
- stopping
description: Current state of the model.
format: format:
type: string type: string
description: State format of the model, distinct from the engine. description: State format of the model, distinct from the engine.
example: ggufv3 example: gguf
source:
type: array
items:
type: object
properties:
url:
format: uri
description: URL to the source of the model.
example: https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf
filename:
type: string
description: Filename of the model.
example: zephyr-7b-beta.Q4_K_M.gguf
settings: settings:
type: object type: object
properties: properties:
ctx_len: ctx_len:
type: string type: integer
description: Context length. description: Context length.
example: "4096" example: 4096
ngl: prompt_template:
type: string type: string
description: Number of layers. example: "<|im_start|>system\n{system_message}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant"
example: "100"
embedding:
type: string
description: Indicates if embedding is enabled.
example: "true"
n_parallel:
type: string
description: Number of parallel processes.
example: "4"
additionalProperties: false additionalProperties: false
parameters: parameters:
type: object type: object
properties: properties:
temperature: temperature:
type: string example: 0.7
description: Temperature setting for the model.
example: "0.7"
token_limit:
type: string
description: Token limit for the model.
example: "4096"
top_k:
type: string
description: Top-k setting for the model.
example: "0"
top_p: top_p:
type: string example: 0.95
description: Top-p setting for the model.
example: "1"
stream: stream:
type: string example: true
description: Indicates if streaming is enabled. max_tokens:
example: "true" example: 4096
stop:
example: []
frequency_penalty:
example: 0
presence_penalty:
example: 0
additionalProperties: false additionalProperties: false
metadata: metadata:
type: object author:
description: Additional metadata.
assets:
type: array
items:
type: string type: string
description: List of assets related to the model. example: Jan
required: tags:
- source example:
- 7B
- Merged
- Featured
size:
example: 4370000000,
cover:
example: https://raw.githubusercontent.com/janhq/jan/main/models/trinity-v1.2-7b/cover.png
engine:
example: nitro
ModelObject: ModelObject:
type: object type: object
properties: properties:
@ -133,7 +101,7 @@ components:
type: string type: string
description: | description: |
The identifier of the model. The identifier of the model.
example: zephyr-7b example: trinity-v1.2-7b
object: object:
type: string type: string
description: | description: |
@ -153,197 +121,89 @@ components:
GetModelResponse: GetModelResponse:
type: object type: object
properties: properties:
source_url:
type: string
format: uri
description: URL to the source of the model.
example: https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_M.gguf
id: id:
type: string type: string
description: The identifier of the model. description:
example: zephyr-7b Unique identifier used in chat-completions model_name, matches
folder name.
example: mistral-ins-7b-q4
object: object:
type: string type: string
description: Type of the object, indicating it's a model. example: model
default: model name:
created:
type: integer
format: int64
description: Unix timestamp representing the creation time of the model.
owned_by:
type: string type: string
description: The entity that owns the model. description: Name of the model.
example: _ example: Mistral Instruct 7B Q4
state: version:
type: string type: string
enum: default: "1.0"
- not_downloaded description: The version number of the model.
- downloaded description:
- running type: string
- stopped description: Description of the model.
description: The current state of the model. example:
source: Trinity is an experimental model merge using the Slerp method.
type: array Recommended for daily assistance purposes.
items: format:
type: object type: string
properties: description: State format of the model, distinct from the engine.
url: example: gguf
format: uri settings:
description: URL to the source of the model.
example: https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf
filename:
type: string
description: Filename of the model.
example: zephyr-7b-beta.Q4_K_M.gguf
engine_parameters:
type: object
properties:
pre_prompt:
type: string
description: Predefined prompt used for setting up internal configurations.
default: ""
example: Initial setup complete.
system_prompt:
type: string
description: Prefix used for system-level prompts.
default: "SYSTEM: "
user_prompt:
type: string
description: Prefix used for user prompts.
default: "USER: "
ai_prompt:
type: string
description: Prefix used for assistant prompts.
default: "ASSISTANT: "
ngl:
type: integer
description: Number of neural network layers loaded onto the GPU for
acceleration.
minimum: 0
maximum: 100
default: 100
example: 100
ctx_len:
type: integer
description: Context length for model operations, varies based on the specific
model.
minimum: 128
maximum: 4096
default: 4096
example: 4096
n_parallel:
type: integer
description: Number of parallel operations, relevant when continuous batching is
enabled.
minimum: 1
maximum: 10
default: 1
example: 4
cont_batching:
type: boolean
description: Indicates if continuous batching is used for processing.
default: false
example: false
cpu_threads:
type: integer
description: Number of threads allocated for CPU-based inference.
minimum: 1
example: 8
embedding:
type: boolean
description: Indicates if embedding layers are enabled in the model.
default: true
example: true
model_parameters:
type: object type: object
properties: properties:
ctx_len: ctx_len:
type: integer type: integer
description: Maximum context length the model can handle. description: Context length.
minimum: 0
maximum: 4096
default: 4096
example: 4096 example: 4096
ngl: prompt_template:
type: integer type: string
description: Number of layers in the neural network. example: "[INST] {prompt} [/INST]"
minimum: 1 additionalProperties: false
maximum: 100 parameters:
default: 100 type: object
example: 100 properties:
embedding:
type: boolean
description: Indicates if embedding layers are used.
default: true
example: true
n_parallel:
type: integer
description: Number of parallel processes the model can run.
minimum: 1
maximum: 10
default: 1
example: 4
temperature: temperature:
type: number
description: Controls randomness in model's responses. Higher values lead to
more random responses.
minimum: 0
maximum: 2
default: 0.7
example: 0.7 example: 0.7
token_limit:
type: integer
description: Maximum number of tokens the model can generate in a single
response.
minimum: 1
maximum: 4096
default: 4096
example: 4096
top_k:
type: integer
description: Limits the model to consider only the top k most likely next tokens
at each step.
minimum: 0
maximum: 100
default: 0
example: 0
top_p: top_p:
type: number example: 0.95
description: Nucleus sampling parameter. The model considers the smallest set of stream:
tokens whose cumulative probability exceeds the top_p value. example: true
minimum: 0 max_tokens:
maximum: 1 example: 4096
default: 1 stop:
example: 1 example: []
frequency_penalty:
example: 0
presence_penalty:
example: 0
additionalProperties: false
metadata: metadata:
type: object author:
properties: type: string
engine: example: MistralAI
type: string tags:
description: The engine used by the model. example:
enum: - 7B
- nitro - Featured
- openai - Foundation Model
- hf_inference size:
quantization: example: 4370000000,
type: string cover:
description: Quantization parameter of the model. example: https://raw.githubusercontent.com/janhq/jan/main/models/mistral-ins-7b-q4/cover.png
example: Q3_K_L engine:
size: example: nitro
type: string
description: Size of the model.
example: 7B
required:
- id
- object
- created
- owned_by
- state
- source
- parameters
- metadata
DeleteModelResponse: DeleteModelResponse:
type: object type: object
properties: properties:
id: id:
type: string type: string
description: The identifier of the model that was deleted. description: The identifier of the model that was deleted.
example: model-zephyr-7B example: mistral-ins-7b-q4
object: object:
type: string type: string
description: Type of the object, indicating it's a model. description: Type of the object, indicating it's a model.

4
docs/openapi/specs/threads.yaml
View File

@ -142,7 +142,7 @@ components:
example: Jan example: Jan
instructions: instructions:
type: string type: string
description: | description: >
The instruction of assistant, defaults to "Be my grammar corrector" The instruction of assistant, defaults to "Be my grammar corrector"
model: model:
type: object type: object
@ -224,4 +224,4 @@ components:
deleted: deleted:
type: boolean type: boolean
description: Indicates whether the thread was successfully deleted. description: Indicates whether the thread was successfully deleted.
example: true example: true