Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update models.md

Browse Source
This commit is contained in:
tikikun 2023-12-01 17:25:52 +07:00
parent 97fcafe61b
commit 242204df24
1 changed files with 5 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
docs/docs/specs/engineering/models.md
View File

@ -51,9 +51,9 @@ jan/ # Jan root folder
Here's a standard example `model.json` for a GGUF model. Here's a standard example `model.json` for a GGUF model.
- `source_url`: https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/.
```js ```js
{
"id": "zephyr-7b", // Defaults to foldername "id": "zephyr-7b", // Defaults to foldername
"object": "model", // Defaults to "model" "object": "model", // Defaults to "model"
"source_url": "https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf", "source_url": "https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf",
@ -64,8 +64,8 @@ Here's a standard example `model.json` for a GGUF model.
"description": null, // Defaults to null "description": null, // Defaults to null
"state": enum[null, "downloading", "ready", "starting", "stopping", ...] "state": enum[null, "downloading", "ready", "starting", "stopping", ...]
"format": "ggufv3", // Defaults to "ggufv3" "format": "ggufv3", // Defaults to "ggufv3"
"engine_parameters": { // Engine parameters inside model.json can override the value inside the base engine.json "engine_parameters": { // Engine parameters inside model.json can override
"ctx_len": 2048, "ctx_len": 2048, // the value inside the base engine.json
"ngl": 100, "ngl": 100,
"embedding": true, "embedding": true,
"n_parallel": 4, "n_parallel": 4,
@ -83,9 +83,10 @@ Here's a standard example `model.json` for a GGUF model.
"assets": [ // Defaults to current dir "assets": [ // Defaults to current dir
"file://.../zephyr-7b-q4_k_m.bin", "file://.../zephyr-7b-q4_k_m.bin",
] ]
}
``` ```
The model settings in the example can be found at: [Nitro's model settings](https://nitro.jan.ai/features/load-unload#table-of-parameters) The engine parameters in the example can be found at: [Nitro's model settings](https://nitro.jan.ai/features/load-unload#table-of-parameters)
The model parameters in the example can be found at: [Nitro's model parameters](https://nitro.jan.ai/api-reference#tag/Chat-Completion) The model parameters in the example can be found at: [Nitro's model parameters](https://nitro.jan.ai/api-reference#tag/Chat-Completion)