Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #742 from janhq/741-chore-adding-correct-example-for-model-parameters-with-examples

Browse Source 
chore: clarification changes to the model settings and model parameters
This commit is contained in:
automaticcat 2023-11-29 16:53:53 +07:00 committed by GitHub
commit 4c2c320105
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 15 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
docs/docs/specs/engineering/models.md
View File

@ -65,17 +65,19 @@ Here's a standard example `model.json` for a GGUF model.
"state": enum[null, "downloading", "ready", "starting", "stopping", ...] "state": enum[null, "downloading", "ready", "starting", "stopping", ...]
"format": "ggufv3", // Defaults to "ggufv3" "format": "ggufv3", // Defaults to "ggufv3"
"settings": { // Models are initialized with settings "settings": { // Models are initialized with settings
"ctx_len": "2048", "ctx_len": 2048,
"ngl": "100", "ngl": 100,
"embedding": "true", "embedding": true,
"n_parallel": "4", "n_parallel": 4,
}, },
"parameters": { // Models are called parameters "parameters": { // Models are called parameters
"temperature": "0.7", "stream": true,
"token_limit": "2048", "max_tokens": 2048,
"top_k": "0", "stop": ["<endofstring>"], // This usually can be left blank, only used with specific need from model author
"top_p": "1", "frequency_penalty": 0,
"stream": "true" "presence_penalty": 0,
"temperature": 0.7,
"top_p": 0.95
}, },
"metadata": {}, // Defaults to {} "metadata": {}, // Defaults to {}
"assets": [ // Defaults to current dir "assets": [ // Defaults to current dir
@ -83,6 +85,10 @@ Here's a standard example `model.json` for a GGUF model.
] ]
``` ```
The model settings in the example can be found at: [Nitro's model settings](https://nitro.jan.ai/features/load-unload#table-of-parameters)
The model parameters in the example can be found at: [Nitro's model parameters](https://nitro.jan.ai/api-reference#tag/Chat-Completion)
## API Reference ## API Reference
Jan's Model API is compatible with [OpenAI's Models API](https://platform.openai.com/docs/api-reference/models), with additional methods for managing and running models locally. Jan's Model API is compatible with [OpenAI's Models API](https://platform.openai.com/docs/api-reference/models), with additional methods for managing and running models locally.