docs: improve quickstart docs
This commit is contained in:
0xSage
parent
2b0048ff51
commit
5e460c2a84
@ -1,5 +0,0 @@
|
||||
---
|
||||
title: Model Management
|
||||
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
|
||||
keywords: [Jan AI, Jan, ChatGPT alternative, local AI, private AI, conversational AI, no-subscription fee, large language model ]
|
||||
---
|
||||
159
docs/docs/guides/models.mdx
Normal file
159
docs/docs/guides/models.mdx
Normal file
@ -0,0 +1,159 @@
|
||||
---
|
||||
title: Model Management
|
||||
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
|
||||
keywords:
|
||||
[
|
||||
Jan AI,
|
||||
Jan,
|
||||
ChatGPT alternative,
|
||||
local AI,
|
||||
private AI,
|
||||
conversational AI,
|
||||
no-subscription fee,
|
||||
large language model,
|
||||
]
|
||||
---
|
||||
|
||||
{/* Imports */}
|
||||
import Tabs from "@theme/Tabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
|
||||
Jan is compatible with all GGUF models.
|
||||
|
||||
If you don't see the model you want in the Hub, or if you have a custom model, you can add it to Jan.
|
||||
|
||||
In this guide we will use our latest model, [Trinity](https://huggingface.co/janhq/trinity-v1-GGUF), as an example.
|
||||
|
||||
> We are fast shipping a UI to make this easier, but it's a bit manual for now. Apologies.
|
||||
|
||||
## 1. Create a model folder
|
||||
|
||||
Navigate to the `~/jan/models` folder on your computer.
|
||||
|
||||
In `App Settings`, go to `Advanced`, then `Open App Directory`.
|
||||
|
||||
<Tabs groupId="operating-systems">
|
||||
<TabItem value="mac" label="macOS">
|
||||
|
||||
```sh
|
||||
cd ~/jan/models
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="win" label="Windows">
|
||||
|
||||
```sh
|
||||
C:/Users/<your_user_name>/jan/models
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="linux" label="Linux">
|
||||
|
||||
```sh
|
||||
cd ~/jan/models
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
In the `models` folder, create a folder with the name of the model.
|
||||
|
||||
<Tabs groupId="operating-systems">
|
||||
<TabItem value="mac" label="macOS">
|
||||
|
||||
```sh
|
||||
mkdir trinity-v1-7b
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="win" label="Windows">
|
||||
|
||||
```sh
|
||||
mkdir trinity-v1-7b
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="linux" label="Linux">
|
||||
|
||||
```sh
|
||||
mkdir trinity-v1-7b
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## 2. Create a model JSON
|
||||
|
||||
Jan follows a folder-based, [standard model template](/specs/models) called a `model.json` to persist the model configurations on your local filesystem.
|
||||
|
||||
This means you can easily & transparently reconfigure your models and export and share your preferences.
|
||||
|
||||
<Tabs groupId="operating-systems">
|
||||
<TabItem value="mac" label="macOS">
|
||||
|
||||
```sh
|
||||
cd trinity-v1-7b
|
||||
touch model.json
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="win" label="Windows">
|
||||
|
||||
```sh
|
||||
cd trinity-v1-7b
|
||||
touch model.json
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="linux" label="Linux">
|
||||
|
||||
```sh
|
||||
cd trinity-v1-7b
|
||||
touch model.json
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
Copy the following configurations into the `model.json`.
|
||||
|
||||
1. Make sure the `id` property is the same as the folder name you created.
|
||||
2. Make sure the `source_url` property is the direct binary download link ending in `.gguf`. In HuggingFace, you can find the directl links in `Files and versions` tab.
|
||||
3. Ensure you are using the correct `prompt_template`. This is usually provided in the HuggingFace model's description page.
|
||||
|
||||
```js
|
||||
{
|
||||
"source_url": "https://huggingface.co/janhq/trinity-v1-GGUF/resolve/main/trinity-v1.Q4_K_M.gguf",
|
||||
"id": "trinity-v1-7b",
|
||||
"object": "model",
|
||||
"name": "Trinity 7B Q4",
|
||||
"version": "1.0",
|
||||
"description": "Trinity is an experimental model merge of GreenNodeLM & LeoScorpius using the Slerp method. Recommended for daily assistance purposes.",
|
||||
"format": "gguf",
|
||||
"settings": {
|
||||
"ctx_len": 2048,
|
||||
"prompt_template": "<|im_start|>system\n{system_message}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant"
|
||||
},
|
||||
"parameters": {
|
||||
"max_tokens": 2048
|
||||
},
|
||||
"metadata": {
|
||||
"author": "Jan",
|
||||
"tags": ["7B", "Merged", "Featured"],
|
||||
"size": 4370000000
|
||||
},
|
||||
"engine": "nitro"
|
||||
}
|
||||
```
|
||||
|
||||
## 3. Download your model
|
||||
|
||||
Restart the Jan application and look for your model in the Hub.
|
||||
|
||||
Click the green `download` button to download your actual model binary. This pulls from the `source_url` you provided above.
|
||||
|
||||
|
||||
|
||||
There you go! You are ready to use your model.
|
||||
|
||||
If you have any questions or want to request for more preconfigured GGUF models, please message us in [Discord](https://discord.gg/Dt7MxDyNNZ).
|
||||
@ -1,97 +0,0 @@
|
||||
---
|
||||
title: Quickstart
|
||||
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
|
||||
keywords:
|
||||
[
|
||||
Jan AI,
|
||||
Jan,
|
||||
ChatGPT alternative,
|
||||
local AI,
|
||||
private AI,
|
||||
conversational AI,
|
||||
no-subscription fee,
|
||||
large language model,
|
||||
]
|
||||
---
|
||||
|
||||
Jan is compatible with all GGUF models.
|
||||
|
||||
In this guide we will use our latest model, [Trinity](https://huggingface.co/janhq/trinity-v1-GGUF), as an example.
|
||||
|
||||
## 1. Create a model folder
|
||||
|
||||
Navigate to `~/jan/models` folder on your computer.
|
||||
|
||||
In `App Settings`, go to `Advanced`, then `Open App Directory`.
|
||||
|
||||
Or, you can directly cd into:
|
||||
|
||||
```sh
|
||||
# Windows
|
||||
C:/Users/<your_user_name>/jan/models
|
||||
|
||||
# MacOS/Linux
|
||||
jan/models
|
||||
```
|
||||
|
||||
In the `models` folder, create a folder with the name of the model.
|
||||
|
||||
```sh
|
||||
mkdir trinity-v1-7b
|
||||
```
|
||||
|
||||
## 2. Create a model JSON
|
||||
|
||||
Jan follows a folder-based, [standard model template](/specs/models) called a `model.json`. This allows for easy model configurations, exporting, and sharing.
|
||||
|
||||
```sh
|
||||
cd trinity-v1-7b
|
||||
touch model.json
|
||||
```
|
||||
|
||||
Copy the following into the `model.json`
|
||||
|
||||
```js
|
||||
{
|
||||
"source_url": "https://huggingface.co/janhq/trinity-v1-GGUF/resolve/main/trinity-v1.Q4_K_M.gguf",
|
||||
"id": "trinity-v1-7b",
|
||||
"object": "model",
|
||||
"name": "Trinity 7B Q4",
|
||||
"version": "1.0",
|
||||
"description": "Trinity is an experimental model merge of GreenNodeLM & LeoScorpius using the Slerp method. Recommended for daily assistance purposes.",
|
||||
"format": "gguf",
|
||||
"settings": {
|
||||
"ctx_len": 2048,
|
||||
"prompt_template": "<|im_start|>system\n{system_message}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant"
|
||||
},
|
||||
"parameters": {
|
||||
"max_tokens": 2048
|
||||
},
|
||||
"metadata": {
|
||||
"author": "Jan",
|
||||
"tags": ["7B", "Merged", "Featured"],
|
||||
"size": 4370000000
|
||||
},
|
||||
"engine": "nitro"
|
||||
}
|
||||
```
|
||||
|
||||
:::caution
|
||||
Ensure the `source_url` property is the direct binary download link ending in `.gguf`. Find the links in Huggingface > `Files and versions`.
|
||||
|
||||
Ensure the `id` property is the model foldername
|
||||
|
||||
Ensure you are using the correct `prompt_template`
|
||||
:::
|
||||
|
||||
## 3. Download your model binary
|
||||
|
||||
Restart the Jan application and look for your model in the Hub.
|
||||
|
||||
Click the green `download` button to download your actual model binary from the URL you provided.
|
||||
|
||||
|
||||
|
||||
There you go! You are ready to use your model.
|
||||
|
||||
If you have any questions or want to request for more preconfigured GGUF models, please message us in [Discord](https://discord.gg/Dt7MxDyNNZ).
|
||||
81
docs/docs/guides/quickstart.mdx
Normal file
81
docs/docs/guides/quickstart.mdx
Normal file
@ -0,0 +1,81 @@
|
||||
---
|
||||
title: Quickstart
|
||||
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
|
||||
keywords:
|
||||
[
|
||||
Jan AI,
|
||||
Jan,
|
||||
ChatGPT alternative,
|
||||
local AI,
|
||||
private AI,
|
||||
conversational AI,
|
||||
no-subscription fee,
|
||||
large language model,
|
||||
]
|
||||
---
|
||||
|
||||
import Tabs from "@theme/Tabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
|
||||
In this quickstart we'll show you how to:
|
||||
|
||||
- Download the Jan Desktop client - Mac, Windows, Linux, (and toaster) compatible
|
||||
- Download and customize models
|
||||
- Import custom models
|
||||
- Use the local server at port `1337`
|
||||
|
||||
## Setup
|
||||
|
||||
### Installation
|
||||
|
||||
- To download the latest stable release: https://jan.ai/
|
||||
|
||||
- To download a nightly release (highly unstable but lots of new features): https://github.com/janhq/jan/releases
|
||||
|
||||
- For a detailed installation guide for your operating system, see the following:
|
||||
|
||||
<Tabs groupId="operating-systems">
|
||||
<TabItem value="mac" label="macOS">
|
||||
[Mac installation guide](/install/mac)
|
||||
</TabItem>
|
||||
<TabItem value="win" label="Windows">
|
||||
[Windows installation guide](/install/windows)
|
||||
</TabItem>
|
||||
<TabItem value="linux" label="Linux">
|
||||
[Linux installation guide](/install/linux)
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
- To build Jan Desktop from scratch (and have the right to tinker!)
|
||||
|
||||
See the [Build from Source](/install/from-source) guide.
|
||||
|
||||
### Working with Models
|
||||
|
||||
Jan provides a list of recommended models to get you started.
|
||||
You can find them in the in-app Hub.
|
||||
|
||||
1. `cmd + k` and type "hub" to open the Hub.
|
||||
2. Download your preferred models.
|
||||
3. `cmd + k` and type "chat" to open the conversation UI and start chatting.
|
||||
4. Your model may take a few seconds to start up.
|
||||
5. You can customize the model settings, at each conversation thread level, on the right panel.
|
||||
6. To change model defaults globally, edit the `model.json` file. See the [Models](/guides/models) guide.
|
||||
|
||||
### Importing Models
|
||||
|
||||
Jan is compatible with all GGUF models.
|
||||
|
||||
For more information on how to import custom models, not found in the Hub, see the [Models](/guides/models) guide.
|
||||
|
||||
## Working with the Local Server
|
||||
|
||||
> This feature is currently under development. So expect bugs!
|
||||
|
||||
Jan runs a local server on port `1337` by default.
|
||||
|
||||
The endpoints are OpenAI compatible.
|
||||
|
||||
See the [API server guide](/guides/server) for more information.
|
||||
|
||||
## Next Steps
|
||||
Loading…
x
Reference in New Issue
Block a user