updated llama.cpp page
This commit is contained in:
parent
b7d220744c
commit
6627505ce4
@ -26,11 +26,11 @@ keywords:
|
|||||||
import { Tabs } from 'nextra/components'
|
import { Tabs } from 'nextra/components'
|
||||||
import { Callout, Steps } from 'nextra/components'
|
import { Callout, Steps } from 'nextra/components'
|
||||||
|
|
||||||
# llama.cpp (Default)
|
# llama.cpp (Cortex)
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
Jan has a default [C++ inference server](https://github.com/janhq/cortex) built on top of [llama.cpp](https://github.com/ggerganov/llama.cpp). This server provides an OpenAI-compatible API, queues, scaling, and additional features on top of the wide capabilities of `llama.cpp`.
|
Jan has [**Cortex**](https://github.com/janhq/cortex) - a default C++ inference server built on top of [llama.cpp](https://github.com/ggerganov/llama.cpp). This server provides an OpenAI-compatible API, queues, scaling, and additional features on top of the wide capabilities of `llama.cpp`.
|
||||||
|
|
||||||
## llama.cpp Engine
|
## llama.cpp Engine
|
||||||
|
|
||||||
@ -56,14 +56,14 @@ Enable the GPU acceleration option within the Jan application by following the [
|
|||||||
## Step-by-step Guide
|
## Step-by-step Guide
|
||||||
<Steps>
|
<Steps>
|
||||||
### Step 1: Open the `model.json`
|
### Step 1: Open the `model.json`
|
||||||
1. Navigate to the **Advanced Settings**.
|
1. Open [Jan Data Folder](/docs/data-folder#open-jan-data-folder)
|
||||||
|
|
||||||
2. On the **Jan Data Folder** click the **folder icon (📂)** to access the data.
|
|
||||||
<br/>
|
<br/>
|
||||||

|

|
||||||
<br/>
|
<br/>
|
||||||
3. Select **models** folder > Click the **name** of the model folder that you want to modify > click the `model.json`.
|
|
||||||
4. This will open up a `model.json`. For example, the `model.json` of `TinyLlama Chat 1.1B Q4` is shown below:
|
2. Select **models** folder > Click **model folder** that you want to modify > click `model.json`
|
||||||
|
3. Once open, `model.json` file looks like below, use model "TinyLlama Chat 1.1B Q4" as an example:
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"sources": [
|
"sources": [
|
||||||
|
|||||||
@ -21,6 +21,8 @@ keywords:
|
|||||||
|
|
||||||
import { Tabs } from 'nextra/components'
|
import { Tabs } from 'nextra/components'
|
||||||
import { Callout, Steps } from 'nextra/components'
|
import { Callout, Steps } from 'nextra/components'
|
||||||
|
import { Settings, FolderOpen } from 'lucide-react'
|
||||||
|
|
||||||
|
|
||||||
# Jan Data Folder
|
# Jan Data Folder
|
||||||
Jan stores your data locally in your own filesystem in a universal file format (JSON). We build for privacy by default and do not collect or sell your data.
|
Jan stores your data locally in your own filesystem in a universal file format (JSON). We build for privacy by default and do not collect or sell your data.
|
||||||
@ -30,9 +32,11 @@ This guide helps you understand where and how this data is stored.
|
|||||||
## Open Jan Data Folder
|
## Open Jan Data Folder
|
||||||
|
|
||||||
To open from Jan's interface:
|
To open from Jan's interface:
|
||||||
1. In Jan, navigate to **Settings** > **Advanced Settings**
|
1. In Jan, navigate to **Settings** (<Settings width={16} height={16} style={{display:"inline"}}/>) > **Advanced Settings**
|
||||||
2. Open **Jan Data Folder**
|
2. Click <FolderOpen width={16} height={16} style={{display:"inline"}}/> icon to open Jan Data Folder
|
||||||
|
<br/>
|
||||||
|

|
||||||
|
<br/>
|
||||||
To open through Terminal:
|
To open through Terminal:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@ -127,7 +127,7 @@ Keep in mind that this build might crash frequently or contain bugs!
|
|||||||
</Tabs.Tab>
|
</Tabs.Tab>
|
||||||
<Tabs.Tab>
|
<Tabs.Tab>
|
||||||
|
|
||||||
Nightly releases are for internal team to test new feature builds everyday, which is very be buggy:
|
Nightly releases are for internal team to test new feature builds everyday, which is very buggy:
|
||||||
|
|
||||||
- Download Jan's Nightly Version on **Ubuntu**: [jan.deb](https://app.jan.ai/download/nightly/linux-amd64-deb)
|
- Download Jan's Nightly Version on **Ubuntu**: [jan.deb](https://app.jan.ai/download/nightly/linux-amd64-deb)
|
||||||
- Download Jan's Nightly Version on **Fedora**: [jan.AppImage](https://app.jan.ai/download/nightly/linux-amd64-appimage)
|
- Download Jan's Nightly Version on **Fedora**: [jan.AppImage](https://app.jan.ai/download/nightly/linux-amd64-appimage)
|
||||||
|
|||||||
@ -244,7 +244,7 @@ Jan stores your data locally in your own filesystem in a universal file format.
|
|||||||
|
|
||||||
**1. Open Jan Data Folder**
|
**1. Open Jan Data Folder**
|
||||||
|
|
||||||
At **Settigns** > **Jan Data Folder**, click <FolderOpen width={16} height={16} style={{display:"inline"}}/> icon to open Jan application's folder:
|
At **Jan Data Folder**, click <FolderOpen width={16} height={16} style={{display:"inline"}}/> icon to open Jan application's folder:
|
||||||
<br/>
|
<br/>
|
||||||

|

|
||||||
<br/>
|
<br/>
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user