Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'dev' into chore/blogpost-finetuning

Browse Source
This commit is contained in:
Henry 2024-02-26 12:12:46 +09:00 committed by GitHub
commit 7827167b37
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 153 additions and 57 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

129
docs/docs/guides/02-installation/05-docker.md
View File

@ -13,13 +13,13 @@ keywords:
no-subscription fee,
large language model,
docker installation,
cpu mode,
gpu mode,
]
---
# Installing Jan using Docker
## Installation
### Pre-requisites
:::note
@ -37,66 +37,87 @@ sudo sh ./get-docker.sh --dry-run
- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation.
### Instructions
### Run Jan in Docker Mode
- Run Jan in Docker mode
| Docker compose Profile | Description |
| ---------------------- | -------------------------------------------- |
| `cpu-fs` | Run Jan in CPU mode with default file system |
| `cpu-s3fs` | Run Jan in CPU mode with S3 file system |
| `gpu-fs` | Run Jan in GPU mode with default file system |
| `gpu-s3fs` | Run Jan in GPU mode with S3 file system |
- **Option 1**: Run Jan in CPU mode
| Environment Variable | Description |
| ----------------------- | ------------------------------------------------------------------------------------------------------- |
| `S3_BUCKET_NAME` | S3 bucket name - leave blank for default file system |
| `AWS_ACCESS_KEY_ID` | AWS access key ID - leave blank for default file system |
| `AWS_SECRET_ACCESS_KEY` | AWS secret access key - leave blank for default file system |
| `AWS_ENDPOINT` | AWS endpoint URL - leave blank for default file system |
| `AWS_REGION` | AWS region - leave blank for default file system |
| `API_BASE_URL` | Jan Server URL, please modify it as your public ip address or domain name default http://localhost:1377 |
- **Option 1**: Run Jan in CPU mode
```bash
# cpu mode with default file system
docker compose --profile cpu-fs up -d
# cpu mode with S3 file system
docker compose --profile cpu-s3fs up -d
```
- **Option 2**: Run Jan in GPU mode
- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output
```bash
nvidia-smi
# Output
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
| 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
| 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
| 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
```
- **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)
- **Step 4**: Run command to start Jan in GPU mode
```bash
docker compose --profile cpu up -d
# GPU mode with default file system
docker compose --profile gpu up -d
# GPU mode with S3 file system
docker compose --profile gpu-s3fs up -d
```
- **Option 2**: Run Jan in GPU mode
- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output
```bash
nvidia-smi
# Output
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
| 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
| 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
| 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
```
- **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)
- **Step 4**: Run command to start Jan in GPU mode
```bash
# GPU mode
docker compose --profile gpu up -d
```
This will start the web server and you can access Jan at `http://localhost:3000`.
This will start the web server and you can access Jan at `http://localhost:3000`.
:::warning
- Docker mode is currently only suitable for development and localhost. Production is not supported yet, and the RAG feature is not available in Docker mode.
- RAG feature is not supported in Docker mode with s3fs yet.
:::

74
docs/docs/how-we-work/engineering/acknowledgements.md Normal file
View File

@ -0,0 +1,74 @@
---
title: Acknowledgements
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
slug: /acknowledgements
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
acknowledgements,
third-party libraries,
]
---
# Acknowledgements
We would like to express our gratitude to the following third-party libraries that have made the development of Jan possible.
### llama.cpp
```
MIT License
Copyright (c) 2023 Georgi Gerganov
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```
### LangChain
```
The MIT License
Copyright (c) LangChain, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
```

1
docs/sidebars.js
View File

@ -44,6 +44,7 @@ const sidebars = {
items: [
"how-we-work/engineering/ci-cd",
"how-we-work/engineering/qa",
"how-we-work/engineering/acknowledgements",
],
},
"how-we-work/product-design/product-design",

4
web/containers/Layout/BottomBar/SystemMonitor/index.tsx
View File

@ -88,12 +88,12 @@ const SystemMonitor = () => {
<div
ref={setElementExpand}
className={twMerge(
'fixed left-16 bottom-12 bg-white w-[calc(100%-64px)] z-50 border-t border-border flex flex-col flex-shrink-0',
'fixed left-16 bottom-12 bg-background w-[calc(100%-64px)] z-50 border-t border-border flex flex-col flex-shrink-0',
showFullScreen && 'h-[calc(100%-48px)]'
)}
>
<div className="h-12 flex items-center border-b border-border px-4 justify-between flex-shrink-0">
<h6 className="font-medium">Running Models</h6>
<h6 className="font-bold">Running Models</h6>
<div className="flex items-center gap-x-2 unset-drag">
{showFullScreen ? (
<ChevronDown

2
web/containers/Layout/BottomBar/index.tsx
View File

@ -34,7 +34,7 @@ const BottomBar = () => {
const progress = useAtomValue(appDownloadProgress)
return (
<div className="fixed bottom-0 left-16 z-20 flex h-12 w-[calc(100%-64px)] items-center justify-between border-t border-border bg-background/80 px-3">
<div className="fixed bottom-0 left-16 z-50 flex h-12 w-[calc(100%-64px)] items-center justify-between border-t border-border bg-background/80 px-3">
<div className="flex flex-shrink-0 items-center gap-x-2">
<div className="flex items-center space-x-2">
{progress && progress > 0 ? (