Merge branch 'dev' into chore/blogpost-finetuning
This commit is contained in:
Henry
GitHub
commit
7827167b37
@ -13,13 +13,13 @@ keywords:
|
|||||||
no-subscription fee,
|
no-subscription fee,
|
||||||
large language model,
|
large language model,
|
||||||
docker installation,
|
docker installation,
|
||||||
|
cpu mode,
|
||||||
|
gpu mode,
|
||||||
]
|
]
|
||||||
---
|
---
|
||||||
|
|
||||||
# Installing Jan using Docker
|
# Installing Jan using Docker
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
### Pre-requisites
|
### Pre-requisites
|
||||||
|
|
||||||
:::note
|
:::note
|
||||||
@ -37,66 +37,87 @@ sudo sh ./get-docker.sh --dry-run
|
|||||||
|
|
||||||
- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation.
|
- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation.
|
||||||
|
|
||||||
### Instructions
|
### Run Jan in Docker Mode
|
||||||
|
|
||||||
- Run Jan in Docker mode
|
| Docker compose Profile | Description |
|
||||||
|
| ---------------------- | -------------------------------------------- |
|
||||||
|
| `cpu-fs` | Run Jan in CPU mode with default file system |
|
||||||
|
| `cpu-s3fs` | Run Jan in CPU mode with S3 file system |
|
||||||
|
| `gpu-fs` | Run Jan in GPU mode with default file system |
|
||||||
|
| `gpu-s3fs` | Run Jan in GPU mode with S3 file system |
|
||||||
|
|
||||||
- **Option 1**: Run Jan in CPU mode
|
| Environment Variable | Description |
|
||||||
|
| ----------------------- | ------------------------------------------------------------------------------------------------------- |
|
||||||
|
| `S3_BUCKET_NAME` | S3 bucket name - leave blank for default file system |
|
||||||
|
| `AWS_ACCESS_KEY_ID` | AWS access key ID - leave blank for default file system |
|
||||||
|
| `AWS_SECRET_ACCESS_KEY` | AWS secret access key - leave blank for default file system |
|
||||||
|
| `AWS_ENDPOINT` | AWS endpoint URL - leave blank for default file system |
|
||||||
|
| `AWS_REGION` | AWS region - leave blank for default file system |
|
||||||
|
| `API_BASE_URL` | Jan Server URL, please modify it as your public ip address or domain name default http://localhost:1377 |
|
||||||
|
|
||||||
|
- **Option 1**: Run Jan in CPU mode
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# cpu mode with default file system
|
||||||
|
docker compose --profile cpu-fs up -d
|
||||||
|
|
||||||
|
# cpu mode with S3 file system
|
||||||
|
docker compose --profile cpu-s3fs up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Option 2**: Run Jan in GPU mode
|
||||||
|
|
||||||
|
- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nvidia-smi
|
||||||
|
|
||||||
|
# Output
|
||||||
|
+---------------------------------------------------------------------------------------+
|
||||||
|
| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
|
||||||
|
|-----------------------------------------+----------------------+----------------------+
|
||||||
|
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
|
||||||
|
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|
||||||
|
| | | MIG M. |
|
||||||
|
|=========================================+======================+======================|
|
||||||
|
| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
|
||||||
|
| 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
|
||||||
|
| | | N/A |
|
||||||
|
+-----------------------------------------+----------------------+----------------------+
|
||||||
|
| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
|
||||||
|
| 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
|
||||||
|
| | | N/A |
|
||||||
|
+-----------------------------------------+----------------------+----------------------+
|
||||||
|
| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
|
||||||
|
| 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
|
||||||
|
| | | N/A |
|
||||||
|
+-----------------------------------------+----------------------+----------------------+
|
||||||
|
|
||||||
|
+---------------------------------------------------------------------------------------+
|
||||||
|
| Processes: |
|
||||||
|
| GPU GI CI PID Type Process name GPU Memory |
|
||||||
|
| ID ID Usage |
|
||||||
|
|=======================================================================================|
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
|
||||||
|
|
||||||
|
- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)
|
||||||
|
|
||||||
|
- **Step 4**: Run command to start Jan in GPU mode
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose --profile cpu up -d
|
# GPU mode with default file system
|
||||||
|
docker compose --profile gpu up -d
|
||||||
|
|
||||||
|
# GPU mode with S3 file system
|
||||||
|
docker compose --profile gpu-s3fs up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
- **Option 2**: Run Jan in GPU mode
|
This will start the web server and you can access Jan at `http://localhost:3000`.
|
||||||
|
|
||||||
- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output
|
|
||||||
|
|
||||||
```bash
|
|
||||||
nvidia-smi
|
|
||||||
|
|
||||||
# Output
|
|
||||||
+---------------------------------------------------------------------------------------+
|
|
||||||
| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
|
|
||||||
|-----------------------------------------+----------------------+----------------------+
|
|
||||||
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
|
|
||||||
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|
|
||||||
| | | MIG M. |
|
|
||||||
|=========================================+======================+======================|
|
|
||||||
| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
|
|
||||||
| 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
|
|
||||||
| | | N/A |
|
|
||||||
+-----------------------------------------+----------------------+----------------------+
|
|
||||||
| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
|
|
||||||
| 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
|
|
||||||
| | | N/A |
|
|
||||||
+-----------------------------------------+----------------------+----------------------+
|
|
||||||
| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
|
|
||||||
| 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
|
|
||||||
| | | N/A |
|
|
||||||
+-----------------------------------------+----------------------+----------------------+
|
|
||||||
|
|
||||||
+---------------------------------------------------------------------------------------+
|
|
||||||
| Processes: |
|
|
||||||
| GPU GI CI PID Type Process name GPU Memory |
|
|
||||||
| ID ID Usage |
|
|
||||||
|=======================================================================================|
|
|
||||||
```
|
|
||||||
|
|
||||||
- **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
|
|
||||||
|
|
||||||
- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)
|
|
||||||
|
|
||||||
- **Step 4**: Run command to start Jan in GPU mode
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# GPU mode
|
|
||||||
docker compose --profile gpu up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
This will start the web server and you can access Jan at `http://localhost:3000`.
|
|
||||||
|
|
||||||
:::warning
|
:::warning
|
||||||
|
|
||||||
- Docker mode is currently only suitable for development and localhost. Production is not supported yet, and the RAG feature is not available in Docker mode.
|
- RAG feature is not supported in Docker mode with s3fs yet.
|
||||||
|
|
||||||
:::
|
:::
|
||||||
|
|||||||
74
docs/docs/how-we-work/engineering/acknowledgements.md
Normal file
74
docs/docs/how-we-work/engineering/acknowledgements.md
Normal file
@ -0,0 +1,74 @@
|
|||||||
|
---
|
||||||
|
title: Acknowledgements
|
||||||
|
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
|
||||||
|
slug: /acknowledgements
|
||||||
|
keywords:
|
||||||
|
[
|
||||||
|
Jan AI,
|
||||||
|
Jan,
|
||||||
|
ChatGPT alternative,
|
||||||
|
local AI,
|
||||||
|
private AI,
|
||||||
|
conversational AI,
|
||||||
|
no-subscription fee,
|
||||||
|
large language model,
|
||||||
|
acknowledgements,
|
||||||
|
third-party libraries,
|
||||||
|
]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Acknowledgements
|
||||||
|
|
||||||
|
We would like to express our gratitude to the following third-party libraries that have made the development of Jan possible.
|
||||||
|
|
||||||
|
### llama.cpp
|
||||||
|
|
||||||
|
```
|
||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2023 Georgi Gerganov
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
|
```
|
||||||
|
|
||||||
|
### LangChain
|
||||||
|
|
||||||
|
```
|
||||||
|
The MIT License
|
||||||
|
|
||||||
|
Copyright (c) LangChain, Inc.
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in
|
||||||
|
all copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||||
|
THE SOFTWARE.
|
||||||
|
```
|
||||||
@ -44,6 +44,7 @@ const sidebars = {
|
|||||||
items: [
|
items: [
|
||||||
"how-we-work/engineering/ci-cd",
|
"how-we-work/engineering/ci-cd",
|
||||||
"how-we-work/engineering/qa",
|
"how-we-work/engineering/qa",
|
||||||
|
"how-we-work/engineering/acknowledgements",
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
"how-we-work/product-design/product-design",
|
"how-we-work/product-design/product-design",
|
||||||
|
|||||||
@ -88,12 +88,12 @@ const SystemMonitor = () => {
|
|||||||
<div
|
<div
|
||||||
ref={setElementExpand}
|
ref={setElementExpand}
|
||||||
className={twMerge(
|
className={twMerge(
|
||||||
'fixed left-16 bottom-12 bg-white w-[calc(100%-64px)] z-50 border-t border-border flex flex-col flex-shrink-0',
|
'fixed left-16 bottom-12 bg-background w-[calc(100%-64px)] z-50 border-t border-border flex flex-col flex-shrink-0',
|
||||||
showFullScreen && 'h-[calc(100%-48px)]'
|
showFullScreen && 'h-[calc(100%-48px)]'
|
||||||
)}
|
)}
|
||||||
>
|
>
|
||||||
<div className="h-12 flex items-center border-b border-border px-4 justify-between flex-shrink-0">
|
<div className="h-12 flex items-center border-b border-border px-4 justify-between flex-shrink-0">
|
||||||
<h6 className="font-medium">Running Models</h6>
|
<h6 className="font-bold">Running Models</h6>
|
||||||
<div className="flex items-center gap-x-2 unset-drag">
|
<div className="flex items-center gap-x-2 unset-drag">
|
||||||
{showFullScreen ? (
|
{showFullScreen ? (
|
||||||
<ChevronDown
|
<ChevronDown
|
||||||
|
|||||||
@ -34,7 +34,7 @@ const BottomBar = () => {
|
|||||||
const progress = useAtomValue(appDownloadProgress)
|
const progress = useAtomValue(appDownloadProgress)
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="fixed bottom-0 left-16 z-20 flex h-12 w-[calc(100%-64px)] items-center justify-between border-t border-border bg-background/80 px-3">
|
<div className="fixed bottom-0 left-16 z-50 flex h-12 w-[calc(100%-64px)] items-center justify-between border-t border-border bg-background/80 px-3">
|
||||||
<div className="flex flex-shrink-0 items-center gap-x-2">
|
<div className="flex flex-shrink-0 items-center gap-x-2">
|
||||||
<div className="flex items-center space-x-2">
|
<div className="flex items-center space-x-2">
|
||||||
{progress && progress > 0 ? (
|
{progress && progress > 0 ? (
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user