Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update run Jan in Docker mode

Browse Source
This commit is contained in:
hieu-jan 2024-02-26 11:37:18 +09:00
parent 9f142848fa
commit d959cbd6f2
1 changed files with 75 additions and 54 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
docs/docs/guides/02-installation/05-docker.md
View File

@ -13,13 +13,13 @@ keywords:
no-subscription fee,
large language model,
docker installation,
cpu mode,
gpu mode,
]
---
# Installing Jan using Docker
## Installation
### Pre-requisites
:::note
@ -37,17 +37,35 @@ sudo sh ./get-docker.sh --dry-run
- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation.
### Instructions
### Run Jan in Docker Mode
- Run Jan in Docker mode
| Docker compose Profile | Description |
| ---------------------- | -------------------------------------------- |
| `cpu-fs` | Run Jan in CPU mode with default file system |
| `cpu-s3fs` | Run Jan in CPU mode with S3 file system |
| `gpu-fs` | Run Jan in GPU mode with default file system |
| `gpu-s3fs` | Run Jan in GPU mode with S3 file system |
- **Option 1**: Run Jan in CPU mode
| Environment Variable | Description |
| ----------------------- | ------------------------------------------------------------------------------------------------------- |
| `S3_BUCKET_NAME` | S3 bucket name - leave blank for default file system |
| `AWS_ACCESS_KEY_ID` | AWS access key ID - leave blank for default file system |
| `AWS_SECRET_ACCESS_KEY` | AWS secret access key - leave blank for default file system |
| `AWS_ENDPOINT` | AWS endpoint URL - leave blank for default file system |
| `AWS_REGION` | AWS region - leave blank for default file system |
| `API_BASE_URL` | Jan Server URL, please modify it as your public ip address or domain name default http://localhost:1377 |
- **Option 1**: Run Jan in CPU mode
```bash
docker compose --profile cpu up -d
# cpu mode with default file system
docker compose --profile cpu-fs up -d
# cpu mode with S3 file system
docker compose --profile cpu-s3fs up -d
```
- **Option 2**: Run Jan in GPU mode
- **Option 2**: Run Jan in GPU mode
- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output
@ -89,14 +107,17 @@ sudo sh ./get-docker.sh --dry-run
- **Step 4**: Run command to start Jan in GPU mode
```bash
# GPU mode
# GPU mode with default file system
docker compose --profile gpu up -d
# GPU mode with S3 file system
docker compose --profile gpu-s3fs up -d
```
This will start the web server and you can access Jan at `http://localhost:3000`.
This will start the web server and you can access Jan at `http://localhost:3000`.
:::warning
- Docker mode is currently only suitable for development and localhost. Production is not supported yet, and the RAG feature is not available in Docker mode.
- RAG feature is not supported in Docker mode with s3fs yet.
:::