Merge pull request #1981 from janhq/docs-add-docker-installation
docs: add Jan installation using Docker
This commit is contained in:
Henry
GitHub
commit
fd78631b04
27
README.md
27
README.md
@ -167,6 +167,7 @@ To reset your installation:
|
|||||||
- Clear Application cache in `~/Library/Caches/jan`
|
- Clear Application cache in `~/Library/Caches/jan`
|
||||||
|
|
||||||
## Requirements for running Jan
|
## Requirements for running Jan
|
||||||
|
|
||||||
- MacOS: 13 or higher
|
- MacOS: 13 or higher
|
||||||
- Windows:
|
- Windows:
|
||||||
- Windows 10 or higher
|
- Windows 10 or higher
|
||||||
@ -194,17 +195,17 @@ Contributions are welcome! Please read the [CONTRIBUTING.md](CONTRIBUTING.md) fi
|
|||||||
|
|
||||||
1. **Clone the repository and prepare:**
|
1. **Clone the repository and prepare:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/janhq/jan
|
git clone https://github.com/janhq/jan
|
||||||
cd jan
|
cd jan
|
||||||
git checkout -b DESIRED_BRANCH
|
git checkout -b DESIRED_BRANCH
|
||||||
```
|
```
|
||||||
|
|
||||||
2. **Run development and use Jan Desktop**
|
2. **Run development and use Jan Desktop**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
make dev
|
make dev
|
||||||
```
|
```
|
||||||
|
|
||||||
This will start the development server and open the desktop app.
|
This will start the development server and open the desktop app.
|
||||||
|
|
||||||
@ -222,14 +223,15 @@ This will build the app MacOS m1/m2 for production (with code signing already do
|
|||||||
|
|
||||||
- Supported OS: Linux, WSL2 Docker
|
- Supported OS: Linux, WSL2 Docker
|
||||||
- Pre-requisites:
|
- Pre-requisites:
|
||||||
- `docker` and `docker compose`, follow instruction [here](https://docs.docker.com/engine/install/ubuntu/)
|
|
||||||
|
- Docker Engine and Docker Compose are required to run Jan in Docker mode. Follow the [instructions](https://docs.docker.com/engine/install/ubuntu/) below to get started with Docker Engine on Ubuntu.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -fsSL https://get.docker.com -o get-docker.sh
|
curl -fsSL https://get.docker.com -o get-docker.sh
|
||||||
sudo sh ./get-docker.sh --dry-run
|
sudo sh ./get-docker.sh --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
- `nvidia-driver` and `nvidia-docker2`, follow instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) (If you want to run with GPU mode)
|
- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation.
|
||||||
|
|
||||||
- Run Jan in Docker mode
|
- Run Jan in Docker mode
|
||||||
|
|
||||||
@ -241,7 +243,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do
|
|||||||
|
|
||||||
- **Option 2**: Run Jan in GPU mode
|
- **Option 2**: Run Jan in GPU mode
|
||||||
|
|
||||||
- **Step 1**: Check cuda compatibility with your nvidia driver by running `nvidia-smi` and check the cuda version in the output
|
- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
nvidia-smi
|
nvidia-smi
|
||||||
@ -274,7 +276,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do
|
|||||||
|=======================================================================================|
|
|=======================================================================================|
|
||||||
```
|
```
|
||||||
|
|
||||||
- **Step 2**: Go to https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags and find the smallest minor version of image tag that matches the cuda version from the output of `nvidia-smi` (e.g. 12.1 -> 12.1.0)
|
- **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
|
||||||
|
|
||||||
- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)
|
- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)
|
||||||
|
|
||||||
@ -286,6 +288,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do
|
|||||||
```
|
```
|
||||||
|
|
||||||
This will start the web server and you can access Jan at `http://localhost:3000`.
|
This will start the web server and you can access Jan at `http://localhost:3000`.
|
||||||
|
|
||||||
> Note: Currently, Docker mode is only work for development and localhost, production is not supported yet. RAG feature is not supported in Docker mode yet.
|
> Note: Currently, Docker mode is only work for development and localhost, production is not supported yet. RAG feature is not supported in Docker mode yet.
|
||||||
|
|
||||||
## Acknowledgements
|
## Acknowledgements
|
||||||
|
|||||||
102
docs/docs/guides/02-installation/05-docker.md
Normal file
102
docs/docs/guides/02-installation/05-docker.md
Normal file
@ -0,0 +1,102 @@
|
|||||||
|
---
|
||||||
|
title: Docker
|
||||||
|
slug: /install/docker
|
||||||
|
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
|
||||||
|
keywords:
|
||||||
|
[
|
||||||
|
Jan AI,
|
||||||
|
Jan,
|
||||||
|
ChatGPT alternative,
|
||||||
|
local AI,
|
||||||
|
private AI,
|
||||||
|
conversational AI,
|
||||||
|
no-subscription fee,
|
||||||
|
large language model,
|
||||||
|
docker installation,
|
||||||
|
]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Installing Jan using Docker
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
### Pre-requisites
|
||||||
|
|
||||||
|
:::note
|
||||||
|
|
||||||
|
**Supported OS**: Linux, WSL2 Docker
|
||||||
|
|
||||||
|
:::
|
||||||
|
|
||||||
|
- Docker Engine and Docker Compose are required to run Jan in Docker mode. Follow the [instructions](https://docs.docker.com/engine/install/ubuntu/) below to get started with Docker Engine on Ubuntu.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsSL https://get.docker.com -o get-docker.sh
|
||||||
|
sudo sh ./get-docker.sh --dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation.
|
||||||
|
|
||||||
|
### Instructions
|
||||||
|
|
||||||
|
- Run Jan in Docker mode
|
||||||
|
|
||||||
|
- **Option 1**: Run Jan in CPU mode
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --profile cpu up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Option 2**: Run Jan in GPU mode
|
||||||
|
|
||||||
|
- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nvidia-smi
|
||||||
|
|
||||||
|
# Output
|
||||||
|
+---------------------------------------------------------------------------------------+
|
||||||
|
| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
|
||||||
|
|-----------------------------------------+----------------------+----------------------+
|
||||||
|
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
|
||||||
|
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|
||||||
|
| | | MIG M. |
|
||||||
|
|=========================================+======================+======================|
|
||||||
|
| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
|
||||||
|
| 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
|
||||||
|
| | | N/A |
|
||||||
|
+-----------------------------------------+----------------------+----------------------+
|
||||||
|
| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
|
||||||
|
| 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
|
||||||
|
| | | N/A |
|
||||||
|
+-----------------------------------------+----------------------+----------------------+
|
||||||
|
| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
|
||||||
|
| 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
|
||||||
|
| | | N/A |
|
||||||
|
+-----------------------------------------+----------------------+----------------------+
|
||||||
|
|
||||||
|
+---------------------------------------------------------------------------------------+
|
||||||
|
| Processes: |
|
||||||
|
| GPU GI CI PID Type Process name GPU Memory |
|
||||||
|
| ID ID Usage |
|
||||||
|
|=======================================================================================|
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
|
||||||
|
|
||||||
|
- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)
|
||||||
|
|
||||||
|
- **Step 4**: Run command to start Jan in GPU mode
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# GPU mode
|
||||||
|
docker compose --profile gpu up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
This will start the web server and you can access Jan at `http://localhost:3000`.
|
||||||
|
|
||||||
|
:::warning
|
||||||
|
|
||||||
|
- Docker mode is currently only suitable for development and localhost. Production is not supported yet, and the RAG feature is not available in Docker mode.
|
||||||
|
|
||||||
|
:::
|
||||||
Loading…
x
Reference in New Issue
Block a user