From d518370e5c3bf858d66d441cfe20212dc0723b96 Mon Sep 17 00:00:00 2001 From: hieu-jan <150573299+hieu-jan@users.noreply.github.com> Date: Fri, 9 Feb 2024 21:00:39 +0700 Subject: [PATCH] docs: add Jan installation using Docker --- README.md | 27 ++--- docs/docs/guides/02-installation/05-docker.md | 102 ++++++++++++++++++ ...5-nightly-build.md => 07-nightly-build.md} | 0 ... => 08-antivirus-compatibility-testing.md} | 0 4 files changed, 117 insertions(+), 12 deletions(-) create mode 100644 docs/docs/guides/02-installation/05-docker.md rename docs/docs/guides/02-installation/{05-nightly-build.md => 07-nightly-build.md} (100%) rename docs/docs/guides/02-installation/{07-antivirus-compatibility-testing.md => 08-antivirus-compatibility-testing.md} (100%) diff --git a/README.md b/README.md index 5b5263ed1..c02517fd5 100644 --- a/README.md +++ b/README.md @@ -167,6 +167,7 @@ To reset your installation: - Clear Application cache in `~/Library/Caches/jan` ## Requirements for running Jan + - MacOS: 13 or higher - Windows: - Windows 10 or higher @@ -194,17 +195,17 @@ Contributions are welcome! Please read the [CONTRIBUTING.md](CONTRIBUTING.md) fi 1. **Clone the repository and prepare:** - ```bash - git clone https://github.com/janhq/jan - cd jan - git checkout -b DESIRED_BRANCH - ``` + ```bash + git clone https://github.com/janhq/jan + cd jan + git checkout -b DESIRED_BRANCH + ``` 2. **Run development and use Jan Desktop** - ```bash - make dev - ``` + ```bash + make dev + ``` This will start the development server and open the desktop app. @@ -222,14 +223,15 @@ This will build the app MacOS m1/m2 for production (with code signing already do - Supported OS: Linux, WSL2 Docker - Pre-requisites: - - `docker` and `docker compose`, follow instruction [here](https://docs.docker.com/engine/install/ubuntu/) + + - Docker Engine and Docker Compose are required to run Jan in Docker mode. Follow the [instructions](https://docs.docker.com/engine/install/ubuntu/) below to get started with Docker Engine on Ubuntu. ```bash curl -fsSL https://get.docker.com -o get-docker.sh sudo sh ./get-docker.sh --dry-run ``` - - `nvidia-driver` and `nvidia-docker2`, follow instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) (If you want to run with GPU mode) + - If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation. - Run Jan in Docker mode @@ -241,7 +243,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do - **Option 2**: Run Jan in GPU mode - - **Step 1**: Check cuda compatibility with your nvidia driver by running `nvidia-smi` and check the cuda version in the output + - **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output ```bash nvidia-smi @@ -274,7 +276,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do |=======================================================================================| ``` - - **Step 2**: Go to https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags and find the smallest minor version of image tag that matches the cuda version from the output of `nvidia-smi` (e.g. 12.1 -> 12.1.0) + - **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0) - **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`) @@ -286,6 +288,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do ``` This will start the web server and you can access Jan at `http://localhost:3000`. + > Note: Currently, Docker mode is only work for development and localhost, production is not supported yet. RAG feature is not supported in Docker mode yet. ## Acknowledgements diff --git a/docs/docs/guides/02-installation/05-docker.md b/docs/docs/guides/02-installation/05-docker.md new file mode 100644 index 000000000..6236ed92e --- /dev/null +++ b/docs/docs/guides/02-installation/05-docker.md @@ -0,0 +1,102 @@ +--- +title: Docker +slug: /install/docker +description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server. +keywords: + [ + Jan AI, + Jan, + ChatGPT alternative, + local AI, + private AI, + conversational AI, + no-subscription fee, + large language model, + docker installation, + ] +--- + +# Installing Jan using Docker + +## Installation + +### Pre-requisites + +:::note + +**Supported OS**: Linux, WSL2 Docker + +::: + +- Docker Engine and Docker Compose are required to run Jan in Docker mode. Follow the [instructions](https://docs.docker.com/engine/install/ubuntu/) below to get started with Docker Engine on Ubuntu. + +```bash +curl -fsSL https://get.docker.com -o get-docker.sh +sudo sh ./get-docker.sh --dry-run +``` + +- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation. + +### Instructions + +- Run Jan in Docker mode + + - **Option 1**: Run Jan in CPU mode + + ```bash + docker compose --profile cpu up -d + ``` + + - **Option 2**: Run Jan in GPU mode + + - **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output + + ```bash + nvidia-smi + + # Output + +---------------------------------------------------------------------------------------+ + | NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 | + |-----------------------------------------+----------------------+----------------------+ + | GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC | + | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | + | | | MIG M. | + |=========================================+======================+======================| + | 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A | + | 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default | + | | | N/A | + +-----------------------------------------+----------------------+----------------------+ + | 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A | + | 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default | + | | | N/A | + +-----------------------------------------+----------------------+----------------------+ + | 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A | + | 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default | + | | | N/A | + +-----------------------------------------+----------------------+----------------------+ + + +---------------------------------------------------------------------------------------+ + | Processes: | + | GPU GI CI PID Type Process name GPU Memory | + | ID ID Usage | + |=======================================================================================| + ``` + + - **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0) + + - **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`) + + - **Step 4**: Run command to start Jan in GPU mode + + ```bash + # GPU mode + docker compose --profile gpu up -d + ``` + + This will start the web server and you can access Jan at `http://localhost:3000`. + +:::warning + +- Docker mode is currently only suitable for development and localhost. Production is not supported yet, and the RAG feature is not available in Docker mode. + +::: diff --git a/docs/docs/guides/02-installation/05-nightly-build.md b/docs/docs/guides/02-installation/07-nightly-build.md similarity index 100% rename from docs/docs/guides/02-installation/05-nightly-build.md rename to docs/docs/guides/02-installation/07-nightly-build.md diff --git a/docs/docs/guides/02-installation/07-antivirus-compatibility-testing.md b/docs/docs/guides/02-installation/08-antivirus-compatibility-testing.md similarity index 100% rename from docs/docs/guides/02-installation/07-antivirus-compatibility-testing.md rename to docs/docs/guides/02-installation/08-antivirus-compatibility-testing.md