docs: Updated the contents
This commit is contained in:
parent
b9fd5cbd22
commit
883fbce0bf
BIN
docs/docs/guides/get-started/asset/download.gif
Normal file
BIN
docs/docs/guides/get-started/asset/download.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 32 MiB |
BIN
docs/docs/guides/get-started/asset/gpt.gif
Normal file
BIN
docs/docs/guides/get-started/asset/gpt.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 17 MiB |
BIN
docs/docs/guides/get-started/asset/model.gif
Normal file
BIN
docs/docs/guides/get-started/asset/model.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 23 MiB |
@ -1,7 +1,7 @@
|
||||
---
|
||||
title: Quickstart
|
||||
slug: /guides/quickstart
|
||||
description: Jan Docs | Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
|
||||
description: Get started quickly with Jan, a ChatGPT-alternative that runs on your own computer, with a local API server. Learn how to install Jan and select an AI model to start chatting.
|
||||
sidebar_position: 2
|
||||
keywords:
|
||||
[
|
||||
@ -14,10 +14,16 @@ keywords:
|
||||
no-subscription fee,
|
||||
large language model,
|
||||
quickstart,
|
||||
getting started,
|
||||
using AI model,
|
||||
installation
|
||||
]
|
||||
---
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
import download from './asset/download.gif';
|
||||
import gpt from './asset/gpt.gif';
|
||||
import model from './asset/model.gif';
|
||||
|
||||
To get started quickly with Jan, follow the steps below:
|
||||
## Step 1: Get Jan Desktop
|
||||
@ -173,10 +179,19 @@ If you are stuck in a broken build, go to the [Broken Build](/guides/common-erro
|
||||
</Tabs>
|
||||
|
||||
## Step 2: Download a Model
|
||||
Before using Jan, you must download a pre-configured AI model. Jan offers a selection of local AI models for various purposes and requirements, available for download without needing an API key.
|
||||
Jan provides a variety of local AI models tailored to different needs, ready for download. These models are installed and run directly on the user's device.
|
||||
|
||||
1. Go to the **Hub**.
|
||||
2. Select the models that you would like to install, to see a model details click the dropdown button.
|
||||
3. Click the **Download** button.
|
||||
|
||||
<br/>
|
||||
|
||||
<div style={{ display: 'flex', justifyContent: 'center' }}>
|
||||
<img src={download} alt="Download a Model" />
|
||||
</div>
|
||||
|
||||
<br/>
|
||||
|
||||
:::note
|
||||
|
||||
@ -184,23 +199,39 @@ Ensure you select the appropriate model size by balancing performance, cost, and
|
||||
:::
|
||||
|
||||
## Step 3: Connect to ChatGPT (Optional)
|
||||
Jan also offers a remote model that requires an API key for access. For instance, to use the ChatGPT model with Jan, you must enter your API key to establish a connection by following the steps below:
|
||||
Jan also provides access to remote models hosted on external servers, requiring an API key for connectivity. For example, to use the ChatGPT model with Jan, you must input your API key by following these steps:
|
||||
|
||||
1. Go to the **Thread** tab.
|
||||
2. Under the Model dropdown menu, select the ChatGPT model.
|
||||
3. Fill in your ChatGPT API Key that you can get in your [OpenAI platform](https://platform.openai.com/account/api-keys).
|
||||
|
||||
<br/>
|
||||
|
||||
<div style={{ display: 'flex', justifyContent: 'center' }}>
|
||||
<img src={gpt} alt="Connect to ChatGPT" />
|
||||
</div>
|
||||
|
||||
<br/>
|
||||
|
||||
## Step 4: Chat with Models
|
||||
After downloading and configuring your model, you can immediately use it in the **Thread** tab.
|
||||
|
||||
<br/>
|
||||
|
||||
<div style={{ display: 'flex', justifyContent: 'center' }}>
|
||||
<img src={model} alt="Chat with a model" />
|
||||
</div>
|
||||
|
||||
<br/>
|
||||
|
||||
## Best Practices
|
||||
This section outlines best practices for developers, analysts, and AI enthusiasts to enhance their experience with Jan when adding AI locally to their computers. Implementing these practices will optimize the performance of AI models.
|
||||
|
||||
### Follow the Quickstart Guide
|
||||
The [quickstart guide](quickstart.mdx) is designed to facilitate a quick setup process. It provides a clear instruction and simple steps to get you up and running with Jan.ai quickly. Even, if you are inexperienced in AI, the quickstart can offer valuable insights and tips to help you get started quickly.
|
||||
The quickstart guide above is designed to facilitate a quick setup process. It provides a clear instruction and simple steps to get you up and running with Jan.ai quickly. Even, if you are inexperienced in AI.
|
||||
|
||||
### Setting up the Right Models
|
||||
Jan offers a range of pre-configured AI models that are tailored to different tasks and industries. You should identify which on that aligns with your objectives. There are factors to be considered:
|
||||
### Select the Right Models
|
||||
Jan offers a range of pre-configured AI models that are suited for different purposes. You should identify which on that aligns with your objectives. There are factors to be considered:
|
||||
- Capabilities
|
||||
- Accuracy
|
||||
- Processing Speed
|
||||
@ -211,10 +242,10 @@ Jan offers a range of pre-configured AI models that are tailored to different ta
|
||||
:::
|
||||
|
||||
### Setting up Jan
|
||||
Ensure that you familiarize yourself with the Jan application. Jan offers advanced settings that you can adjust. These settings may influence how your AI behaves locally. Please see the [Advanced Settings](/guides/advanced/) article for a complete list of Jan's configurations and instructions on how to configure them.
|
||||
Ensure that you familiarize yourself with the Jan application. Jan offers advanced settings that you can adjust. These settings may influence how your AI behaves locally. Please see the [Advanced Settings](advanced-settings.mdx) article for a complete list of Jan's configurations and instructions on how to configure them.
|
||||
|
||||
### Integrations
|
||||
One of Jan's key features is its ability to integrate with many systems. Whether you are incorporating Jan.ai with any open-source LLM provider or other tools, it is important to understand the integration capabilities and limitations.
|
||||
Jan can work with many different systems and tools. Whether you are incorporating Jan.ai with any open-source LLM provider or other tools, it is important to understand the integration capabilities and limitations.
|
||||
|
||||
### Mastering the Prompt Engineering
|
||||
Prompt engineering is an important aspect when dealing with AI models to generate the desired outputs. Mastering this skill can significantly enhance the performance and the responses of the AI. Below are some tips that you can do for prompt engineering:
|
||||
|
||||
@ -18,7 +18,11 @@ keywords:
|
||||
---
|
||||
|
||||
## Jan Device Compatible
|
||||
Jan support Mac, Windows, and Linux devices.
|
||||
Jan is compatible with macOS, Windows, and Linux, making it accessible for a wide range of users. This compatibility allows users to leverage Jan's AI tools effectively, regardless of their device or operating system.
|
||||
|
||||
:::note
|
||||
For detailed system requirements and setup instructions, refer to our [Hardware Setup](/guides/hardware/) guide.
|
||||
:::
|
||||
|
||||
import DocCardList from "@theme/DocCardList";
|
||||
|
||||
|
||||
@ -1,6 +1,5 @@
|
||||
---
|
||||
title: LlamaCPP Extension
|
||||
slug: /guides/providers/llamacpp
|
||||
sidebar_position: 1
|
||||
description: A step-by-step guide on how to customize the LlamaCPP extension.
|
||||
keywords:
|
||||
@ -25,7 +24,10 @@ import TabItem from '@theme/TabItem';
|
||||
## Overview
|
||||
[Nitro](https://github.com/janhq/nitro) is an inference server on top of [llama.cpp](https://github.com/ggerganov/llama.cpp). It provides an OpenAI-compatible API, queue, & scaling.
|
||||
|
||||
## LlamaCPP Extension
|
||||
:::note
|
||||
Nitro is the default AI engine downloaded with Jan. There is no additional setup needed.
|
||||
:::
|
||||
|
||||
In this guide, we'll walk you through the process of customizing your engine settings by configuring the `nitro.json` file
|
||||
|
||||
|
||||
@ -1,6 +1,5 @@
|
||||
---
|
||||
title: TensorRT-LLM Extension
|
||||
slug: /guides/providers/tensorrt-llm
|
||||
sidebar_position: 2
|
||||
description: A step-by-step guide on how to customize the TensorRT-LLM extension.
|
||||
keywords:
|
||||
@ -27,9 +26,9 @@ Users with Nvidia GPUs can get **20-40% faster token speeds** compared to using
|
||||
This guide walks you through how to install Jan's official [TensorRT-LLM Extension](https://github.com/janhq/nitro-tensorrt-llm). This extension uses [Nitro-TensorRT-LLM](https://github.com/janhq/nitro-tensorrt-llm) as the AI engine, instead of the default [Nitro-Llama-CPP](https://github.com/janhq/nitro). It includes an efficient C++ server to natively execute the [TRT-LLM C++ runtime](https://nvidia.github.io/TensorRT-LLM/gpt_runtime.html). It also comes with additional feature and performance improvements like OpenAI compatibility, tokenizer improvements, and queues.
|
||||
|
||||
:::warning
|
||||
This feature is only available for Windows users. Linux is coming soon.
|
||||
- This feature is only available for Windows users. Linux is coming soon.
|
||||
|
||||
Additionally, we only prebuilt a few demo models. You can always build your desired models directly on your machine. [Read here](#build-your-own-tensorrt-models).
|
||||
- Additionally, we only prebuilt a few demo models. You can always build your desired models directly on your machine. For more information, please see [here](#build-your-own-tensorrt-models).
|
||||
|
||||
:::
|
||||
|
||||
@ -39,14 +38,14 @@ Additionally, we only prebuilt a few demo models. You can always build your desi
|
||||
- Nvidia GPU(s): Ada or Ampere series (i.e. RTX 4000s & 3000s). More will be supported soon.
|
||||
- 3GB+ of disk space to download TRT-LLM artifacts and a Nitro binary
|
||||
- Jan v0.4.9+ or Jan v0.4.8-321+ (nightly)
|
||||
- Nvidia Driver v535+ (For installation guide please see [here](/troubleshooting/#1-ensure-gpu-mode-requirements))
|
||||
- CUDA Toolkit v12.2+ (For installation guide please see [here](/troubleshooting/#1-ensure-gpu-mode-requirements))
|
||||
- Nvidia Driver v535+ (For installation guide, please see [here](/troubleshooting/#1-ensure-gpu-mode-requirements))
|
||||
- CUDA Toolkit v12.2+ (For installation guide, please see [here](/troubleshooting/#1-ensure-gpu-mode-requirements))
|
||||
|
||||
### Step 1: Install TensorRT-Extension
|
||||
|
||||
1. Go to Settings > Extensions
|
||||
2. Click install next to the TensorRT-LLM Extension
|
||||
3. Check that files are correctly downloaded
|
||||
1. Go to **Settings** > **Extensions**.
|
||||
2. Click **Install** next to the TensorRT-LLM Extension.
|
||||
3. Check that files are correctly downloaded.
|
||||
|
||||
```sh
|
||||
ls ~\jan\extensions\@janhq\tensorrt-llm-extension\dist\bin
|
||||
@ -58,8 +57,12 @@ TensorRT-LLM can only run models in `TensorRT` format. These models, aka "Tensor
|
||||
|
||||
We offer a handful of precompiled models for Ampere and Ada cards that you can immediately download and play with:
|
||||
|
||||
1. Restart the application and go to the Hub
|
||||
2. Look for models with the `TensorRT-LLM` label in the recommended models list. Click download. This step might take some time. 🙏
|
||||
1. Restart the application and go to the Hub.
|
||||
2. Look for models with the `TensorRT-LLM` label in the recommended models list > Click **Download**.
|
||||
|
||||
:::note
|
||||
This step might take some time. 🙏
|
||||
:::
|
||||
|
||||

|
||||
|
||||
@ -87,9 +90,10 @@ coming soon
|
||||
For now, the model versions are pinned to the extension versions.
|
||||
|
||||
### Uninstall Extension
|
||||
To uninstall the extension, follow the steps below:
|
||||
|
||||
1. Quit the app
|
||||
2. Go to Settings > Extensions
|
||||
1. Quit the app.
|
||||
2. Go to **Settings** > **Extensions**.
|
||||
3. Delete the entire Extensions folder.
|
||||
4. Reopen the app, only the default extensions should be restored.
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user