---
title: Remote Server Integration
sidebar_position: 2
description: A step-by-step guide on how to set up Jan to connect with any remote or local API server.
keywords:
  [
    Jan AI,
    Jan,
    ChatGPT alternative,
    local AI,
    private AI,
    conversational AI,
    no-subscription fee,
    large language model,
    import-models-manually,
    remote server,
    OAI compatible,
  ]
---

This guide will show you how to configure Jan as a client and point it to any remote & local (self-hosted) API server.

## OpenAI Platform Configuration

### 1. Create a Model JSON

1. In `~/jan/models`, create a folder named `gpt-3.5-turbo-16k`.

2. In this folder, add a `model.json` file with Filename as `model.json`, `id` matching folder name, `Format` as `api`, `Engine` as `openai`, and `State` as `ready`.


```json title="~/jan/models/gpt-3.5-turbo-16k/model.json"
{
  "sources": [
    {
      "filename": "openai",
      "url": "https://openai.com"
    }
  ],
  "id": "gpt-3.5-turbo-16k",
  "object": "model",
  "name": "OpenAI GPT 3.5 Turbo 16k",
  "version": "1.0",
  "description": "OpenAI GPT 3.5 Turbo 16k model is extremely good",
  "format": "api",
  "settings": {},
  "parameters": {},
  "metadata": {
    "author": "OpenAI",
    "tags": ["General", "Big Context Length"]
  },
  "engine": "openai"
}
```

### `model.json`

The `model.json` file is used to set up your local models. 
:::note
- If you've set up your model's configuration in `nitro.json`, please note that `model.json` can overwrite the settings. 
- When using OpenAI models like GPT-3.5 and GPT-4, you can use the default settings in `model.json` file.
:::

There are two important fields in model.json that you need to setup:

#### Settings
This is the field where to set your engine configurations, there are two imporant field that you need to define for your local models:

| Term              | Description                                             |
|-------------------|---------------------------------------------------------|
| `ctx_len`         | Defined based on the model's context size.              |
| `prompt_template` | Defined based on the model's trained template (e.g., ChatML, Alpaca). |

To set up the `prompt_template` based on your model, follow the steps below:
    1. Visit [Hugging Face](https://huggingface.co/), an open-source machine learning platform.
    2. Find the current model that you're using (e.g., [Gemma 7b it](https://huggingface.co/google/gemma-7b-it)).
    3. Review the text and identify the template.

#### Parameters
`parameters` is the adjustable settings that affect how your model operates or processes the data.
The fields in `parameters` are typically general and can be the same across models. An example is provided below:

```json
"parameters":{
  "temperature": 0.7,
  "top_p": 0.95,
  "stream": true,
  "max_tokens": 4096,
  "frequency_penalty": 0,
  "presence_penalty": 0
}
```


:::tip

 - You can find the list of available models in the [OpenAI Platform](https://platform.openai.com/docs/models/overview).
 - The `id` property needs to match the model name in the list.
   - For example, if you want to use the [GPT-4 Turbo](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo), you must set the `id` property to `gpt-4-1106-preview`.

:::

### 2. Configure OpenAI API Keys

1. Find your API keys in the [OpenAI Platform](https://platform.openai.com/api-keys).
2. Set the OpenAI API keys in `~/jan/engines/openai.json` file.

```json title="~/jan/engines/openai.json"
{
  "full_url": "https://api.openai.com/v1/chat/completions",
  "api_key": "sk-<your key here>"
}
```

### 3. Start the Model

Restart Jan and navigate to the Hub. Then, select your configured model and start the model.

## Engines with OAI Compatible Configuration

This section will show you how to configure a client connection to a remote/local server using Jan's API server running model `mistral-ins-7b-q4` as an example.

:::note

Currently, you can only connect to one OpenAI-compatible endpoint at a time.

:::

### 1. Configure a Client Connection

1. Navigate to the `~/jan/engines` folder.
2. Modify the `openai.json file`. 

:::note

Please note that currently, the code that supports any OpenAI-compatible endpoint only reads `engine/openai.json` file. Thus, it will not search any other files in this directory.

:::

3. Configure `full_url` properties with the endpoint server that you want to connect. For example, if you're going to communicate to Jan's API server, you can configure it as follows:

```json title="~/jan/engines/openai.json"
{
  // "full_url": "https://<server-ip-address>:<port>/v1/chat/completions"
  "full_url": "https://<server-ip-address>:1337/v1/chat/completions"
  // Skip api_key if your local server does not require authentication
  // "api_key": "sk-<your key here>"
}
```

### 2. Create a Model JSON

1. In `~/jan/models`, create a folder named `mistral-ins-7b-q4`.

2. In this folder, add a `model.json` file with Filename as `model.json`, ensure the following configurations:
  - `id` matching folder name.
  - `Format` set to `api`.
  - `Engine` set to `openai`
  - `State` set to `ready`.


```json title="~/jan/models/mistral-ins-7b-q4/model.json"
{
  "sources": [
    {
      "filename": "janai",
      "url": "https://jan.ai"
    }
  ],
  "id": "mistral-ins-7b-q4",
  "object": "model",
  "name": "Mistral Instruct 7B Q4 on Jan API Server",
  "version": "1.0",
  "description": "Jan integration with remote Jan API server",
  "format": "api",
  "settings": {},
  "parameters": {},
  "metadata": {
    "author": "MistralAI, The Bloke",
    "tags": ["remote", "awesome"]
  },
  "engine": "openai"
}

```
### 3. Start the Model

1. Restart Jan and navigate to the **Hub**.
2. Locate your model and click the **Use** button.

:::info[Assistance and Support]

If you have questions or want more preconfigured GGUF models, please join our [Discord community](https://discord.gg/Dt7MxDyNNZ) for support, updates, and discussions.

:::