Merge pull request #659 from janhq/docs/add-models-swagger
Migrate Model definitions to Swagger/OpenAPI
This commit is contained in:
Daniel
GitHub
commit
bea0678e26
@ -2,9 +2,7 @@
|
||||
title: About Jan
|
||||
---
|
||||
|
||||
Jan is a free, open source alternative to OpenAI's platform that runs on a local folder of open-format files.
|
||||
|
||||
We believe in the need for an open source AI ecosystem, and are building the infra and tooling to allow open source AIs to be as usable and comprehensive as proprietary ones.
|
||||
Jan believes in the need for an open source AI ecosystem, and are building the infra and tooling to allow open source AIs to compete on a level playing field with proprietary ones.
|
||||
|
||||
Jan's long-term vision is to build a cognitive framework for future robots, who are practical, useful assistants for humans and businesses in everyday life.
|
||||
|
||||
@ -89,3 +87,5 @@ Drop us a message in our [Discord](https://discord.gg/af6SaTdzpx) and we'll get
|
||||
### Careers
|
||||
|
||||
Jan has a culture of ownership, independent thought, and lightning fast execution. If you'd like to join us, we have open positions on our [careers page](https://janai.bamboohr.com/careers).
|
||||
|
||||
## Footnotes
|
||||
@ -1,3 +1,20 @@
|
||||
---
|
||||
title: API Server
|
||||
---
|
||||
---
|
||||
|
||||
:::warning
|
||||
|
||||
This page is under construction.
|
||||
|
||||
:::
|
||||
|
||||
Jan ships with a built-in API server, that can be used as a drop-in replacement for OpenAI's API.
|
||||
|
||||
Jan runs on port `1337` by default, but this can be changed in Settings.
|
||||
|
||||
Check out the [API Reference](/api-reference) for more information on the API endpoints.
|
||||
|
||||
```
|
||||
curl https://localhost:1337/v1/chat/completions
|
||||
|
||||
```
|
||||
@ -3,11 +3,11 @@ title: Introduction
|
||||
slug: /intro
|
||||
---
|
||||
|
||||
Jan is a ChatGPT-alternative that runs on your own computer.
|
||||
Jan is a ChatGPT-alternative that runs on your own computer, with a [local API server](/api).
|
||||
|
||||
Jan uses [open-source AI models](/guide/models), stores data in [open file formats](/specs/data-structures) and is is highly customizable via [extensions](/guide/extensions).
|
||||
Jan uses [open-source AI models](/docs/models), stores data in [open file formats](/specs/data-structures), is highly customizable via [extensions](/docs/extensions).
|
||||
|
||||
Jan ships with an [OpenAI-compatible API](/api) and a powerful [Assistant framework](/guide/assistants) to create custom AIs.
|
||||
Jan believes in the need for an open source AI ecosystem. We aim to build infra and tooling to allow open source AIs to compete on a level playing field with proprietary offerings.
|
||||
|
||||
## Why Jan?
|
||||
|
||||
@ -15,7 +15,7 @@ Jan ships with an [OpenAI-compatible API](/api) and a powerful [Assistant framew
|
||||
Jan runs 100% on your own machine, [predictably](https://www.reddit.com/r/LocalLLaMA/comments/17mghqr/comment/k7ksti6/?utm_source=share&utm_medium=web2x&context=3), privately and even offline. No one else can see your conversations, not even us.
|
||||
|
||||
#### 🏗️ Extensions
|
||||
Jan ships with a powerful [extension framework](/guide/extensions), which allows developers to extend and customize Jan's functionality. In fact, most core modules of Jan are [built as extensions](/specs/architecture) and use the same extensions API.
|
||||
Jan ships with a powerful [extension framework](/docs/extensions), which allows developers to extend and customize Jan's functionality. In fact, most core modules of Jan are [built as extensions](/specs/architecture) and use the same extensions API.
|
||||
|
||||
#### 🗂️ Open File Formats
|
||||
Jan stores data in a [local folder of non-proprietary files](/specs/data-structures). You're never locked-in and can do what you want with your data with extensions, or even a different app.
|
||||
@ -31,4 +31,4 @@ Minimalism: https://docusaurus.io/docs#design-principles. Not having abstraction
|
||||
|
||||
File-based: User should be able to look at a Jan directory and intuit how it works. Transparency. Editing things via a text editor, vs. needing a database tool for SQLite.
|
||||
|
||||
Participatory: https://www.getlago.com/blog/the-5-reasons-why-we-chose-open-source -->
|
||||
Participatory: https://www.getlago.com/blog/the-5-reasons-why-we-chose-open-source -->
|
||||
@ -2,6 +2,12 @@
|
||||
title: Architecture
|
||||
---
|
||||
|
||||
:::warning
|
||||
|
||||
This page is still under construction, and should be read as a scratchpad
|
||||
|
||||
:::
|
||||
|
||||
- Jan is built using modules
|
||||
- Plugin architecture (on Pluggable-Electron)
|
||||
|
||||
|
||||
@ -4,7 +4,7 @@ title: Chats
|
||||
|
||||
:::warning
|
||||
|
||||
Draft Specification: functionality has not been implemented yet.
|
||||
This page is still under construction, and should be read as a scratchpad
|
||||
|
||||
:::
|
||||
|
||||
|
||||
@ -2,6 +2,12 @@
|
||||
title: Data Structures
|
||||
---
|
||||
|
||||
:::warning
|
||||
|
||||
This page is still under construction, and should be read as a scratchpad
|
||||
|
||||
:::
|
||||
|
||||
|
||||
```sh
|
||||
janroot/
|
||||
@ -18,8 +24,6 @@ janroot/
|
||||
model.json
|
||||
```
|
||||
|
||||
|
||||
|
||||
Jan use the local filesystem for data persistence, similar to VSCode. This allows for composability and tinkerability.
|
||||
|
||||
```sh=
|
||||
|
||||
@ -2,6 +2,8 @@
|
||||
title: Models
|
||||
---
|
||||
|
||||
import ApiSchema from '@theme/ApiSchema';
|
||||
|
||||
:::warning
|
||||
|
||||
Draft Specification: functionality has not been implemented yet.
|
||||
@ -14,118 +16,145 @@ Feedback: [HackMD: Models Spec](https://hackmd.io/ulO3uB1AQCqLa5SAAMFOQw)
|
||||
|
||||
Jan's Model API aims to be as similar as possible to [OpenAI's Models API](https://platform.openai.com/docs/api-reference/models), with additional methods for managing and running models locally.
|
||||
|
||||
### User Objectives
|
||||
### Objectives
|
||||
|
||||
- Users can start/stop models and use them in a thread (or via Chat Completions API)
|
||||
- Users can download, import and delete models
|
||||
- User can configure model settings at the model level or override it at thread-level
|
||||
- Users can use remote models (e.g. OpenAI, OpenRouter)
|
||||
- Users can start/stop models and use them in a thread (or via Chat Completions API)
|
||||
- User can configure default model parameters at the model level (to be overridden later at message or thread level)
|
||||
|
||||
## Models Folder
|
||||
|
||||
Models in Jan are stored in the `/models` folder.
|
||||
Models in Jan are stored in the `/models` folder.
|
||||
|
||||
`<model-name>.json` files.
|
||||
Models are stored and organized by folders, which are atomic representations of a model for easy packaging and version control.
|
||||
|
||||
- Everything needed to represent a `model` is packaged into an `Model folder`.
|
||||
- The `folder` is standalone and can be easily zipped, imported, and exported, e.g. to Github.
|
||||
- The `folder` always contains at least one `Model Object`, declared in a `json` format.
|
||||
- The `folder` and `file` do not have to share the same name
|
||||
- The model `id` is made up of `folder_name/filename` and is thus always unique.
|
||||
A model's folder name is its `model.id` and contains:
|
||||
|
||||
```sh
|
||||
/janroot
|
||||
- `<model-id>.json`, i.e. the [Model Object](#model-object)
|
||||
- Binaries (may be downloaded later)
|
||||
|
||||
```shell
|
||||
/jan # Jan root folder
|
||||
/models
|
||||
azure-openai/ # Folder name
|
||||
azure-openai-gpt3-5.json # File name
|
||||
# GGUF model
|
||||
/llama2-70b
|
||||
llama2-70b.json
|
||||
llama2-70b-q4_k_m.gguf
|
||||
|
||||
llama2-70b/
|
||||
model.json
|
||||
.gguf
|
||||
# Recommended Model (yet to be downloaded)
|
||||
/mistral-7b
|
||||
mistral-7b.json # Contains download instructions
|
||||
# Note: mistral-7b-*.gguf binaries not downloaded yet
|
||||
|
||||
# Remote model
|
||||
/azure-openai-gpt3-5
|
||||
azure-openai-gpt3-5.json
|
||||
# Note: No binaries
|
||||
|
||||
# Multiple Binaries
|
||||
# COMING SOON
|
||||
|
||||
# Multiple Quantizations
|
||||
# COMING SOON
|
||||
|
||||
# Imported model (autogenerated .json)
|
||||
random-model-q4_k_m.bin
|
||||
# Note: will be moved into a autogenerated folder
|
||||
# /random-model-q4_k_m
|
||||
# random-model-q4_k_m.bin
|
||||
# random-model-q4_k_m.json (autogenerated)
|
||||
```
|
||||
|
||||
### Importing Models
|
||||
|
||||
:::warning
|
||||
|
||||
- This has not been confirmed
|
||||
- Dan's view: Jan should auto-detect and create folders automatically
|
||||
- Jan's UI will allow users to rename folders and add metadata
|
||||
|
||||
:::
|
||||
|
||||
You can import a model by just dragging it into the `/models` folder, similar to Oobabooga.
|
||||
|
||||
- Jan will detect and generate a corresponding `model-filename.json` file based on filename
|
||||
- Jan will move it into its own `/model-id` folder once you define a `model-id` via the UI
|
||||
- Jan will populate the model's `model-id.json` as you add metadata through the UI
|
||||
|
||||
## Model Object
|
||||
|
||||
Models in Jan are represented as `json` objects, and are colloquially known as `model.jsons`.
|
||||
:::warning
|
||||
|
||||
Jan's models follow a `<model_name>.json` naming convention.
|
||||
- This is currently not finalized
|
||||
- Dan's view: I think the current JSON is extremely clunky
|
||||
- We should move `init` to top-level (e.g. "settings"?)
|
||||
- We should move `runtime` to top-level (e.g. "parameters"?)
|
||||
- `metadata` is extremely overloaded and should be refactored
|
||||
- Dan's view: we should make a model object very extensible
|
||||
- A `GGUF` model would "extend" a common model object with extra fields (at top level)
|
||||
- Dan's view: State is extremely badly named
|
||||
- Recommended: `downloaded`, `started`, `stopped`, null (for yet-to-download)
|
||||
- We should also note that this is only for local models (not remote)
|
||||
|
||||
Jan's `model.json` aims for rough equivalence with [OpenAI's Model Object](https://platform.openai.com/docs/api-reference/models/object), and add additional properties to support local models.
|
||||
:::
|
||||
|
||||
Jan's `model.json` object properties are optional, i.e. users should be able to run a model declared by an empty `json` file.
|
||||
Jan represents models as `json`-based Model Object files, known colloquially as `model.jsons`. Jan aims for rough equivalence with [OpenAI's Model Object](https://platform.openai.com/docs/api-reference/models/object) with additional properties to support local models.
|
||||
|
||||
```json
|
||||
// ./models/zephr/zephyr-7b-beta-Q4_K_M.json
|
||||
{
|
||||
"source_url": "https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf",
|
||||
"parameters": {
|
||||
"init": {
|
||||
"ctx_len": "2048",
|
||||
"ngl": "100",
|
||||
"embedding": "true",
|
||||
"n_parallel": "4",
|
||||
"pre_prompt": "A chat between a curious user and an artificial intelligence",
|
||||
"user_prompt": "USER: ",
|
||||
"ai_prompt": "ASSISTANT: "
|
||||
},
|
||||
"runtime": {
|
||||
"temperature": "0.7",
|
||||
"token_limit": "2048",
|
||||
"top_k": "0",
|
||||
"top_p": "1",
|
||||
"stream": "true"
|
||||
}
|
||||
},
|
||||
"metadata": {
|
||||
"engine": "llamacpp",
|
||||
"quantization": "Q4_K_M",
|
||||
"size": "7B",
|
||||
}
|
||||
}
|
||||
```
|
||||
Jan's models follow a `model_id.json` naming convention, and are built to be extremely lightweight, with the only mandatory field being a `source_url` to download the model binaries.
|
||||
|
||||
| Property | Type | Description | Validation |
|
||||
| ----------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------ |
|
||||
| `object` | enum: `model`, `assistant`, `thread`, `message` | Type of the Jan Object. Always `model` | Defaults to "model" |
|
||||
| `source_url` | string | The model download source. It can be an external url or a local filepath. | Defaults to `pwd`. See [Source_url](#Source_url) |
|
||||
| `parameters` | map | Defines default model run parameters used by any assistant. | Defaults to `{}` |
|
||||
| `description` | string | A vanity description of the model | Defaults to "" |
|
||||
| `metadata` | map | Stores additional structured information about the model. | Defaults to `{}` |
|
||||
| `metadata.engine` | enum: `llamacpp`, `api`, `tensorrt` | The model backend used to run model. | Defaults to "llamacpp" |
|
||||
| `metadata.quantization` | string | Supported formats only | See [Custom importers](#Custom-importers) |
|
||||
| `metadata.binaries` | array | Supported formats only. | See [Custom importers](#Custom-importers) |
|
||||
| `state` | enum[`to_download` , `downloading`, `ready` , `running`] | Needs more thought | Defaults to `to_download` |
|
||||
| `name` | string | A vanity name | Defaults to filename |
|
||||
<ApiSchema example pointer="#/components/schemas/Model" />
|
||||
|
||||
### Model Source
|
||||
### Types of Models
|
||||
|
||||
There are 3 types of model sources
|
||||
:::warning
|
||||
|
||||
- This is currently not in the Model Object, and requires further discussion.
|
||||
- Dan's view: we should have a field to differentiate between `local` and `remote` models
|
||||
|
||||
:::
|
||||
|
||||
There are 3 types of models.
|
||||
|
||||
- Local model
|
||||
- Remote source
|
||||
- Cloud API
|
||||
- Local model, yet-to-be downloaded (we have the URL)
|
||||
- Remote model (i.e. OpenAI API)
|
||||
|
||||
- Users can download models from a `remote` source or reference an existing `local` model.
|
||||
- If this property is not specified in the Model Object file, then the default behavior is to look in the current directory.
|
||||
- Users can import a local model by providing the filepath to the model
|
||||
#### Local Models
|
||||
|
||||
:::warning
|
||||
|
||||
- This is currently not finalized
|
||||
- Dan's view: we should have `download_url` and `local_url` for local models (and possibly more)
|
||||
|
||||
:::
|
||||
|
||||
A `model.json` for a local model should always reference the following fields:
|
||||
|
||||
- `download_url`: the original download source of the model
|
||||
- `local_url`: the current location of the model binaries (may be array of multiple binaries)
|
||||
|
||||
```json
|
||||
// ./models/llama2/llama2-7bn-gguf.json
|
||||
"source_url": "~/Downloads/llama-2-7bn-q5-k-l.gguf",
|
||||
|
||||
// Default, if property is omitted
|
||||
"source_url": "./",
|
||||
"local_url": "~/Downloads/llama-2-7bn-q5-k-l.gguf",
|
||||
```
|
||||
|
||||
- Users can download a model by remote URL.
|
||||
- Supported url formats:
|
||||
- `https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/blob/main/llama-2-7b-chat.Q3_K_L.gguf`
|
||||
- `https://any-source.com/.../model-binary.bin`
|
||||
#### Remote Models
|
||||
|
||||
- Using a remote API to access model `model-azure-openai-gpt4-turbo.json`
|
||||
- See [source](https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?tabs=command-line%2Cpython&pivots=rest-api)
|
||||
:::warning
|
||||
|
||||
- This is currently not finalized
|
||||
- Dan's view: each cloud model should be provided via a syste module, or define its own params field on the `model` or `model.init` object
|
||||
|
||||
:::
|
||||
|
||||
A `model.json` for a remote model should always reference the following fields:
|
||||
|
||||
- `api_url`: the API endpoint of the model
|
||||
- Any authentication parameters
|
||||
|
||||
```json
|
||||
// Dan's view: This needs to be refactored pretty significantly
|
||||
"source_url": "https://docs-test-001.openai.azure.com/openai.azure.com/docs-test-001/gpt4-turbo",
|
||||
"parameters": {
|
||||
"init" {
|
||||
@ -142,24 +171,75 @@ There are 3 types of model sources
|
||||
}
|
||||
}
|
||||
"metadata": {
|
||||
"engine": "api",
|
||||
"engine": "api", // Dan's view: this should be a `type` field
|
||||
}
|
||||
```
|
||||
|
||||
### Model Formats
|
||||
### Importers
|
||||
|
||||
Additionally, Jan supports importing popular formats. For example, if you provide a HuggingFace URL for a `TheBloke` model, Jan automatically downloads and catalogs all quantizations. Custom importers autofills properties like `metadata.quantization` and `metadata.size`.
|
||||
:::caution
|
||||
|
||||
Supported URL formats with custom importers:
|
||||
- This is only an idea, has not been confirmed as part of spec
|
||||
|
||||
- `huggingface/thebloke`: [Link](https://huggingface.co/TheBloke/Llama-2-7B-GGUF)
|
||||
- `huggingface/thebloke`: [Link](https://huggingface.co/TheBloke/Llama-2-7B-GGUF)
|
||||
- `janhq`: `TODO: put URL here`
|
||||
- `azure_openai`: `https://docs-test-001.openai.azure.com/openai.azure.com/docs-test-001/gpt4-turbo`
|
||||
- `openai`: `api.openai.com`
|
||||
:::
|
||||
|
||||
<details>
|
||||
<summary>Example: Zephyr 7B</summary>
|
||||
Jan builds "importers" for users to seamlessly import models from a single URL.
|
||||
|
||||
We currently only provide this for [TheBloke models on Huggingface](https://huggingface.co/TheBloke) (i.e. one of the patron saints of llama.cpp), but we plan to add more in the future.
|
||||
|
||||
Currently, pasting a TheBloke Huggingface link in the Explore Models page will fire an importer, resulting in an:
|
||||
|
||||
- Nicely-formatted model card
|
||||
- Fully-annotated `model.json` file
|
||||
|
||||
### Multiple Binaries
|
||||
|
||||
:::warning
|
||||
|
||||
- This is currently not finalized
|
||||
- Dan's view: having these fields under `model.metadata` is not maintainable
|
||||
- We should explore some sort of `local_url` structure
|
||||
|
||||
:::
|
||||
|
||||
- Model has multiple binaries `model-llava-1.5-ggml.json`
|
||||
- See [source](https://huggingface.co/mys/ggml_llava-v1.5-13b)
|
||||
|
||||
```json
|
||||
"source_url": "https://huggingface.co/mys/ggml_llava-v1.5-13b",
|
||||
"parameters": {"init": {}, "runtime": {}}
|
||||
"metadata": {
|
||||
"mmproj_binary": "https://huggingface.co/mys/ggml_llava-v1.5-13b/blob/main/mmproj-model-f16.gguf",
|
||||
"ggml_binary": "https://huggingface.co/mys/ggml_llava-v1.5-13b/blob/main/ggml-model-q5_k.gguf",
|
||||
"engine": "llamacpp",
|
||||
"quantization": "Q5_K"
|
||||
}
|
||||
```
|
||||
|
||||
## Models API
|
||||
|
||||
:::warning
|
||||
|
||||
- We should use the OpenAPI spec to discuss APIs
|
||||
- Dan's view: This needs @louis and App Pod to review as they are more familiar with this
|
||||
- Dan's view: Start/Stop model should have some UI indicator (show state, block input)
|
||||
|
||||
:::
|
||||
|
||||
See http://localhost:3001/api-reference#tag/Models.
|
||||
|
||||
| Method | API Call | OpenAI-equivalent |
|
||||
| -------------- | ------------------------------- | ----------------- |
|
||||
| List Models | GET /v1/models | true |
|
||||
| Get Model | GET /v1/models/{model_id} | true |
|
||||
| Delete Model | DELETE /v1/models/{model_id} | true |
|
||||
| Start Model | PUT /v1/models/{model_id}/start | |
|
||||
| Stop Model | PUT /v1/models/{model_id}/start | |
|
||||
| Download Model | POST /v1/models/ | |
|
||||
|
||||
## Examples
|
||||
|
||||
### Local Model
|
||||
|
||||
- Model has 1 binary `model-zephyr-7B.json`
|
||||
- See [source](https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/)
|
||||
@ -192,208 +272,8 @@ Supported URL formats with custom importers:
|
||||
"size": "7B",
|
||||
}
|
||||
```
|
||||
</details>
|
||||
|
||||
### Multiple binaries
|
||||
|
||||
- Model has multiple binaries `model-llava-1.5-ggml.json`
|
||||
- See [source](https://huggingface.co/mys/ggml_llava-v1.5-13b)
|
||||
|
||||
```json
|
||||
"source_url": "https://huggingface.co/mys/ggml_llava-v1.5-13b",
|
||||
"parameters": {"init": {}, "runtime": {}}
|
||||
"metadata": {
|
||||
"mmproj_binary": "https://huggingface.co/mys/ggml_llava-v1.5-13b/blob/main/mmproj-model-f16.gguf",
|
||||
"ggml_binary": "https://huggingface.co/mys/ggml_llava-v1.5-13b/blob/main/ggml-model-q5_k.gguf",
|
||||
"engine": "llamacpp",
|
||||
"quantization": "Q5_K"
|
||||
}
|
||||
```
|
||||
|
||||
## Models API
|
||||
|
||||
### Get Model
|
||||
|
||||
- OpenAI Equivalent: https://platform.openai.com/docs/api-reference/models/retrieve
|
||||
- OpenAI Equivalent: https://platform.openai.com/docs/api-reference/models/object
|
||||
- The `Jan Model Object` maps into the `OpenAI Model Object`.
|
||||
- Properties marked with `*` are compatible with the [OpenAI `model` object](https://platform.openai.com/docs/api-reference/models)
|
||||
- Note: The `Jan Model Object` has additional properties when retrieved via its API endpoint.
|
||||
|
||||
#### Request
|
||||
|
||||
```shell
|
||||
curl {JAN_URL}/v1/models/{model_id}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "model-zephyr-7B",
|
||||
"object": "model",
|
||||
"created_at": 1686935002,
|
||||
"owned_by": "thebloke",
|
||||
"state": "running",
|
||||
"source_url": "https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf",
|
||||
"parameters": {
|
||||
"ctx_len": 2048,
|
||||
"ngl": 100,
|
||||
"embedding": true,
|
||||
"n_parallel": 4,
|
||||
"pre_prompt": "A chat between a curious user and an artificial intelligence",
|
||||
"user_prompt": "USER: ",
|
||||
"ai_prompt": "ASSISTANT: ",
|
||||
"temperature": "0.7",
|
||||
"token_limit": "2048",
|
||||
"top_k": "0",
|
||||
"top_p": "1",
|
||||
},
|
||||
"metadata": {
|
||||
"engine": "llamacpp",
|
||||
"quantization": "Q3_K_L",
|
||||
"size": "7B",
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### List models
|
||||
Lists the currently available models, and provides basic information about each one such as the owner and availability.
|
||||
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/models/list
|
||||
|
||||
#### Request
|
||||
|
||||
```shell=
|
||||
curl {JAN_URL}/v1/models
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```json
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "model-zephyr-7B",
|
||||
"object": "model",
|
||||
"created_at": 1686935002,
|
||||
"owned_by": "thebloke",
|
||||
"state": "running"
|
||||
},
|
||||
{
|
||||
"id": "ft-llama-70b-gguf",
|
||||
"object": "model",
|
||||
"created_at": 1686935002,
|
||||
"owned_by": "you",
|
||||
"state": "stopped"
|
||||
},
|
||||
{
|
||||
"id": "model-azure-openai-gpt4-turbo",
|
||||
"object": "model",
|
||||
"created_at": 1686935002,
|
||||
"owned_by": "azure_openai",
|
||||
"state": "running"
|
||||
},
|
||||
],
|
||||
"object": "list"
|
||||
}
|
||||
```
|
||||
|
||||
### Delete Model
|
||||
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/models/delete
|
||||
|
||||
#### Request
|
||||
|
||||
```shell
|
||||
curl -X DELETE {JAN_URL}/v1/models/{model_id}
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "model-zephyr-7B",
|
||||
"object": "model",
|
||||
"deleted": true,
|
||||
"state": "to_download"
|
||||
}
|
||||
```
|
||||
|
||||
### Start Model
|
||||
> Jan-only endpoint
|
||||
The request to start `model` by changing model state from `ready` to `running`
|
||||
|
||||
#### Request
|
||||
|
||||
```shell
|
||||
curl -X PUT {JAN_URL}/v1/models{model_id}/start
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "model-zephyr-7B",
|
||||
"object": "model",
|
||||
"state": "running"
|
||||
}
|
||||
```
|
||||
|
||||
### Stop Model
|
||||
> Jan-only endpoint
|
||||
The request to start `model` by changing model state from `running` to `ready`
|
||||
|
||||
#### Request
|
||||
|
||||
```shell
|
||||
curl -X PUT {JAN_URL}/v1/models/{model_id}/stop
|
||||
```
|
||||
|
||||
#### Response
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "model-zephyr-7B",
|
||||
"object": "model",
|
||||
"state": "ready"
|
||||
}
|
||||
```
|
||||
|
||||
### Download Model
|
||||
> Jan-only endpoint
|
||||
The request to download `model` by changing model state from `to_download` to `downloading` then `ready`once it's done.
|
||||
|
||||
#### Request
|
||||
```shell
|
||||
curl -X POST {JAN_URL}/v1/models/
|
||||
```
|
||||
|
||||
#### Response
|
||||
```json
|
||||
{
|
||||
"id": "model-zephyr-7B",
|
||||
"object": "model",
|
||||
"state": "downloading"
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Pre-loaded Models
|
||||
|
||||
- Jan ships with a default model folders containing recommended models
|
||||
- Only the Model Object `json` files are included
|
||||
- Users must later explicitly download the model binaries
|
||||
-
|
||||
```sh
|
||||
models/
|
||||
mistral-7b/
|
||||
mistral-7b.json
|
||||
hermes-7b/
|
||||
hermes-7b.json
|
||||
```
|
||||
|
||||
### Azure OpenAI
|
||||
### Remote Model
|
||||
|
||||
- Using a remote API to access model `model-azure-openai-gpt4-turbo.json`
|
||||
- See [source](https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?tabs=command-line%2Cpython&pivots=rest-api)
|
||||
@ -419,9 +299,24 @@ models/
|
||||
}
|
||||
```
|
||||
|
||||
### Deferred Download
|
||||
|
||||
- Jan ships with a default model folders containing recommended models
|
||||
- Only the Model Object `json` files are included
|
||||
- Users must later explicitly download the model binaries
|
||||
-
|
||||
```sh
|
||||
models/
|
||||
mistral-7b/
|
||||
mistral-7b.json
|
||||
hermes-7b/
|
||||
hermes-7b.json
|
||||
```
|
||||
|
||||
### Multiple quantizations
|
||||
|
||||
- Each quantization has its own `Jan Model Object` file
|
||||
- TODO: `model.json`?
|
||||
|
||||
```sh
|
||||
llama2-7b-gguf/
|
||||
@ -441,9 +336,7 @@ llava-ggml/
|
||||
ggml
|
||||
```
|
||||
|
||||
### Your locally fine-tuned model
|
||||
|
||||
- ??
|
||||
### Locally fine-tuned model
|
||||
|
||||
```sh
|
||||
llama-70b-finetune/
|
||||
|
||||
@ -2,6 +2,12 @@
|
||||
title: User Interface
|
||||
---
|
||||
|
||||
:::warning
|
||||
|
||||
This page is still under construction, and should be read as a scratchpad
|
||||
|
||||
:::
|
||||
|
||||
Jan provides a UI Kit for customize the UI for your use case. This means you can personalize the entire application according to your own brand and visual styles.
|
||||
|
||||
This page gives you an overview of how to customize the UI.
|
||||
|
||||
@ -2309,12 +2309,14 @@ paths:
|
||||
]
|
||||
}
|
||||
|
||||
# Models
|
||||
/models:
|
||||
get:
|
||||
operationId: listModels
|
||||
tags:
|
||||
- Models
|
||||
summary: Lists the currently available models, and provides basic information about each one such as the owner and availability.
|
||||
summary: List Models
|
||||
description: Lists the currently available models, and provides basic information about each one such as the owner and availability.
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
@ -2373,12 +2375,68 @@ paths:
|
||||
],
|
||||
"object": "list"
|
||||
}
|
||||
post:
|
||||
operationId: importModel
|
||||
tags:
|
||||
- Models
|
||||
summary: Import Model
|
||||
description: Imports a model instance. The model can be from a local folder, remote source, or an API endpoint. The model importer will examine the source_url for formatting.
|
||||
parameters:
|
||||
- in: path
|
||||
name: source_url
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
# ideally this will be an actual ID, so this will always work from browser
|
||||
example: https://huggingface.com/thebloke/example.gguf
|
||||
description: The ID of the model to use for this request
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/Model"
|
||||
x-oaiMeta:
|
||||
name: Import model
|
||||
returns: The [model](/docs/api-reference/models/object) object matching the specified ID.
|
||||
examples:
|
||||
request:
|
||||
curl: |
|
||||
curl https://api.openai.com/v1/models/VAR_model_id \
|
||||
-H "Authorization: Bearer $OPENAI_API_KEY"
|
||||
python: |
|
||||
from openai import OpenAI
|
||||
client = OpenAI()
|
||||
|
||||
client.models.retrieve("VAR_model_id")
|
||||
node.js: |-
|
||||
import OpenAI from "openai";
|
||||
|
||||
const openai = new OpenAI();
|
||||
|
||||
async function main() {
|
||||
const model = await openai.models.retrieve("gpt-3.5-turbo");
|
||||
|
||||
console.log(model);
|
||||
}
|
||||
|
||||
main();
|
||||
response: &retrieve_model_response |
|
||||
{
|
||||
"id": "VAR_model_id",
|
||||
"object": "model",
|
||||
"created": 1686935002,
|
||||
"owned_by": "openai",
|
||||
"state": "ready"
|
||||
}
|
||||
/models/{model}:
|
||||
get:
|
||||
operationId: retrieveModel
|
||||
tags:
|
||||
- Models
|
||||
summary: Retrieves a model instance, providing basic information about the model such as the owner and permissioning.
|
||||
summary: Retrieve Model
|
||||
description: Retrieves a model instance, providing basic information about the model such as the owner and permissioning.
|
||||
parameters:
|
||||
- in: path
|
||||
name: model
|
||||
@ -2431,7 +2489,8 @@ paths:
|
||||
operationId: deleteModel
|
||||
tags:
|
||||
- Models
|
||||
summary: Delete a fine-tuned model. You must have the Owner role in your organization to delete a model.
|
||||
summary: Delete Model
|
||||
description: Delete a fine-tuned model. You must have the Owner role in your organization to delete a model.
|
||||
parameters:
|
||||
- in: path
|
||||
name: model
|
||||
@ -2478,6 +2537,70 @@ paths:
|
||||
"object": "model",
|
||||
"deleted": true
|
||||
}
|
||||
post:
|
||||
operationId: startModel
|
||||
tags:
|
||||
- Models
|
||||
summary: Start Model
|
||||
description: Starts an imported model. Loads the model into V/RAM.
|
||||
parameters:
|
||||
- in: path
|
||||
name: model
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
# ideally this will be an actual ID, so this will always work from browser
|
||||
example: gpt-3.5-turbo
|
||||
description: The ID of the model to use for this request
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/Model"
|
||||
x-oaiMeta:
|
||||
name: Import model
|
||||
returns: The [model](/docs/api-reference/models/object) object matching the specified ID.
|
||||
examples:
|
||||
response: &retrieve_model_response |
|
||||
{
|
||||
"id": "VAR_model_id",
|
||||
"object": "model",
|
||||
"created": 1686935002,
|
||||
"owned_by": "openai"
|
||||
}
|
||||
/models/{model}/stop:
|
||||
post:
|
||||
operationId: stopModel
|
||||
tags:
|
||||
- Models
|
||||
summary: Stop Model
|
||||
description: Stops a running model. Unloads the model from V/RAM.
|
||||
parameters:
|
||||
- in: path
|
||||
name: model
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
description: The ID of the model that is running.
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/RunObject"
|
||||
x-oaiMeta:
|
||||
name: Stop a running model
|
||||
beta: true
|
||||
returns: The modified [run](/docs/api-reference/runs/object) object matching the specified ID.
|
||||
examples:
|
||||
request:
|
||||
response: |
|
||||
{
|
||||
"todo": "run_BeRGmpGt2wb1VI22ZRniOkrR"
|
||||
}
|
||||
|
||||
/moderations:
|
||||
post:
|
||||
@ -3059,6 +3182,7 @@ paths:
|
||||
"deleted": true
|
||||
}
|
||||
|
||||
# Threads
|
||||
/threads:
|
||||
post:
|
||||
operationId: createThread
|
||||
@ -7314,29 +7438,146 @@ components:
|
||||
- model
|
||||
- input
|
||||
- voice
|
||||
|
||||
Model:
|
||||
title: Model
|
||||
description: Describes an OpenAI model offering that can be used with the API.
|
||||
description: Describes an Jan model
|
||||
properties:
|
||||
id:
|
||||
type: string
|
||||
description: The model identifier, which can be referenced in the API endpoints.
|
||||
created:
|
||||
type: integer
|
||||
description: The Unix timestamp (in seconds) when the model was created.
|
||||
object:
|
||||
type: string
|
||||
description: The object type, which is always "model".
|
||||
enum: [model]
|
||||
owned_by:
|
||||
default: model
|
||||
version:
|
||||
type: integer
|
||||
description: The version of the Model Object file
|
||||
default: 1
|
||||
source_url:
|
||||
type: string
|
||||
description: The organization that owns the model.
|
||||
format: uri
|
||||
example: https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf
|
||||
description: The model download source. It can be an external url or a local filepath.
|
||||
id: # OpenAI-equivalent
|
||||
type: string
|
||||
description: The model identifier, which can be referenced in the API endpoints.
|
||||
example: zephyr-7b
|
||||
name:
|
||||
type: string
|
||||
description: Human-readable name that is used for UI
|
||||
owned_by: # OpenAI-equivalent
|
||||
type: string
|
||||
description: The organization that owns the model (you!)
|
||||
default: you # TODO
|
||||
created:
|
||||
type: integer
|
||||
description: The Unix timestamp (in seconds) for when the model was created
|
||||
description:
|
||||
type: string
|
||||
default: A cool model from Huggingface
|
||||
state:
|
||||
type: string
|
||||
enum: [to_download, downloading, ready, running]
|
||||
default: to_download
|
||||
parameters:
|
||||
type: object
|
||||
description:
|
||||
properties:
|
||||
init:
|
||||
type: object
|
||||
properties:
|
||||
ctx_len:
|
||||
type: string
|
||||
description: TODO
|
||||
default: 2048
|
||||
ngl:
|
||||
type: string
|
||||
description: TODO
|
||||
default: 100
|
||||
embedding:
|
||||
type: bool
|
||||
description: TODO
|
||||
default: true
|
||||
n_parallel:
|
||||
type: string
|
||||
description: TODO
|
||||
default: 4
|
||||
pre_prompt:
|
||||
type: string
|
||||
description: TODO
|
||||
default: A chat between a curious user and an artificial intelligence
|
||||
user_prompt:
|
||||
type: string
|
||||
description: TODO
|
||||
default: "USER:"
|
||||
ai_prompt:
|
||||
type: string
|
||||
description: TODO
|
||||
default: "ASSISTANT:"
|
||||
default:
|
||||
{
|
||||
ctx_len: 2048,
|
||||
ngl: 100,
|
||||
embedding: true,
|
||||
n_parallel: 4,
|
||||
pre_prompt: "A chat between a curious user and an artificial intelligence",
|
||||
user_prompt: "USER:",
|
||||
ai_prompt: "ASSISTANT:",
|
||||
}
|
||||
runtime:
|
||||
type: object
|
||||
properties:
|
||||
temperature:
|
||||
type: string
|
||||
description: TODO
|
||||
default: 0.7
|
||||
token_limit:
|
||||
type: string
|
||||
description: TODO
|
||||
default: 2048
|
||||
top_k:
|
||||
type: string
|
||||
description: TODO
|
||||
default: 0
|
||||
top_p:
|
||||
type: string
|
||||
description: TODO
|
||||
default: 1
|
||||
stream:
|
||||
type: string
|
||||
description: TODO
|
||||
default: true
|
||||
default:
|
||||
{
|
||||
temperature: 0.7,
|
||||
token_limit: 2048,
|
||||
top_k: 0,
|
||||
top_p: 1,
|
||||
stream: true,
|
||||
}
|
||||
metadata:
|
||||
type: object
|
||||
properties:
|
||||
engine:
|
||||
type: string
|
||||
enum: [llamacpp, api, tensorrt]
|
||||
default: llamacpp
|
||||
quantization:
|
||||
type: string
|
||||
description: TODO
|
||||
default: Q4_K_M
|
||||
size:
|
||||
type: string
|
||||
default: 7b
|
||||
binaries:
|
||||
type: array
|
||||
description: TODO
|
||||
default: TODO
|
||||
default:
|
||||
{ engine: llamacpp, quantization: Q4_K_M, size: 7b, binaries: TODO }
|
||||
required:
|
||||
- id
|
||||
- object
|
||||
- created
|
||||
- owned_by
|
||||
- id # From OpenAI
|
||||
- version
|
||||
- source_url
|
||||
- created # From OpenAI, autogenerated in Jan
|
||||
- object # From OpenAI, autogenerated in Jan
|
||||
- owned_by # From OpenAI, autogenerated in Jan
|
||||
x-oaiMeta:
|
||||
name: The model object
|
||||
example: *retrieve_model_response
|
||||
@ -9494,4 +9735,4 @@ x-oaiMeta:
|
||||
path: object
|
||||
- type: endpoint
|
||||
key: createEdit
|
||||
path: create
|
||||
path: create
|
||||
|
||||
@ -61,14 +61,14 @@ const sidebars = {
|
||||
items: [
|
||||
"specs/chats",
|
||||
"specs/models",
|
||||
"specs/threads",
|
||||
"specs/messages",
|
||||
"specs/assistants",
|
||||
"specs/files",
|
||||
"specs/jan",
|
||||
"specs/fine-tuning",
|
||||
"specs/settings",
|
||||
"specs/prompts",
|
||||
// "specs/threads",
|
||||
// "specs/messages",
|
||||
// "specs/assistants",
|
||||
// "specs/files",
|
||||
// "specs/jan",
|
||||
// "specs/fine-tuning",
|
||||
// "specs/settings",
|
||||
// "specs/prompts",
|
||||
],
|
||||
},
|
||||
],
|
||||
|
||||
@ -95,6 +95,7 @@
|
||||
padding-left: 32px;
|
||||
li {
|
||||
@apply leading-loose;
|
||||
line-height: 1.5;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user