Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Changes to Jan Model Spec and fix issues in Swagger

Browse Source
This commit is contained in:
Daniel 2023-11-19 20:11:20 +08:00
parent ef273a7e04
commit 126c2e0132
2 changed files with 58 additions and 46 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

75
docs/docs/specs/models.md
View File

@ -18,52 +18,67 @@ Jan's Model API aims to be as similar as possible to [OpenAI's Models API](https
### Objectives
- Users can start/stop models and use them in a thread (or via Chat Completions API)
- Users can download, import and delete models
- User can configure model settings at the model level or override it at thread-level
- Users can use remote models (e.g. OpenAI, OpenRouter)
- Users can start/stop models and use them in a thread (or via Chat Completions API)
- User can configure default model parameters at the model level (to be overridden later at message or thread level)
## Models Folder
Models in Jan are stored in the `/models` folder.
Models in Jan are stored in the `/models` folder.
`<model-name>.json` files.
Models are stored and organized by folders, which are atomic representations of a model for easy packaging and version control.
- Everything needed to represent a `model` is packaged into an `Model folder`.
- The `folder` is standalone and can be easily zipped, imported, and exported, e.g. to Github.
- The `folder` always contains at least one `Model Object`, declared in a `json` format.
- The `folder` and `file` do not have to share the same name
- The model `id` is made up of `folder_name/filename` and is thus always unique.
A model's folder name is its `model.id` and contains:
Folders are human readable names
- `<model-id>.json`, i.e. the [Model Object](#model-object)
- Binaries (may be downloaded later)
### Auto-detection
You can import a model by just dragging it into the `/models` folder
- Jan will detect and generate a corresponding `.json` file based on filename
- Jan will move it into its own folder once you choose a human readable name for it
- Jan will fill out the model.json file as you add metadata through the UI
```sh
/janroot
```shell
/jan # Jan root folder
/models
azure-openai/ # Folder name
azure-openai-gpt3-5.json # File name
# GGUF model
/llama2-70b
llama2-70b.json
llama2-70b-q4_k_m.gguf
llama2-70b/
model.json
.gguf
# Recommended Model (yet to be downloaded)
/mistral-7b
mistral-7b.json # Contains download instructions
# Note: mistral-7b-*.gguf binaries not downloaded yet
# Remote model
/azure-openai-gpt3-5
azure-openai-gpt3-5.json
# Note: No binaries
# Multiple Binaries
# COMING SOON
# Multiple Quantizations
# COMING SOON
# Imported model (autogenerated .json)
random-model-q4_k_m.bin
# Note: will be moved into a autogenerated folder
# /random-model-q4_k_m
# random-model-q4_k_m.bin
# random-model-q4_k_m.json (autogenerated)
```
### Importing Models
You can import a model by just dragging it into the `/models` folder, similar to Oobabooga.
- Jan will detect and generate a corresponding `model-filename.json` file based on filename
- Jan will move it into its own `/model-id` folder once you define a `model-id` via the UI
- Jan will populate the model's `model-id.json` as you add metadata through the UI
## Model Object
Models in Jan are represented as `json` objects, and are colloquially known as `model.jsons`.
Jan represents models as `json`-based Model Object files, known colloquially as `model.jsons`. Jan aims for rough equivalence with [OpenAI's Model Object](https://platform.openai.com/docs/api-reference/models/object) with additional properties to support local models.
Jan's models follow a `<model_name>.json` naming convention.
Jan's `model.json` aims for rough equivalence with [OpenAI's Model Object](https://platform.openai.com/docs/api-reference/models/object), and add additional properties to support local models.
Jan's `model.json` object properties are optional, i.e. users should be able to run a model declared by an empty `json` file.
Jan's models follow a `model_id.json` naming convention, and are built to be extremely lightweight, with the only mandatory field being a `source_url` to download the model binaries.
<ApiSchema example pointer="#/components/schemas/Model" />

29
docs/openapi/jan.yaml
View File

@ -7321,9 +7321,8 @@ components:
title: Model
description: Describes an Jan model
properties:
type:
object:
type: string
enum: [model, assistant, thread, message] # This should be specified
default: model
version:
type: integer
@ -7332,15 +7331,15 @@ components:
source_url:
type: string
format: uri
default: https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf
example: https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf
description: The model download source. It can be an external url or a local filepath.
id: # OpenAI-equivalent
type: string
description: The model identifier, which can be referenced in the API endpoints.
default: zephyr-7b
description:
example: zephyr-7b
name:
type: string
default: A cool model from Huggingface
description: Human-readable name that is used for UI
owned_by: # OpenAI-equivalent
type: string
description: The organization that owns the model (you!)
@ -7348,6 +7347,9 @@ components:
created:
type: integer
description: The Unix timestamp (in seconds) for when the model was created
description:
type: string
default: A cool model from Huggingface
state:
type: string
enum: [to_download, downloading, ready, running]
@ -7451,17 +7453,12 @@ components:
binaries: TODO
}
required:
- object
- source_url
- parameters
- description
- metadata
- state
- name
- id # From OpenAI
- object # From OpenAI
- created # From OpenAI
- owned_by # From OpenAI
- version
- source_url
- created # From OpenAI, autogenerated in Jan
- object # From OpenAI, autogenerated in Jan
- owned_by # From OpenAI, autogenerated in Jan
x-oaiMeta:
name: The model object
example: *retrieve_model_response