Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #646 from janhq/assistants-threads

Browse Source 
docs: assistants and threads specs
This commit is contained in:
Daniel 2023-11-17 18:07:04 +08:00 committed by GitHub
commit dc4a09da4f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 314 additions and 96 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

227
docs/docs/docs/specs/assistants.md
View File

@ -2,79 +2,188 @@
title: "Assistants" title: "Assistants"
--- ---
Assistants can use models and tools. :::warning
- Jan's `Assistants` are even more powerful than OpenAI due to customizable code in `index.js` Draft Specification: functionality has not been implemented yet.
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/assistants Feedback: [HackMD: Assistants Spec](https://hackmd.io/KKAznzZvS668R6Vmyf8fCg)
## Assistant Object :::
- `assistant.json`
- Equivalent to: https://platform.openai.com/docs/api-reference/assistants/object ## User Stories
_Users can chat with an assistant_
- [Wireframes - show asst object properties]
- See [Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
_Users can use Jan - the default assistant_
- [Wireframes here - show model picker]
- See [Default Jan Object](#Default-Jan-Example)
_Users can create an assistant from scratch_
- [Wireframes here - show create asst flow]
- Users can select any model for an assistant. See Model Spec
_Users can create an assistant from an existing assistant_
- [Wireframes showing asst edit mode]
## Jan Assistant Object
- A `Jan Assistant Object` is a "representation of an assistant"
- Objects are defined by `assistant-uuid.json` files in `json` format
- Objects are designed to be compatible with `OpenAI Assistant Objects` with additional properties needed to run on our infrastructure.
- ALL object properties are optional, i.e. users should be able to use an assistant declared by an empty `json` file.
| Property | Type | Description | Validation |
| ------------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------- |
| `object` | enum: `model`, `assistant`, `thread`, `message` | The Jan Object type | Defaults to `assistant` |
| `name` | string | A vanity name. | Defaults to filename |
| `description` | string | A vanity description. | Max `n` chars. Defaults to `""` |
| `models` | array | A list of Model Objects that the assistant can use. | Defaults to ALL models |
| `metadata` | map | This can be useful for storing additional information about the object in a structured format. | Defaults to `{}` |
| `tools` | array | TBA. | TBA |
| `files` | array | TBA. | TBA |
### Generic Example
```json ```json
{ // janroot/assistants/example/example.json
// Jan specific properties "name": "Homework Helper",
"avatar": "https://lala.png"
"thread_location": "ROOT/threads" // Default to root (optional field)
// TODO: add moar
// OpenAI compatible properties: https://platform.openai.com/docs/api-reference/assistants // Option 1 (default): all models in janroot/models are available via Model Picker
"id": "asst_abc123", "models": [],
"object": "assistant",
"created_at": 1698984975, // Option 2: creator can configure custom parameters on existing models in `janroot/models` &&
"name": "Math Tutor", // Option 3: creator can package a custom model with the assistant
"description": null, "models": [{ ...modelObject1 }, { ...modelObject2 }],
"model": reference model.json,
"instructions": reference model.json,
"tools": [
{
"type": "rag"
}
],
"file_ids": [],
"metadata": {}
}
``` ```
## Assistants API ### Default Jan Example
- _TODO_: What would modifying Assistant do? (doesn't mutate `index.js`?) - Every user install has a default "Jan Assistant" declared below.
> Q: can we omit most properties in `jan.json`? It's all defaults anyway.
```json
// janroot/assistants/jan/jan.json
"description": "Use Jan to chat with all models",
```
## Filesystem
- Everything needed to represent & run an assistant is packaged into an `Assistant folder`.
- The folder is standalone and can be easily zipped, imported, and exported, e.g. to Github.
- The folder always contains an `Assistant Object`, declared in an `assistant-uuid.json`.
- The folder and file must share the same name: `assistant-uuid`
- In the future, the folder will contain all of the resources an assistant needs to run, e.g. custom model binaries, pdf files, custom code, etc.
```sh ```sh
GET https://api.openai.com/v1/assistants # List janroot/
POST https://api.openai.com/v1/assistants # C assistants/
GET https://api.openai.com/v1/assistants/{assistant_id} # R jan/ # Assistant Folder
POST https://api.openai.com/v1/assistants/{assistant_id} # U jan.json # Assistant Object
DELETE https://api.openai.com/v1/assistants/{assistant_id} # D homework-helper/ # Assistant Folder
homework-helper.json # Assistant Object
``` ```
## Assistants Filesystem ### Custom Code
> Not in scope yet. Sharing as a preview only.
- Assistants can call custom code in the future
- Custom code extends beyond `function calling` to any features that can be implemented in `/src`
```sh ```sh
/assistants example/ # Assistant Folder
/jan example.json # Assistant Object
assistant.json # Assistant configs (see below) package.json
src/
# For any custom code index.ts
package.json # Import npm modules helpers.ts
# e.g. Langchain, Llamaindex
/src # Supporting files (needs better name)
index.js # Entrypoint
process.js # For electron IPC processes (needs better name)
# `/threads` at root level
# `/models` at root level
/shakespeare
assistant.json
model.json # Creator chooses model and settings
package.json
/src
index.js
process.js
/threads # Assistants remember conversations in the future
/models # Users can upload custom models
/finetuned-model
``` ```
### Knowledge Files
> Not in scope yet. Sharing as a preview only
- Assistants can do `retrieval` in future
```sh
example/ # Assistant Folder
example.json # Assistant Object
files/
```
## Jan API
### Assistant API Object
#### `GET /v1/assistants/{assistant_id}`
- The `Jan Assistant Object` maps into the `OpenAI Assistant Object`.
- Properties marked with `*` are compatible with the [OpenAI `assistant` object](https://platform.openai.com/docs/api-reference/assistants)
- Note: The `Jan Assistant Object` has additional properties when retrieved via its API endpoint.
- https://platform.openai.com/docs/api-reference/assistants/getAssistant
| Property | Type | Public Description | Jan Assistant Object (`a`) Property |
| ---------------- | -------------- | ------------------------------------------------------------------------- | ----------------------------------- |
| `id`\* | string | Assistant uuid, also the name of the Jan Assistant Object file: `id.json` | `json` filename |
| `object`\* | string | Always "assistant" | `a.object` |
| `created_at`\* | integer | Timestamp when assistant was created. | `a.json` creation time |
| `name`\* | string or null | A display name | `a.name` or `id` |
| `description`\* | string or null | A description | `a.description` |
| `model`\* | string | Text | `a.models[0].name` |
| `instructions`\* | string or null | Text | `a.models[0].parameters.prompt` |
| `tools`\* | array | TBA | `a.tools` |
| `file_ids`\* | array | TBA | `a.files` |
| `metadata`\* | map | TBA | `a.metadata` |
| `models` | array | TBA | `a.models` |
### Create Assistant
#### `POST /v1/assistants`
- https://platform.openai.com/docs/api-reference/assistants/createAssistant
### Retrieve Assistant
#### `GET v1/assistants/{assistant_id}`
- https://platform.openai.com/docs/api-reference/assistants/getAssistant
### Modify Assistant
#### `POST v1/assistants/{assistant_id}`
- https://platform.openai.com/docs/api-reference/assistants/modifyAssistant
### Delete Assistant
#### `DELETE v1/assistants/{assistant_id}`
- https://platform.openai.com/docs/api-reference/assistants/deleteAssistant
### List Assistants
#### `GET v1/assistants`
- https://platform.openai.com/docs/api-reference/assistants/listAssistants
### CRUD Assistant.Models
- This is a Jan-only endpoint, since Jan supports the ModelPicker, i.e. an `assistant` can be created to run with many `models`.
#### `POST /v1/assistants/{assistant_id}/models`
#### `GET /v1/assistants/{assistant_id}/models`
#### `GET /v1/assistants/{assistant_id}/models/{model_id}`
#### `DELETE /v1/assistants/{assistant_id}/models`
Note: There's no need to implement `Modify Assistant.Models`

6
docs/docs/docs/specs/chats.md
View File

@ -2,6 +2,12 @@
title: "Chats" title: "Chats"
--- ---
:::warning
Draft Specification: functionality has not been implemented yet.
:::
Chats are essentially inference requests to a model Chats are essentially inference requests to a model
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/chat > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/chat

6
docs/docs/docs/specs/files.md
View File

@ -2,6 +2,12 @@
title: "Files" title: "Files"
--- ---
:::warning
Draft Specification: functionality has not been implemented yet.
:::
Files can be used by `threads`, `assistants` and `fine-tuning` Files can be used by `threads`, `assistants` and `fine-tuning`
> Equivalent to: https://platform.openai.com/docs/api-reference/files > Equivalent to: https://platform.openai.com/docs/api-reference/files

8
docs/docs/docs/specs/messages.md
View File

@ -2,6 +2,14 @@
title: "Messages" title: "Messages"
--- ---
:::warning
Draft Specification: functionality has not been implemented yet.
Feedback: [HackMD: Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
:::
Messages are within `threads` and capture additional metadata. Messages are within `threads` and capture additional metadata.
- Equivalent to: https://platform.openai.com/docs/api-reference/messages - Equivalent to: https://platform.openai.com/docs/api-reference/messages

8
docs/docs/docs/specs/models.md
View File

@ -2,6 +2,14 @@
title: "Models" title: "Models"
--- ---
:::warning
Draft Specification: functionality has not been implemented yet.
Feedback: [HackMD: Models Spec](https://hackmd.io/ulO3uB1AQCqLa5SAAMFOQw)
:::
Models are AI models like Llama and Mistral Models are AI models like Llama and Mistral
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/models > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/models

155
docs/docs/docs/specs/threads.md
View File

@ -2,52 +2,133 @@
title: "Threads" title: "Threads"
--- ---
Threads contain `messages` history with assistants. Messages in a thread share context. :::warning
- Note: For now, threads "lock the model" after a `message` is sent Draft Specification: functionality has not been implemented yet.
- When a new `thread` is created with Jan, users can choose the models
- Users can still edit model parameters/system prompts
- Note: future Assistants may customize this behavior
- Note: Assistants will be able to specify default thread location in the future
- Jan uses root-level threads, to allow for future multi-assistant threads
- Assistant Y may store threads in its own folder, to allow for [long-term assistant memory](https://github.com/janhq/jan/issues/344)
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads Feedback: [HackMD: Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
## Thread Object :::
- `thread.json` ## User Stories
- Equivalent to: https://platform.openai.com/docs/api-reference/threads/object
_Users can chat with an assistant in a thread_
- See [Messages Spec]
_Users can change model in a new thread_
- Wireframes here
_Users can change model parameters in a thread_
- Wireframes of
_Users can delete all thread history_
- Wireframes of settings page.
## Jan Thread Object
- A `Jan Thread Object` is a "representation of a conversation thread" between an `assistant` and the user
- Objects are defined by `thread-uuid.json` files in `json` format
- Objects are designed to be compatible with `OpenAI Thread Objects` with additional properties needed to run on our infrastructure.
- Objects contain a `models` field, to track when the user overrides the assistant's default model parameters.
| Property | Type | Description | Validation |
| ---------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
| `object` | enum: `model`, `assistant`, `thread`, `message` | The Jan Object type | Defaults to `thread` |
| `models` | array | An array of Jan Model Objects. Threads can "override" an assistant's model run parameters. Thread-level model parameters are directly saved in the `thread.models` property! (see Models spec) | Defaults to `assistant.models` |
| `messages` | array | An array of Jan Message Objects. (see Messages spec) | Defaults to `[]` |
| `metadata` | map | Useful for storing additional information about the object in a structured format. | Defaults to `{}` |
### Generic Example
```json ```json
{ // janroot/threads/jan_1700123404.json
// Jan specific properties: "messages": [
"summary": "HCMC restaurant recommendations", {...message0}, {...message1}
"messages": {see below} ],
"metadata": {
// OpenAI compatible properties: https://platform.openai.com/docs/api-reference/threads) "summary": "funny physics joke",
"id": "thread_abc123", },
"object": "thread",
"created_at": 1698107661,
"metadata": {}
}
``` ```
## Threads API ## Filesystem
- Equivalent to: https://platform.openai.com/docs/api-reference/threads - `Jan Thread Objects`' `json` files always has the naming schema: `assistant_uuid` + `unix_time_thread_created_at. See below.
- Threads are all saved in the `janroot/threads` folder in a flat folder structure.
```sh= - The folder is standalone and can be easily zipped, exported, and cleared.
POST https://localhost:1337/v1/threads/{thread_id} # Create thread
GET https://localhost:1337/v1/threads/{thread_id} # Get thread
DELETE https://localhost:1337/v1/models/{thread_id} # Delete thread
```
## Threads Filesystem
```sh ```sh
/assistants janroot/
/homework-helper threads/
/threads # context is "permanently remembered" by assistant in future conversations jan_1700123404.json
/threads # context is only retained within a single thread homework_helper_700120003.json
``` ```
## Jan API
### Thread API Object
#### `GET /v1/threads/{thread_id}`
- The `Jan Thread Object` maps into the `OpenAI Thread Object`.
- Properties marked with `*` are compatible with the [OpenAI `thread` object](https://platform.openai.com/docs/api-reference/threads)
- Note: The `Jan Thread Object` has additional properties when retrieved via its API endpoint.
- https://platform.openai.com/docs/api-reference/threads/getThread
| Property | Type | Public Description | Jan Thread Object (`t`) Property |
| -------------- | ------- | ------------------------------------------------------------------- | -------------------------------- |
| `id`\* | string | Thread uuid, also the name of the Jan Thread Object file: `id.json` | `json` filename |
| `object`\* | string | Always "thread" | `t.object` |
| `created_at`\* | integer | | `json` file creation time |
| `metadata`\* | map | | `t.metadata` |
| `models` | array | | `t.models` |
| `messages` | array | | `t.messages` |
### Create Thread
#### `POST /v1/threads`
- https://platform.openai.com/docs/api-reference/threads/createThread
### Retrieve Thread
#### `GET v1/threads/{thread_id}`
- https://platform.openai.com/docs/api-reference/threads/getThread
### Modify Thread
#### `POST v1/threads/{thread_id}`
- https://platform.openai.com/docs/api-reference/threads/modifyThread
### Delete Thread
#### `DELETE v1/threads/{thread_id}`
- https://platform.openai.com/docs/api-reference/threads/deleteThread
### List Threads
> This is a Jan-only endpoint, not supported by OAI yet.
#### `GET v1/threads`
### Get & Modify `Thread.Models`
> This is a Jan-only endpoint, not supported by OAI yet.
#### `GET v1/threads/{thread_id}/models`
#### `POST v1/threads/{thread_id}/models/{model_id}`
- Since users can change model parameters in an existing thread
### List `Thread.Messages`
> This is a Jan-only endpoint, not supported by OAI yet.
#### `GET v1/threads/{thread_id}/messages`