Merge pull request #646 from janhq/assistants-threads
docs: assistants and threads specs
This commit is contained in:
Daniel
GitHub
commit
dc4a09da4f
@ -2,79 +2,188 @@
|
||||
title: "Assistants"
|
||||
---
|
||||
|
||||
Assistants can use models and tools.
|
||||
:::warning
|
||||
|
||||
- Jan's `Assistants` are even more powerful than OpenAI due to customizable code in `index.js`
|
||||
Draft Specification: functionality has not been implemented yet.
|
||||
|
||||
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/assistants
|
||||
Feedback: [HackMD: Assistants Spec](https://hackmd.io/KKAznzZvS668R6Vmyf8fCg)
|
||||
|
||||
## Assistant Object
|
||||
:::
|
||||
|
||||
- `assistant.json`
|
||||
- Equivalent to: https://platform.openai.com/docs/api-reference/assistants/object
|
||||
|
||||
## User Stories
|
||||
|
||||
_Users can chat with an assistant_
|
||||
|
||||
- [Wireframes - show asst object properties]
|
||||
- See [Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
|
||||
|
||||
_Users can use Jan - the default assistant_
|
||||
|
||||
- [Wireframes here - show model picker]
|
||||
- See [Default Jan Object](#Default-Jan-Example)
|
||||
|
||||
_Users can create an assistant from scratch_
|
||||
|
||||
- [Wireframes here - show create asst flow]
|
||||
- Users can select any model for an assistant. See Model Spec
|
||||
|
||||
_Users can create an assistant from an existing assistant_
|
||||
|
||||
- [Wireframes showing asst edit mode]
|
||||
|
||||
## Jan Assistant Object
|
||||
|
||||
- A `Jan Assistant Object` is a "representation of an assistant"
|
||||
- Objects are defined by `assistant-uuid.json` files in `json` format
|
||||
- Objects are designed to be compatible with `OpenAI Assistant Objects` with additional properties needed to run on our infrastructure.
|
||||
- ALL object properties are optional, i.e. users should be able to use an assistant declared by an empty `json` file.
|
||||
|
||||
| Property | Type | Description | Validation |
|
||||
| ------------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------- |
|
||||
| `object` | enum: `model`, `assistant`, `thread`, `message` | The Jan Object type | Defaults to `assistant` |
|
||||
| `name` | string | A vanity name. | Defaults to filename |
|
||||
| `description` | string | A vanity description. | Max `n` chars. Defaults to `""` |
|
||||
| `models` | array | A list of Model Objects that the assistant can use. | Defaults to ALL models |
|
||||
| `metadata` | map | This can be useful for storing additional information about the object in a structured format. | Defaults to `{}` |
|
||||
| `tools` | array | TBA. | TBA |
|
||||
| `files` | array | TBA. | TBA |
|
||||
|
||||
### Generic Example
|
||||
|
||||
```json
|
||||
{
|
||||
// Jan specific properties
|
||||
"avatar": "https://lala.png"
|
||||
"thread_location": "ROOT/threads" // Default to root (optional field)
|
||||
// TODO: add moar
|
||||
// janroot/assistants/example/example.json
|
||||
"name": "Homework Helper",
|
||||
|
||||
// OpenAI compatible properties: https://platform.openai.com/docs/api-reference/assistants
|
||||
"id": "asst_abc123",
|
||||
"object": "assistant",
|
||||
"created_at": 1698984975,
|
||||
"name": "Math Tutor",
|
||||
"description": null,
|
||||
"model": reference model.json,
|
||||
"instructions": reference model.json,
|
||||
"tools": [
|
||||
{
|
||||
"type": "rag"
|
||||
}
|
||||
],
|
||||
"file_ids": [],
|
||||
"metadata": {}
|
||||
}
|
||||
// Option 1 (default): all models in janroot/models are available via Model Picker
|
||||
"models": [],
|
||||
|
||||
// Option 2: creator can configure custom parameters on existing models in `janroot/models` &&
|
||||
// Option 3: creator can package a custom model with the assistant
|
||||
"models": [{ ...modelObject1 }, { ...modelObject2 }],
|
||||
```
|
||||
|
||||
## Assistants API
|
||||
### Default Jan Example
|
||||
|
||||
- _TODO_: What would modifying Assistant do? (doesn't mutate `index.js`?)
|
||||
- Every user install has a default "Jan Assistant" declared below.
|
||||
> Q: can we omit most properties in `jan.json`? It's all defaults anyway.
|
||||
|
||||
```json
|
||||
// janroot/assistants/jan/jan.json
|
||||
"description": "Use Jan to chat with all models",
|
||||
```
|
||||
|
||||
## Filesystem
|
||||
|
||||
- Everything needed to represent & run an assistant is packaged into an `Assistant folder`.
|
||||
- The folder is standalone and can be easily zipped, imported, and exported, e.g. to Github.
|
||||
- The folder always contains an `Assistant Object`, declared in an `assistant-uuid.json`.
|
||||
- The folder and file must share the same name: `assistant-uuid`
|
||||
- In the future, the folder will contain all of the resources an assistant needs to run, e.g. custom model binaries, pdf files, custom code, etc.
|
||||
|
||||
```sh
|
||||
GET https://api.openai.com/v1/assistants # List
|
||||
POST https://api.openai.com/v1/assistants # C
|
||||
GET https://api.openai.com/v1/assistants/{assistant_id} # R
|
||||
POST https://api.openai.com/v1/assistants/{assistant_id} # U
|
||||
DELETE https://api.openai.com/v1/assistants/{assistant_id} # D
|
||||
janroot/
|
||||
assistants/
|
||||
jan/ # Assistant Folder
|
||||
jan.json # Assistant Object
|
||||
homework-helper/ # Assistant Folder
|
||||
homework-helper.json # Assistant Object
|
||||
```
|
||||
|
||||
## Assistants Filesystem
|
||||
### Custom Code
|
||||
|
||||
> Not in scope yet. Sharing as a preview only.
|
||||
|
||||
- Assistants can call custom code in the future
|
||||
- Custom code extends beyond `function calling` to any features that can be implemented in `/src`
|
||||
|
||||
```sh
|
||||
/assistants
|
||||
/jan
|
||||
assistant.json # Assistant configs (see below)
|
||||
|
||||
# For any custom code
|
||||
package.json # Import npm modules
|
||||
# e.g. Langchain, Llamaindex
|
||||
/src # Supporting files (needs better name)
|
||||
index.js # Entrypoint
|
||||
process.js # For electron IPC processes (needs better name)
|
||||
|
||||
# `/threads` at root level
|
||||
# `/models` at root level
|
||||
/shakespeare
|
||||
assistant.json
|
||||
model.json # Creator chooses model and settings
|
||||
package.json
|
||||
/src
|
||||
index.js
|
||||
process.js
|
||||
|
||||
/threads # Assistants remember conversations in the future
|
||||
/models # Users can upload custom models
|
||||
/finetuned-model
|
||||
example/ # Assistant Folder
|
||||
example.json # Assistant Object
|
||||
package.json
|
||||
src/
|
||||
index.ts
|
||||
helpers.ts
|
||||
```
|
||||
|
||||
### Knowledge Files
|
||||
|
||||
> Not in scope yet. Sharing as a preview only
|
||||
|
||||
- Assistants can do `retrieval` in future
|
||||
|
||||
```sh
|
||||
|
||||
example/ # Assistant Folder
|
||||
example.json # Assistant Object
|
||||
files/
|
||||
```
|
||||
|
||||
## Jan API
|
||||
|
||||
### Assistant API Object
|
||||
|
||||
#### `GET /v1/assistants/{assistant_id}`
|
||||
|
||||
- The `Jan Assistant Object` maps into the `OpenAI Assistant Object`.
|
||||
- Properties marked with `*` are compatible with the [OpenAI `assistant` object](https://platform.openai.com/docs/api-reference/assistants)
|
||||
- Note: The `Jan Assistant Object` has additional properties when retrieved via its API endpoint.
|
||||
- https://platform.openai.com/docs/api-reference/assistants/getAssistant
|
||||
|
||||
| Property | Type | Public Description | Jan Assistant Object (`a`) Property |
|
||||
| ---------------- | -------------- | ------------------------------------------------------------------------- | ----------------------------------- |
|
||||
| `id`\* | string | Assistant uuid, also the name of the Jan Assistant Object file: `id.json` | `json` filename |
|
||||
| `object`\* | string | Always "assistant" | `a.object` |
|
||||
| `created_at`\* | integer | Timestamp when assistant was created. | `a.json` creation time |
|
||||
| `name`\* | string or null | A display name | `a.name` or `id` |
|
||||
| `description`\* | string or null | A description | `a.description` |
|
||||
| `model`\* | string | Text | `a.models[0].name` |
|
||||
| `instructions`\* | string or null | Text | `a.models[0].parameters.prompt` |
|
||||
| `tools`\* | array | TBA | `a.tools` |
|
||||
| `file_ids`\* | array | TBA | `a.files` |
|
||||
| `metadata`\* | map | TBA | `a.metadata` |
|
||||
| `models` | array | TBA | `a.models` |
|
||||
|
||||
### Create Assistant
|
||||
|
||||
#### `POST /v1/assistants`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/assistants/createAssistant
|
||||
|
||||
### Retrieve Assistant
|
||||
|
||||
#### `GET v1/assistants/{assistant_id}`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/assistants/getAssistant
|
||||
|
||||
### Modify Assistant
|
||||
|
||||
#### `POST v1/assistants/{assistant_id}`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/assistants/modifyAssistant
|
||||
|
||||
### Delete Assistant
|
||||
|
||||
#### `DELETE v1/assistants/{assistant_id}`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/assistants/deleteAssistant
|
||||
|
||||
### List Assistants
|
||||
|
||||
#### `GET v1/assistants`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/assistants/listAssistants
|
||||
|
||||
### CRUD Assistant.Models
|
||||
|
||||
- This is a Jan-only endpoint, since Jan supports the ModelPicker, i.e. an `assistant` can be created to run with many `models`.
|
||||
|
||||
#### `POST /v1/assistants/{assistant_id}/models`
|
||||
|
||||
#### `GET /v1/assistants/{assistant_id}/models`
|
||||
|
||||
#### `GET /v1/assistants/{assistant_id}/models/{model_id}`
|
||||
|
||||
#### `DELETE /v1/assistants/{assistant_id}/models`
|
||||
|
||||
Note: There's no need to implement `Modify Assistant.Models`
|
||||
|
||||
@ -2,6 +2,12 @@
|
||||
title: "Chats"
|
||||
---
|
||||
|
||||
:::warning
|
||||
|
||||
Draft Specification: functionality has not been implemented yet.
|
||||
|
||||
:::
|
||||
|
||||
Chats are essentially inference requests to a model
|
||||
|
||||
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/chat
|
||||
|
||||
@ -2,6 +2,12 @@
|
||||
title: "Files"
|
||||
---
|
||||
|
||||
:::warning
|
||||
|
||||
Draft Specification: functionality has not been implemented yet.
|
||||
|
||||
:::
|
||||
|
||||
Files can be used by `threads`, `assistants` and `fine-tuning`
|
||||
|
||||
> Equivalent to: https://platform.openai.com/docs/api-reference/files
|
||||
|
||||
@ -2,6 +2,14 @@
|
||||
title: "Messages"
|
||||
---
|
||||
|
||||
:::warning
|
||||
|
||||
Draft Specification: functionality has not been implemented yet.
|
||||
|
||||
Feedback: [HackMD: Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
|
||||
|
||||
:::
|
||||
|
||||
Messages are within `threads` and capture additional metadata.
|
||||
|
||||
- Equivalent to: https://platform.openai.com/docs/api-reference/messages
|
||||
|
||||
@ -2,6 +2,14 @@
|
||||
title: "Models"
|
||||
---
|
||||
|
||||
:::warning
|
||||
|
||||
Draft Specification: functionality has not been implemented yet.
|
||||
|
||||
Feedback: [HackMD: Models Spec](https://hackmd.io/ulO3uB1AQCqLa5SAAMFOQw)
|
||||
|
||||
:::
|
||||
|
||||
Models are AI models like Llama and Mistral
|
||||
|
||||
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/models
|
||||
|
||||
@ -2,52 +2,133 @@
|
||||
title: "Threads"
|
||||
---
|
||||
|
||||
Threads contain `messages` history with assistants. Messages in a thread share context.
|
||||
:::warning
|
||||
|
||||
- Note: For now, threads "lock the model" after a `message` is sent
|
||||
- When a new `thread` is created with Jan, users can choose the models
|
||||
- Users can still edit model parameters/system prompts
|
||||
- Note: future Assistants may customize this behavior
|
||||
- Note: Assistants will be able to specify default thread location in the future
|
||||
- Jan uses root-level threads, to allow for future multi-assistant threads
|
||||
- Assistant Y may store threads in its own folder, to allow for [long-term assistant memory](https://github.com/janhq/jan/issues/344)
|
||||
Draft Specification: functionality has not been implemented yet.
|
||||
|
||||
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads
|
||||
Feedback: [HackMD: Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
|
||||
|
||||
## Thread Object
|
||||
:::
|
||||
|
||||
- `thread.json`
|
||||
- Equivalent to: https://platform.openai.com/docs/api-reference/threads/object
|
||||
## User Stories
|
||||
|
||||
_Users can chat with an assistant in a thread_
|
||||
|
||||
- See [Messages Spec]
|
||||
|
||||
_Users can change model in a new thread_
|
||||
|
||||
- Wireframes here
|
||||
|
||||
_Users can change model parameters in a thread_
|
||||
|
||||
- Wireframes of
|
||||
|
||||
_Users can delete all thread history_
|
||||
|
||||
- Wireframes of settings page.
|
||||
|
||||
## Jan Thread Object
|
||||
|
||||
- A `Jan Thread Object` is a "representation of a conversation thread" between an `assistant` and the user
|
||||
- Objects are defined by `thread-uuid.json` files in `json` format
|
||||
- Objects are designed to be compatible with `OpenAI Thread Objects` with additional properties needed to run on our infrastructure.
|
||||
- Objects contain a `models` field, to track when the user overrides the assistant's default model parameters.
|
||||
|
||||
| Property | Type | Description | Validation |
|
||||
| ---------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
|
||||
| `object` | enum: `model`, `assistant`, `thread`, `message` | The Jan Object type | Defaults to `thread` |
|
||||
| `models` | array | An array of Jan Model Objects. Threads can "override" an assistant's model run parameters. Thread-level model parameters are directly saved in the `thread.models` property! (see Models spec) | Defaults to `assistant.models` |
|
||||
| `messages` | array | An array of Jan Message Objects. (see Messages spec) | Defaults to `[]` |
|
||||
| `metadata` | map | Useful for storing additional information about the object in a structured format. | Defaults to `{}` |
|
||||
|
||||
### Generic Example
|
||||
|
||||
```json
|
||||
{
|
||||
// Jan specific properties:
|
||||
"summary": "HCMC restaurant recommendations",
|
||||
"messages": {see below}
|
||||
|
||||
// OpenAI compatible properties: https://platform.openai.com/docs/api-reference/threads)
|
||||
"id": "thread_abc123",
|
||||
"object": "thread",
|
||||
"created_at": 1698107661,
|
||||
"metadata": {}
|
||||
}
|
||||
// janroot/threads/jan_1700123404.json
|
||||
"messages": [
|
||||
{...message0}, {...message1}
|
||||
],
|
||||
"metadata": {
|
||||
"summary": "funny physics joke",
|
||||
},
|
||||
```
|
||||
|
||||
## Threads API
|
||||
## Filesystem
|
||||
|
||||
- Equivalent to: https://platform.openai.com/docs/api-reference/threads
|
||||
|
||||
```sh=
|
||||
POST https://localhost:1337/v1/threads/{thread_id} # Create thread
|
||||
GET https://localhost:1337/v1/threads/{thread_id} # Get thread
|
||||
DELETE https://localhost:1337/v1/models/{thread_id} # Delete thread
|
||||
```
|
||||
|
||||
## Threads Filesystem
|
||||
- `Jan Thread Objects`' `json` files always has the naming schema: `assistant_uuid` + `unix_time_thread_created_at. See below.
|
||||
- Threads are all saved in the `janroot/threads` folder in a flat folder structure.
|
||||
- The folder is standalone and can be easily zipped, exported, and cleared.
|
||||
|
||||
```sh
|
||||
/assistants
|
||||
/homework-helper
|
||||
/threads # context is "permanently remembered" by assistant in future conversations
|
||||
/threads # context is only retained within a single thread
|
||||
janroot/
|
||||
threads/
|
||||
jan_1700123404.json
|
||||
homework_helper_700120003.json
|
||||
```
|
||||
|
||||
## Jan API
|
||||
|
||||
### Thread API Object
|
||||
|
||||
#### `GET /v1/threads/{thread_id}`
|
||||
|
||||
- The `Jan Thread Object` maps into the `OpenAI Thread Object`.
|
||||
- Properties marked with `*` are compatible with the [OpenAI `thread` object](https://platform.openai.com/docs/api-reference/threads)
|
||||
- Note: The `Jan Thread Object` has additional properties when retrieved via its API endpoint.
|
||||
- https://platform.openai.com/docs/api-reference/threads/getThread
|
||||
|
||||
| Property | Type | Public Description | Jan Thread Object (`t`) Property |
|
||||
| -------------- | ------- | ------------------------------------------------------------------- | -------------------------------- |
|
||||
| `id`\* | string | Thread uuid, also the name of the Jan Thread Object file: `id.json` | `json` filename |
|
||||
| `object`\* | string | Always "thread" | `t.object` |
|
||||
| `created_at`\* | integer | | `json` file creation time |
|
||||
| `metadata`\* | map | | `t.metadata` |
|
||||
| `models` | array | | `t.models` |
|
||||
| `messages` | array | | `t.messages` |
|
||||
|
||||
### Create Thread
|
||||
|
||||
#### `POST /v1/threads`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/threads/createThread
|
||||
|
||||
### Retrieve Thread
|
||||
|
||||
#### `GET v1/threads/{thread_id}`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/threads/getThread
|
||||
|
||||
### Modify Thread
|
||||
|
||||
#### `POST v1/threads/{thread_id}`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/threads/modifyThread
|
||||
|
||||
### Delete Thread
|
||||
|
||||
#### `DELETE v1/threads/{thread_id}`
|
||||
|
||||
- https://platform.openai.com/docs/api-reference/threads/deleteThread
|
||||
|
||||
### List Threads
|
||||
|
||||
> This is a Jan-only endpoint, not supported by OAI yet.
|
||||
|
||||
#### `GET v1/threads`
|
||||
|
||||
### Get & Modify `Thread.Models`
|
||||
|
||||
> This is a Jan-only endpoint, not supported by OAI yet.
|
||||
|
||||
#### `GET v1/threads/{thread_id}/models`
|
||||
|
||||
#### `POST v1/threads/{thread_id}/models/{model_id}`
|
||||
|
||||
- Since users can change model parameters in an existing thread
|
||||
|
||||
### List `Thread.Messages`
|
||||
|
||||
> This is a Jan-only endpoint, not supported by OAI yet.
|
||||
|
||||
#### `GET v1/threads/{thread_id}/messages`
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user