Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: assistants and threads specs

Browse Source
This commit is contained in:
0xSage 2023-11-17 14:18:31 +08:00
parent b60978bab3
commit b701e8a8fe
2 changed files with 269 additions and 96 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

218
docs/docs/docs/specs/assistants.md
View File

@ -2,79 +2,179 @@
title: "Assistants"
---
Assistants can use models and tools.
## User Stories
- Jan's `Assistants` are even more powerful than OpenAI due to customizable code in `index.js`
_Users can chat with an assistant_
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/assistants
- [Wireframes - show asst object properties]
- See [Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
## Assistant Object
_Users can use Jan - the default assistant_
- `assistant.json`
- Equivalent to: https://platform.openai.com/docs/api-reference/assistants/object
- [Wireframes here - show model picker]
- See [Default Jan Object](#Default-Jan-Example)
_Users can create an assistant from scratch_
- [Wireframes here - show create asst flow]
- Users can select any model for an assistant. See [Model Spec]()
_Users can create an assistant from an existing assistant_
- [Wireframes showing asst edit mode]
## Jan Assistant Object
- A `Jan Assistant Object` is a "representation of an assistant"
- Objects are defined by `assistant-uuid.json` files in `json` format
- Objects are designed to be compatible with `OpenAI Assistant Objects` with additional properties needed to run on our infrastructure.
- ALL object properties are optional, i.e. users should be able to use an assistant declared by an empty `json` file.
| Property | Type | Description | Validation |
| ------------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------- |
| `object` | enum: `model`, `assistant`, `thread`, `message` | The Jan Object type | Defaults to `assistant` |
| `name` | string | A vanity name. | Defaults to filename |
| `description` | string | A vanity description. | Max `n` chars. Defaults to `""` |
| `models` | array | A list of Model Objects that the assistant can use. | Defaults to ALL models |
| `metadata` | map | This can be useful for storing additional information about the object in a structured format. | Defaults to `{}` |
| `tools` | array | TBA. | TBA |
| `files` | array | TBA. | TBA |
### Generic Example
```json
{
// Jan specific properties
"avatar": "https://lala.png"
"thread_location": "ROOT/threads" // Default to root (optional field)
// TODO: add moar
// janroot/assistants/example/example.json
"name": "Homework Helper",
// OpenAI compatible properties: https://platform.openai.com/docs/api-reference/assistants
"id": "asst_abc123",
"object": "assistant",
"created_at": 1698984975,
"name": "Math Tutor",
"description": null,
"model": reference model.json,
"instructions": reference model.json,
"tools": [
{
"type": "rag"
}
],
"file_ids": [],
"metadata": {}
}
// Option 1 (default): all models in janroot/models are available via Model Picker
"models": [],
// Option 2: creator can configure custom parameters on existing models in `janroot/models` &&
// Option 3: creator can package a custom model with the assistant
"models": [{ ...modelObject1 }, { ...modelObject2 }],
```
## Assistants API
### Default Jan Example
- _TODO_: What would modifying Assistant do? (doesn't mutate `index.js`?)
- Every user install has a default "Jan Assistant" declared below.
> Q: can we omit most properties in `jan.json`? It's all defaults anyway.
```json
// janroot/assistants/jan/jan.json
"description": "Use Jan to chat with all models",
```
## Filesystem
- Everything needed to represent & run an assistant is packaged into an `Assistant folder`.
- The folder is standalone and can be easily zipped, imported, and exported, e.g. to Github.
- The folder always contains an `Assistant Object`, declared in an `assistant-uuid.json`.
- The folder and file must share the same name: `assistant-uuid`
- In the future, the folder will contain all of the resources an assistant needs to run, e.g. custom model binaries, pdf files, custom code, etc.
```sh
GET https://api.openai.com/v1/assistants # List
POST https://api.openai.com/v1/assistants # C
GET https://api.openai.com/v1/assistants/{assistant_id} # R
POST https://api.openai.com/v1/assistants/{assistant_id} # U
DELETE https://api.openai.com/v1/assistants/{assistant_id} # D
janroot/
assistants/
jan/ # Assistant Folder
jan.json # Assistant Object
homework-helper/ # Assistant Folder
homework-helper.json # Assistant Object
```
## Assistants Filesystem
### Custom Code
> Not in scope yet. Sharing as a preview only.
- Assistants can call custom code in the future
- Custom code extends beyond `function calling` to any features that can be implemented in `/src`
```sh
/assistants
/jan
assistant.json # Assistant configs (see below)
# For any custom code
package.json # Import npm modules
# e.g. Langchain, Llamaindex
/src # Supporting files (needs better name)
index.js # Entrypoint
process.js # For electron IPC processes (needs better name)
# `/threads` at root level
# `/models` at root level
/shakespeare
assistant.json
model.json # Creator chooses model and settings
package.json
/src
index.js
process.js
/threads # Assistants remember conversations in the future
/models # Users can upload custom models
/finetuned-model
example/ # Assistant Folder
example.json # Assistant Object
package.json
src/
index.ts
helpers.ts
```
### Knowledge Files
> Not in scope yet. Sharing as a preview only
- Assistants can do `retrieval` in future
```sh
example/ # Assistant Folder
example.json # Assistant Object
files/
```
## Jan API
### Assistant API Object
#### `GET /v1/assistants/{assistant_id}`
- The `Jan Assistant Object` maps into the `OpenAI Assistant Object`.
- Properties marked with `*` are compatible with the [OpenAI `assistant` object](https://platform.openai.com/docs/api-reference/assistants)
- Note: The `Jan Assistant Object` has additional properties when retrieved via its API endpoint.
- https://platform.openai.com/docs/api-reference/assistants/getAssistant
| Property | Type | Public Description | Jan Assistant Object (`a`) Property |
| ---------------- | -------------- | ------------------------------------------------------------------------- | ----------------------------------- |
| `id`\* | string | Assistant uuid, also the name of the Jan Assistant Object file: `id.json` | `json` filename |
| `object`\* | string | Always "assistant" | `a.object` |
| `created_at`\* | integer | Timestamp when assistant was created. | `a.json` creation time |
| `name`\* | string or null | A display name | `a.name` or `id` |
| `description`\* | string or null | A description | `a.description` |
| `model`\* | string | Text | `a.models[0].name` |
| `instructions`\* | string or null | Text | `a.models[0].parameters.prompt` |
| `tools`\* | array | TBA | `a.tools` |
| `file_ids`\* | array | TBA | `a.files` |
| `metadata`\* | map | TBA | `a.metadata` |
| `models` | array | TBA | `a.models` |
### Create Assistant
#### `POST /v1/assistants`
- https://platform.openai.com/docs/api-reference/assistants/createAssistant
### Retrieve Assistant
#### `GET v1/assistants/{assistant_id}`
- https://platform.openai.com/docs/api-reference/assistants/getAssistant
### Modify Assistant
#### `POST v1/assistants/{assistant_id}`
- https://platform.openai.com/docs/api-reference/assistants/modifyAssistant
### Delete Assistant
#### `DELETE v1/assistants/{assistant_id}`
- https://platform.openai.com/docs/api-reference/assistants/deleteAssistant
### List Assistants
#### `GET v1/assistants`
- https://platform.openai.com/docs/api-reference/assistants/listAssistants
### CRUD Assistant.Models
- This is a Jan-only endpoint, since Jan supports the ModelPicker, i.e. an `assistant` can be created to run with many `models`.
#### `POST /v1/assistants/{assistant_id}/models`
#### `GET /v1/assistants/{assistant_id}/models`
#### `GET /v1/assistants/{assistant_id}/models/{model_id}`
#### `DELETE /v1/assistants/{assistant_id}/models`
Note: There's no need to implement `Modify Assistant.Models`

147
docs/docs/docs/specs/threads.md
View File

@ -2,52 +2,125 @@
title: "Threads"
---
Threads contain `messages` history with assistants. Messages in a thread share context.
## User Stories
- Note: For now, threads "lock the model" after a `message` is sent
- When a new `thread` is created with Jan, users can choose the models
- Users can still edit model parameters/system prompts
- Note: future Assistants may customize this behavior
- Note: Assistants will be able to specify default thread location in the future
- Jan uses root-level threads, to allow for future multi-assistant threads
- Assistant Y may store threads in its own folder, to allow for [long-term assistant memory](https://github.com/janhq/jan/issues/344)
_Users can chat with an assistant in a thread_
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads
- See [Messages Spec]
## Thread Object
_Users can change model in a new thread_
- `thread.json`
- Equivalent to: https://platform.openai.com/docs/api-reference/threads/object
- Wireframes here
_Users can change model parameters in a thread_
- Wireframes of
_Users can delete all thread history_
- Wireframes of settings page.
## Jan Thread Object
- A `Jan Thread Object` is a "representation of a conversation thread" between an `assistant` and the user
- Objects are defined by `thread-uuid.json` files in `json` format
- Objects are designed to be compatible with `OpenAI Thread Objects` with additional properties needed to run on our infrastructure.
- Objects contain a `models` field, to track when the user overrides the assistant's default model parameters.
| Property | Type | Description | Validation |
| ---------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
| `object` | enum: `model`, `assistant`, `thread`, `message` | The Jan Object type | Defaults to `thread` |
| `models` | array | An array of Jan Model Objects. Threads can "override" an assistant's model run parameters. Thread-level model parameters are directly saved in the `thread.models` property! [See `model` spec]() | Defaults to `assistant.models` |
| `messages` | array | An array of Jan Message Objects. [See `message` spec](https://hackmd.io/7a4Tcaa6QDurEzMY9g8HNg) | Defaults to `[]` |
| `metadata` | map | Useful for storing additional information about the object in a structured format. | Defaults to `{}` |
### Generic Example
```json
{
// Jan specific properties:
"summary": "HCMC restaurant recommendations",
"messages": {see below}
// OpenAI compatible properties: https://platform.openai.com/docs/api-reference/threads)
"id": "thread_abc123",
"object": "thread",
"created_at": 1698107661,
"metadata": {}
}
// janroot/threads/jan_1700123404.json
"messages": [
{...message0}, {...message1}
],
"metadata": {
"summary": "funny physics joke",
},
```
## Threads API
## Filesystem
- Equivalent to: https://platform.openai.com/docs/api-reference/threads
```sh=
POST https://localhost:1337/v1/threads/{thread_id} # Create thread
GET https://localhost:1337/v1/threads/{thread_id} # Get thread
DELETE https://localhost:1337/v1/models/{thread_id} # Delete thread
```
## Threads Filesystem
- `Jan Thread Objects`' `json` files always has the naming schema: `assistant_uuid` + `unix_time_thread_created_at. See below.
- Threads are all saved in the `janroot/threads` folder in a flat folder structure.
- The folder is standalone and can be easily zipped, exported, and cleared.
```sh
/assistants
/homework-helper
/threads # context is "permanently remembered" by assistant in future conversations
/threads # context is only retained within a single thread
janroot/
threads/
jan_1700123404.json
homework_helper_700120003.json
```
## Jan API
### Thread API Object
#### `GET /v1/threads/{thread_id}`
- The `Jan Thread Object` maps into the `OpenAI Thread Object`.
- Properties marked with `*` are compatible with the [OpenAI `thread` object](https://platform.openai.com/docs/api-reference/threads)
- Note: The `Jan Thread Object` has additional properties when retrieved via its API endpoint.
- https://platform.openai.com/docs/api-reference/threads/getThread
| Property | Type | Public Description | Jan Thread Object (`t`) Property |
| -------------- | ------- | ------------------------------------------------------------------- | -------------------------------- |
| `id`\* | string | Thread uuid, also the name of the Jan Thread Object file: `id.json` | `json` filename |
| `object`\* | string | Always "thread" | `t.object` |
| `created_at`\* | integer | | `json` file creation time |
| `metadata`\* | map | | `t.metadata` |
| `models` | array | | `t.models` |
| `messages` | array | | `t.messages` |
### Create Thread
#### `POST /v1/threads`
- https://platform.openai.com/docs/api-reference/threads/createThread
### Retrieve Thread
#### `GET v1/threads/{thread_id}`
- https://platform.openai.com/docs/api-reference/threads/getThread
### Modify Thread
#### `POST v1/threads/{thread_id}`
- https://platform.openai.com/docs/api-reference/threads/modifyThread
### Delete Thread
#### `DELETE v1/threads/{thread_id}`
- https://platform.openai.com/docs/api-reference/threads/deleteThread
### List Threads
> This is a Jan-only endpoint, not supported by OAI yet.
#### `GET v1/threads`
### Get & Modify `Thread.Models`
> This is a Jan-only endpoint, not supported by OAI yet.
#### `GET v1/threads/{thread_id}/models`
#### `POST v1/threads/{thread_id}/models/{model_id}`
- Since users can change model parameters in an existing thread
### List `Thread.Messages`
> This is a Jan-only endpoint, not supported by OAI yet.
#### `GET v1/threads/{thread_id}/messages`