Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #681 from janhq/threads

Browse Source 
docs: threads and messages
This commit is contained in:
0xSage 2023-11-22 19:08:00 +08:00 committed by GitHub
commit 5a9e39e68e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 302 additions and 188 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

154
docs/docs/specs/messages.md
View File

@ -2,31 +2,79 @@
title: Messages title: Messages
--- ---
:::warning :::caution
Draft Specification: functionality has not been implemented yet. This is currently under development.
Feedback: [HackMD: Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
::: :::
Messages are within `threads` and capture additional metadata. ## Overview
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/messages
`Messages` capture a conversation's content. This can include the content from LLM responses and other metadata from [chat completions](/specs/chats).
- Users and assistants can send multimedia messages.
- An [OpenAI Message API](https://platform.openai.com/docs/api-reference/messages) compatible endpoint at `localhost:3000/v1/messages`.
## Folder Structure
Messages are saved in the `/threads/{thread_id}` folder in `messages.jsonl` files
```sh
jan/
threads/
assistant_name_unix_timestamp/
...
messages.jsonl
jan_2341243134/
...
messages.jsonl
```
## `message.jsonl`
Individual messages are saved in `jsonl` format for indexing purposes.
## Message Object
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/messages/object
```json ```json
{ {...message_2}
// Jan specific properties {...message_1}
"updatedAt": "...", // that's it I think {...message_0}
```
// OpenAI compatible properties: https://platform.openai.com/docs/api-reference/messages) ### Examples
"id": "msg_dKYDWyQvtjDBi3tudL1yWKDa",
"object": "thread.message", Here's a standard example `message` sent from a user.
"created_at": 1698983503,
"thread_id": "thread_RGUhOuO9b2nrktrmsQ2uSR6I", ```json
"role": "assistant", "id": "0", // Sequential or UUID
"content": [ "object": "thread.message", // Defaults to "thread.message"
"created_at": 1698983503,
"thread_id": "thread_asdf", // Defaults to parent thread
"assistant_id": "jan", // Defaults to parent thread
"role": "user", // From either "user" or "assistant"
"content": [
{
"type": "text",
"text": {
"value": "Hi!?",
"annotations": []
}
}
],
"metadata": {}, // Defaults to {}
// "run_id": "...", // Rather than `run` id abstraction
// "file_ids": [],
```
Here's an example `message` response from an assistant.
```json
"id": "0", // Sequential or UUID
"object": "thread.message", // Defaults to "thread.message"
"created_at": 1698983503,
"thread_id": "thread_asdf", // Defaults to parent thread
"assistant_id": "jan", // Defaults to parent thread
"role": "assistant", // From either "user" or "assistant"
"content": [ // Usually from Chat Completion obj
{ {
"type": "text", "type": "text",
"text": { "text": {
@ -34,25 +82,34 @@ Messages are within `threads` and capture additional metadata.
"annotations": [] "annotations": []
} }
} }
], ],
"file_ids": [], "metadata": {}, // Defaults to {}
"assistant_id": "asst_ToSF7Gb04YMj8AMMm50ZLLtY", // "run_id": "...", // KIV
"run_id": "run_BjylUJgDqYK9bOhy4yjAiMrn", // "file_ids": [], // KIV
"metadata": {} // "usage": {} // KIV: saving chat completion properties https://platform.openai.com/docs/api-reference/chat/object
}
``` ```
## Messages API ## API Reference
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/messages
Jan's `messages` API is compatible with [OpenAI's Messages API](https://platform.openai.com/docs/api-reference/messages), with additional methods for managing messages locally.
See [Jan Messages API](https://jan.ai/api-reference#tag/Messages)
<!-- TODO clean this part up into API -->
<!--
### Get list message ### Get list message
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/messages/getMessage > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/messages/getMessage
- Example request - Example request
```shell ```shell
curl {JAN_URL}/v1/threads/{thread_id}/messages/{message_id} \ curl {JAN_URL}/v1/threads/{thread_id}/messages/{message_id} \
-H "Content-Type: application/json" -H "Content-Type: application/json"
``` ```
- Example response - Example response
```json ```json
{ {
"id": "msg_abc123", "id": "msg_abc123",
@ -75,9 +132,13 @@ Messages are within `threads` and capture additional metadata.
"metadata": {} "metadata": {}
} }
``` ```
### Create message ### Create message
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/messages/createMessage > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/messages/createMessage
- Example request - Example request
```shell ```shell
curl -X POST {JAN_URL}/v1/threads/{thread_id}/messages \ curl -X POST {JAN_URL}/v1/threads/{thread_id}/messages \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
@ -86,9 +147,11 @@ Messages are within `threads` and capture additional metadata.
"content": "How does AI work? Explain it in simple terms." "content": "How does AI work? Explain it in simple terms."
}' }'
``` ```
- Example response - Example response
```json ```json
{ {
"id": "msg_abc123", "id": "msg_abc123",
"object": "thread.message", "object": "thread.message",
"created_at": 1699017614, "created_at": 1699017614,
@ -107,18 +170,24 @@ Messages are within `threads` and capture additional metadata.
"assistant_id": null, "assistant_id": null,
"run_id": null, "run_id": null,
"metadata": {} "metadata": {}
} }
``` ```
### Get message ### Get message
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/assistants/listAssistants > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/assistants/listAssistants
- Example request - Example request
```shell ```shell
curl {JAN_URL}/v1/threads/{thread_id}/messages/{message_id} \ curl {JAN_URL}/v1/threads/{thread_id}/messages/{message_id} \
-H "Content-Type: application/json" -H "Content-Type: application/json"
``` ```
- Example response - Example response
```json ```json
{ {
"id": "msg_abc123", "id": "msg_abc123",
"object": "thread.message", "object": "thread.message",
"created_at": 1699017614, "created_at": 1699017614,
@ -137,42 +206,53 @@ Messages are within `threads` and capture additional metadata.
"assistant_id": null, "assistant_id": null,
"run_id": null, "run_id": null,
"metadata": {} "metadata": {}
} }
``` ```
### Modify message ### Modify message
> Jan: TODO: Do we need to modify message? Or let user create new message? > Jan: TODO: Do we need to modify message? Or let user create new message?
# Get message file # Get message file
> OpenAI Equivalent: https://api.openai.com/v1/threads/{thread_id}/messages/{message_id}/files/{file_id} > OpenAI Equivalent: https://api.openai.com/v1/threads/{thread_id}/messages/{message_id}/files/{file_id}
- Example request - Example request
```shell ```shell
curl {JAN_URL}/v1/threads/{thread_id}/messages/{message_id}/files/{file_id} \ curl {JAN_URL}/v1/threads/{thread_id}/messages/{message_id}/files/{file_id} \
-H "Content-Type: application/json" -H "Content-Type: application/json"
``` ```
- Example response - Example response
```json ```json
{ {
"id": "file-abc123", "id": "file-abc123",
"object": "thread.message.file", "object": "thread.message.file",
"created_at": 1699061776, "created_at": 1699061776,
"message_id": "msg_abc123" "message_id": "msg_abc123"
} }
``` ```
# List message files # List message files
> OpenAI Equivalent: https://api.openai.com/v1/threads/{thread_id}/messages/{message_id}/files > OpenAI Equivalent: https://api.openai.com/v1/threads/{thread_id}/messages/{message_id}/files
```
````
- Example request - Example request
```shell ```shell
curl {JAN_URL}/v1/threads/{thread_id}/messages/{message_id}/files/{file_id} \ curl {JAN_URL}/v1/threads/{thread_id}/messages/{message_id}/files/{file_id} \
-H "Content-Type: application/json" -H "Content-Type: application/json"
``` ````
- Example response - Example response
```json ```json
{ {
"id": "file-abc123", "id": "file-abc123",
"object": "thread.message.file", "object": "thread.message.file",
"created_at": 1699061776, "created_at": 1699061776,
"message_id": "msg_abc123" "message_id": "msg_abc123"
} }
``` ``` -->

12
docs/docs/specs/models.md
View File

@ -4,7 +4,7 @@ title: Models
:::caution :::caution
Draft Specification: functionality has not been implemented yet. This is currently under development.
::: :::
@ -46,19 +46,19 @@ jan/ # Jan root folder
- `model.json` contains metadata and default parameters used to run a model. - `model.json` contains metadata and default parameters used to run a model.
- The only required field is `source_url`. - The only required field is `source_url`.
### GGUF Example ### Example
Here's a standard example `model.json` for a GGUF model. Here's a standard example `model.json` for a GGUF model.
- `source_url`: https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/. - `source_url`: https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/.
```json ```json
"source_url": "https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf",
"type": "model", // Defaults to "model"
"version": "1", // Defaults to 1
"id": "zephyr-7b" // Defaults to foldername "id": "zephyr-7b" // Defaults to foldername
"object": "model", // Defaults to "model"
"source_url": "https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/blob/main/zephyr-7b-beta.Q4_K_M.gguf",
"name": "Zephyr 7B" // Defaults to foldername "name": "Zephyr 7B" // Defaults to foldername
"owned_by": "you" // Defaults to you "owned_by": "you" // Defaults to you
"version": "1", // Defaults to 1
"created": 1231231 // Defaults to file creation time "created": 1231231 // Defaults to file creation time
"description": "" "description": ""
"state": enum[null, "downloading", "ready", "starting", "stopping", ...] "state": enum[null, "downloading", "ready", "starting", "stopping", ...]
@ -95,7 +95,7 @@ See [Jan Models API](https://jan.ai/api-reference#tag/Models)
:::caution :::caution
This is current under development. This is currently under development.
::: :::

166
docs/docs/specs/threads.md
View File

@ -2,89 +2,101 @@
title: Threads title: Threads
--- ---
:::warning :::caution
Draft Specification: functionality has not been implemented yet. This is currently under development.
Feedback: [HackMD: Threads Spec](https://hackmd.io/BM_8o_OCQ-iLCYhunn2Aug)
::: :::
## User Stories ## Overview
_Users can chat with an assistant in a thread_ `Threads` are conversations between an `assistant` and the user:
- See [Messages Spec](./messages.md) - Users can tweak `model` params and `assistant` behavior within each thread.
- Users can import and export threads.
- An [OpenAI Thread API](https://platform.openai.com/docs/api-reference/threads) compatible endpoint at `localhost:3000/v1/threads`.
_Users can change assistant and model parameters in a thread_ ## Folder Structure
- Wireframes of - Threads are saved in the `/threads` folder.
- Threads are organized by folders, one for each thread, and can be easily zipped, exported, and cleared.
_Users can delete all thread history_ - Thread folders follow the naming: `assistant_id` + `thread_created_at`.
- Thread folders also contain `messages.jsonl` files. See [messages](/specs/messages).
- Wireframes of settings page.
## Jan Thread Object
- A `Jan Thread Object` is a "representation of a conversation thread" between an `assistant` and the user
- Objects are defined by `thread-uuid.json` files in `json` format
- Objects are designed to be compatible with `OpenAI Thread Objects` with additional properties needed to run on our infrastructure.
- Objects contain a `models` field, to track when the user overrides the assistant's default model parameters.
| Property | Type | Description | Validation |
| ---------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
| `object` | enum: `model`, `assistant`, `thread`, `message` | The Jan Object type | Defaults to `thread` |
| `assistants` | array | An array of Jan Assistant Objects. Threads can "override" an assistant's parameters. Thread-level model parameters are directly saved in the `thread.models` property! (see Models spec) | Defaults to `assistant.name` |
| `messages` | array | An array of Jan Message Objects. (see Messages spec) | Defaults to `[]` |
| `metadata` | map | Useful for storing additional information about the object in a structured format. | Defaults to `{}` |
### Generic Example
```json
// janroot/threads/jan_1700123404.json
"assistants": ["assistant-123"],
"messages": [
{...message0}, {...message1}
],
"metadata": {
"summary": "funny physics joke",
},
```
## Filesystem
- `Jan Thread Objects`'s `json` files always has the naming schema: `assistant_uuid` + `unix_time_thread_created_at. See below.
- Threads are all saved in the `janroot/threads` folder in a flat folder structure.
- The folder is standalone and can be easily zipped, exported, and cleared.
```sh ```sh
janroot/ jan/
threads/ threads/
jan_1700123404.json assistant_name_unix_timestamp/
homework_helper_700120003.json thread.json
messages.jsonl
jan_2341243134/
thread.json
``` ```
## Jan API ## `thread.json`
- Each `thread` folder contains a `thread.json` file, which is a representation of a thread.
- `thread.json` contains metadata and model parameter overrides.
- There are no required fields.
### Example
Here's a standard example `thread.json` for a conversation between the user and the default Jan assistant.
```json
"id": "thread_....", // Defaults to foldername
"object": "thread", // Defaults to "thread"
"title": "funny physics joke", // Defaults to ""
"assistants": [
{
"assistant_id": "jan", // Defaults to "jan"
"model": { // Defaults to 1 currently active model (can be changed before thread is begun)
"settings": {}, // Defaults to and overrides assistant.json's "settings" (and if none, then model.json "settings")
"parameters": {}, // Defaults to and overrides assistant.json's "parameters" (and if none, then model.json "parameters")
}
},
],
"created": 1231231 // Defaults to file creation time
"metadata": {}, // Defaults to {}
```
## API Reference
Jan's Threads API is compatible with [OpenAI's Threads API](https://platform.openai.com/docs/api-reference/threads), with additional methods for managing threads locally.
See [Jan Threads API](https://jan.ai/api-reference#tag/Threads)
<!-- TODO clean this part up into API -->
<!--
### Get thread ### Get thread
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads/getThread > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads/getThread
- Example request - Example request
```shell ```shell
curl {JAN_URL}/v1/threads/{thread_id} curl {JAN_URL}/v1/threads/{thread_id}
``` ```
- Example response - Example response
```json ```json
{ {
"id": "thread_abc123", "id": "thread_abc123",
"object": "thread", "object": "thread",
"created_at": 1699014083, "created_at": 1699014083,
"assistants": ["assistant-001"], "assistants": ["assistant-001"],
"metadata": {}, "metadata": {},
"messages": [] "messages": []
} }
``` ```
### Create Thread ### Create Thread
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads/createThread > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads/createThread
- Example request - Example request
```shell ```shell
curl -X POST {JAN_URL}/v1/threads \ curl -X POST {JAN_URL}/v1/threads \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
@ -99,18 +111,24 @@ janroot/
}] }]
}' }'
``` ```
- Example response - Example response
```json ```json
{ {
"id": 'thread_abc123', "id": "thread_abc123",
"object": 'thread', "object": "thread",
"created_at": 1699014083, "created_at": 1699014083,
"metadata": {} "metadata": {}
} }
``` ```
### Modify Thread ### Modify Thread
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads/modifyThread > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads/modifyThread
- Example request - Example request
```shell ```shell
curl -X POST {JAN_URL}/v1/threads/{thread_id} \ curl -X POST {JAN_URL}/v1/threads/{thread_id} \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
@ -125,43 +143,55 @@ janroot/
}] }]
}' }'
``` ```
- Example response - Example response
```json ```json
{ {
"id": 'thread_abc123', "id": "thread_abc123",
"object": 'thread', "object": "thread",
"created_at": 1699014083, "created_at": 1699014083,
"metadata": {} "metadata": {}
} }
``` ```
- https://platform.openai.com/docs/api-reference/threads/modifyThread - https://platform.openai.com/docs/api-reference/threads/modifyThread
### Delete Thread ### Delete Thread
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads/deleteThread > OpenAI Equivalent: https://platform.openai.com/docs/api-reference/threads/deleteThread
- Example request - Example request
```shell ```shell
curl -X DELETE {JAN_URL}/v1/threads/{thread_id} curl -X DELETE {JAN_URL}/v1/threads/{thread_id}
``` ```
- Example response - Example response
```json ```json
{ {
"id": "thread_abc123", "id": "thread_abc123",
"object": "thread.deleted", "object": "thread.deleted",
"deleted": true "deleted": true
} }
``` ```
### List Threads ### List Threads
> This is a Jan-only endpoint, not supported by OAI yet. > This is a Jan-only endpoint, not supported by OAI yet.
- Example request - Example request
```shell ```shell
curl {JAN_URL}/v1/threads \ curl {JAN_URL}/v1/threads \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
``` ```
- Example response - Example response
```json ```json
[ [
{ {
"id": "thread_abc123", "id": "thread_abc123",
"object": "thread", "object": "thread",
@ -175,19 +205,23 @@ janroot/
"object": "thread", "object": "thread",
"created_at": 1699014083, "created_at": 1699014083,
"assistants": ["assistant-002", "assistant-002"], "assistants": ["assistant-002", "assistant-002"],
"metadata": {}, "metadata": {}
} }
] ]
``` ```
### Get & Modify `Thread.Assistants` ### Get & Modify `Thread.Assistants`
-> Can achieve this goal by calling `Modify Thread` API -> Can achieve this goal by calling `Modify Thread` API
#### `GET v1/threads/{thread_id}/assistants` #### `GET v1/threads/{thread_id}/assistants`
-> Can achieve this goal by calling `Get Thread` API -> Can achieve this goal by calling `Get Thread` API
#### `POST v1/threads/{thread_id}/assistants/{assistant_id}` #### `POST v1/threads/{thread_id}/assistants/{assistant_id}`
-> Can achieve this goal by calling `Modify Assistant` API with `thread.assistant[]` -> Can achieve this goal by calling `Modify Assistant` API with `thread.assistant[]`
### List `Thread.Messages` ### List `Thread.Messages`
-> Can achieve this goal by calling `Get Thread` API
-> Can achieve this goal by calling `Get Thread` API -->

4
docs/sidebars.js
View File

@ -61,8 +61,8 @@ const sidebars = {
items: [ items: [
"specs/chats", "specs/chats",
"specs/models", "specs/models",
// "specs/threads", "specs/threads",
// "specs/messages", "specs/messages",
// "specs/assistants", // "specs/assistants",
// "specs/files", // "specs/files",
// "specs/jan", // "specs/jan",