docs: chats

This commit is contained in:
0xSage 2023-11-24 16:55:15 +08:00
parent 50fc20858d
commit 53bbfc5bce
4 changed files with 23 additions and 9 deletions

View File

@ -2,15 +2,29 @@
title: Chats title: Chats
--- ---
:::warning :::caution
This page is still under construction, and should be read as a scratchpad This is currently under development.
::: :::
Chats are essentially inference requests to a model ## Overview
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/chat In Jan, `chats` are LLM responses in the form of OpenAI compatible `chat completion objects`.
- This should reference Nitro ChatCompletion API page to reduce duplication. - Models take a list of messages and return a model-generated response as output.
- We are fine with adding Jan API for this but it makes sense to use Nitro as reference as Nitro is default inference engine for Jan in this release - An [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat) compatible endpoint at `localhost:3000/v1/chats`.
## Folder Structure
Chats are stateless, thus are not saved in `janroot`. Any content and relevant metadata from calling this endpoint is extracted and persisted through [Messages](/specs/messages).
## API Reference
Jan's Chat API is compatible with [OpenAI's Chat API](https://platform.openai.com/docs/api-reference/chat).
See [Jan Chat API](https://jan.ai/api-reference/#tag/Chat-Completion)
## Implementation
Under the hood, the `/chat` endpoint simply reroutes an existing endpoint from [Nitro server](https://nitro.jan.ai). Nitro is a lightweight & local inference server, written in C++ and embedded into the Jan app. See [Nitro documentation](https://nitro.jan.ai/docs).

View File

@ -86,4 +86,4 @@ Here's an example `message` response from an assistant.
Jan's `messages` API is compatible with [OpenAI's Messages API](https://platform.openai.com/docs/api-reference/messages), with additional methods for managing messages locally. Jan's `messages` API is compatible with [OpenAI's Messages API](https://platform.openai.com/docs/api-reference/messages), with additional methods for managing messages locally.
See [Jan Messages API](https://jan.ai/api-reference#tag/Messages) See [Jan Messages API](https://jan.ai/api-reference#tag/Messages).

View File

@ -86,7 +86,7 @@ Here's a standard example `model.json` for a GGUF model.
Jan's Model API is compatible with [OpenAI's Models API](https://platform.openai.com/docs/api-reference/models), with additional methods for managing and running models locally. Jan's Model API is compatible with [OpenAI's Models API](https://platform.openai.com/docs/api-reference/models), with additional methods for managing and running models locally.
See [Jan Models API](https://jan.ai/api-reference#tag/Models) See [Jan Models API](https://jan.ai/api-reference#tag/Models).
## Importing Models ## Importing Models

View File

@ -62,4 +62,4 @@ Here's a standard example `thread.json` for a conversation between the user and
Jan's Threads API is compatible with [OpenAI's Threads API](https://platform.openai.com/docs/api-reference/threads), with additional methods for managing threads locally. Jan's Threads API is compatible with [OpenAI's Threads API](https://platform.openai.com/docs/api-reference/threads), with additional methods for managing threads locally.
See [Jan Threads API](https://jan.ai/api-reference#tag/Threads) See [Jan Threads API](https://jan.ai/api-reference#tag/Threads).