docs: chats
This commit is contained in:
parent
50fc20858d
commit
53bbfc5bce
@ -2,15 +2,29 @@
|
||||
title: Chats
|
||||
---
|
||||
|
||||
:::warning
|
||||
:::caution
|
||||
|
||||
This page is still under construction, and should be read as a scratchpad
|
||||
This is currently under development.
|
||||
|
||||
:::
|
||||
|
||||
Chats are essentially inference requests to a model
|
||||
## Overview
|
||||
|
||||
> OpenAI Equivalent: https://platform.openai.com/docs/api-reference/chat
|
||||
In Jan, `chats` are LLM responses in the form of OpenAI compatible `chat completion objects`.
|
||||
|
||||
- This should reference Nitro ChatCompletion API page to reduce duplication.
|
||||
- We are fine with adding Jan API for this but it makes sense to use Nitro as reference as Nitro is default inference engine for Jan in this release
|
||||
- Models take a list of messages and return a model-generated response as output.
|
||||
- An [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat) compatible endpoint at `localhost:3000/v1/chats`.
|
||||
|
||||
## Folder Structure
|
||||
|
||||
Chats are stateless, thus are not saved in `janroot`. Any content and relevant metadata from calling this endpoint is extracted and persisted through [Messages](/specs/messages).
|
||||
|
||||
## API Reference
|
||||
|
||||
Jan's Chat API is compatible with [OpenAI's Chat API](https://platform.openai.com/docs/api-reference/chat).
|
||||
|
||||
See [Jan Chat API](https://jan.ai/api-reference/#tag/Chat-Completion)
|
||||
|
||||
## Implementation
|
||||
|
||||
Under the hood, the `/chat` endpoint simply reroutes an existing endpoint from [Nitro server](https://nitro.jan.ai). Nitro is a lightweight & local inference server, written in C++ and embedded into the Jan app. See [Nitro documentation](https://nitro.jan.ai/docs).
|
||||
|
||||
@ -86,4 +86,4 @@ Here's an example `message` response from an assistant.
|
||||
|
||||
Jan's `messages` API is compatible with [OpenAI's Messages API](https://platform.openai.com/docs/api-reference/messages), with additional methods for managing messages locally.
|
||||
|
||||
See [Jan Messages API](https://jan.ai/api-reference#tag/Messages)
|
||||
See [Jan Messages API](https://jan.ai/api-reference#tag/Messages).
|
||||
|
||||
@ -86,7 +86,7 @@ Here's a standard example `model.json` for a GGUF model.
|
||||
|
||||
Jan's Model API is compatible with [OpenAI's Models API](https://platform.openai.com/docs/api-reference/models), with additional methods for managing and running models locally.
|
||||
|
||||
See [Jan Models API](https://jan.ai/api-reference#tag/Models)
|
||||
See [Jan Models API](https://jan.ai/api-reference#tag/Models).
|
||||
|
||||
## Importing Models
|
||||
|
||||
|
||||
@ -62,4 +62,4 @@ Here's a standard example `thread.json` for a conversation between the user and
|
||||
|
||||
Jan's Threads API is compatible with [OpenAI's Threads API](https://platform.openai.com/docs/api-reference/threads), with additional methods for managing threads locally.
|
||||
|
||||
See [Jan Threads API](https://jan.ai/api-reference#tag/Threads)
|
||||
See [Jan Threads API](https://jan.ai/api-reference#tag/Threads).
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user