* feat: Prompt progress when streaming
- BE changes:
- Add a `return_progress` flag to `chatCompletionRequest` and a corresponding `prompt_progress` payload in `chatCompletionChunk`. Introduce `chatCompletionPromptProgress` interface to capture cache, processed, time, and total token counts.
- Update the Llamacpp extension to always request progress data when streaming, enabling UI components to display real‑time generation progress and leverage llama.cpp’s built‑in progress reporting.
* Make return_progress optional
* chore: update ui prompt progress before streaming content
* chore: remove log
* chore: remove progress when percentage >= 100
* chore: set timeout prompt progress
* chore: move prompt progress outside streaming content
* fix: tests
---------
Co-authored-by: Faisal Amir <urmauur@gmail.com>
Co-authored-by: Louis <louis@jan.ai>
@janhq/core
This module includes functions for communicating with core APIs, registering app extensions, and exporting type definitions.
Usage
Import the package
// Web / extension runtime
import * as core from '@janhq/core'
Build an Extension
-
Download an extension template, for example, https://github.com/menloresearch/extension-template.
-
Update the source code:
-
Open
index.tsin your code editor. -
Rename the extension class from
SampleExtensionto your preferred extension name. -
Import modules from the core package.
import * as core from '@janhq/core' -
In the
onLoad()method, add your code:// Example of listening to app events and providing customized inference logic: import * as core from '@janhq/core' export default class MyExtension extends BaseExtension { // On extension load onLoad() { core.events.on(MessageEvent.OnMessageSent, (data) => MyExtension.inference(data, this)) } // Customized inference logic private static inference(incomingMessage: MessageRequestData) { // Prepare customized message content const content: ThreadContent = { type: ContentType.Text, text: { value: "I'm Jan Assistant!", annotations: [], }, } // Modify message and send out const outGoingMessage: ThreadMessage = { ...incomingMessage, content, } } }
-
-
Build the extension:
- Navigate to the extension directory.
- Install dependencies.
yarn install - Compile the source code. The following command keeps running in the terminal and rebuilds the extension when you modify the source code.
yarn build - Select the generated .tgz from Jan > Settings > Extension > Manual Installation.