* refactor: move session management & port allocation to backend
- Remove the in‑process `activeSessions` map and its cleanup logic from the TypeScript side.
- Introduce new Tauri commands in Rust:
- `get_random_port` – picks an unused port using a seeded RNG and checks availability.
- `find_session_by_model` – returns the `SessionInfo` for a given model ID.
- `get_loaded_models` – returns a list of currently loaded model IDs.
- Update the extension’s TypeScript code to use these commands via `invoke`:
- `findSessionByModel`, `load`, `unload`, `chat`, `getLoadedModels`, and `embed` now operate asynchronously and query the backend.
- Remove the old `is_port_available` command and the custom port‑checking loop.
- Simplify `onUnload` – session termination is now handled by the backend.
- Drop unused helpers (`sleep`, `waitForModelLoad`) and related port‑availability code.
- Add missing Rust imports (`rand::{StdRng,Rng,SeedableRng}`, `HashSet`) and improve error handling.
- Register the new commands in `src-tauri/src/lib.rs` (replace `is_port_available` with the three new commands).
This refactor centralises session state and port allocation in the Rust backend, eliminates duplicated logic, and resolves race conditions around model loading and session cleanup.
* Use String(e) for error
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
This change improves the robustness of the llama.cpp extension's server port selection.
Previously, the `getRandomPort()` method only checked for ports already in use by active sessions, which could lead to model load failures if the chosen port was occupied by another external process.
This change introduces a new Tauri command, `is_port_available`, which performs a system-level check to ensure the randomly selected port is truly free before attempting to start the llama-server. It also adds a retry mechanism with a maximum number of attempts (20,000) to find an available port, throwing an error if no suitable port is found within the specified range after all attempts.
This enhancement prevents port conflicts and improves the reliability and user experience of the llama.cpp extension within Jan.
Closes#5965
* feat: add support for querying available backend devices
This change introduces a new `get_devices` method to the `llamacpp_extension` engine that allows the frontend to query and display a list of available devices (e.g., Vulkan, CUDA, SYCL) from the compiled `llama-server` binary.
* Added `DeviceList` interface to represent GPU/device metadata.
* Implemented `getDevices(): Promise<DeviceList[]>` method.
* Splits `version/backend`, ensures backend is ready.
* Invokes the new Tauri command `get_devices`.
* Introduced a new `get_devices` Tauri command.
* Parses `llama-server --list-devices` output to extract available devices with memory info.
* Introduced `DeviceInfo` struct (`id`, `name`, `mem`, `free`) and exposed it via serialization.
* Robust parsing logic using string processing (non-regex) to locate memory stats.
* Registered the new command in the `tauri::Builder` in `lib.rs`.
* Fixed logic to correctly parse multiple devices from the llama-server output.
* Handles common failure modes: binary not found, malformed memory info, etc.
This sets the foundation for device selection, memory-aware model loading, and improved diagnostics in Jan AI engine setup flows.
* Update extensions/llamacpp-extension/src/index.ts
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Change the llama_server_process state from an Option<Child> to a HashMap<String, Child> to support managing multiple server instances by PID. This allows precise process tracking and termination, replacing the previous single-process limitation.
Previously, only one server process could be tracked at a time. Now, each process is stored with its PID as the key, enabling:
- Accurate session matching during unloading
- Proper termination of specific processes
- Better error handling for mismatched PIDs
The load_llama_model function now inserts processes into the map, and unload_llama_model removes them by PID.
* wip
* update
* add download logic
* add decompress. support delete file
* download backend upon selecting setting
* add some logging and nootes
* add note on race condition
* remove then catch
* default to none backend. only download if it's not installed
* merge version and backend. fetch version from GH
* restrict scope of output_dir
* add note on unpack
This commit introduces API key generation for the Llama.cpp extension. The API key is now generated on the server side using HMAC-SHA256 and a secret key to ensure security and uniqueness. The frontend now passes the model ID and API secret to the server to generate the key. This addresses the requirement for secure model access and authorization.
* add pull and abortPull
* add model import (download only)
* write model.yaml. support local model import
* remove cortex-related command
* add TODO
* remove cortex-related command
* feat: implement retry mechanism for MCP server activation with exponential backoff
feat: enhance MCP server activation with configurable retry attempts
feat: implement MCP server restart monitoring and cleanup functionality
feat: enhance MCP server restart logic with improved monitoring and configuration handling
feat: add manual deactivation for MCP servers to prevent automatic restarts
* feat: enhance MCP server startup with initial attempt tracking and health monitoring
* chore: remove legacy themes
* refactor: clean up dependencies
* chore: remove cuda 11 dependency - fix linux LD_LIBRARY_PATH
* fix: load models issue on Linux
# Conflicts:
# src-tauri/src/core/setup.rs
* chore: do not download cuda 11 by default
* chore: remove cuda 11 from installer
* fix: cuda lookup on Linux
* enhancement: ux change data folder with confirmation and reveal in finder logs
* chore: update button open logs local api server
* Update web-app/src/components/ui/button.tsx
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* chore: handle error when change location data folder failed
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* feat: migrate legacy local storage data to new app
* chore: refactor localstorage db read
* chore: clean up
* chore: migrate api key setting
* chore: apply proxy configs
* chore: fix key
* chore: enhance tool use loop
* fix: create new custom provider is not saved
* chore: bump llama.cpp b5488
* chore: normalize reasoning assistant response
* chore: fix tool call parse in stream mode
* fix: give tool call default generated id
* fix: system instruction should be on top of the history
* chore: allow users to add parameters
* feat: Implement Cortex server auto-restart and webview notification
Implements a robust auto-restart mechanism for the Cortex server (sidecar)
managed by the Tauri backend.
Key changes:
Backend (src-tauri):
- Modified `core/setup.rs` to:
- Loop sidecar spawning, attempting up to `MAX_RESTARTS` (5) times with a
`RESTART_DELAY_MS` (5 seconds) between attempts.
- Monitor the sidecar process for unexpected termination (crashes or
non-zero exit codes).
- Reset the restart attempt count to 0 in `AppState` upon a successful
server spawn.
- Emit a "cortex_max_restarts_reached" event to the webview if the
server fails to start after `MAX_RESTARTS`.
- Updated `core/state.rs` to include `cortex_restart_count: Arc<Mutex<u32>>`
in `AppState` to track restart attempts.
- Added a new Tauri command `reset_cortex_restart_count` in `core/cmd.rs`
to allow the webview (or other parts of the app) to reset this counter.
- Registered the new command and initialized the `cortex_restart_count`
in `lib.rs`.
Frontend (web-app):
- Created a new component `CortexFailureDialog.tsx` in
`src/containers/dialogs/` to:
- Listen for the "cortex_max_restarts_reached" event from Tauri.
- Display a dialog informing the user that the local AI engine (Cortex)
failed to start after multiple attempts.
- Offer options to "Contact Support" (opens jan.ai/support),
"Restart Jan" (invokes the `relaunch` Tauri command), or "Okay"
(dismisses the dialog).
- Integrated the `CortexFailureDialog` into the `RootLayout` in
`src/routes/__root.tsx` so it's globally available.
- Corrected button variants in `__root.tsx` to use `variant="default"`
with appropriate classNames for outline styling, resolving TypeScript
errors.
* refactor: Improve async handling and logging in setup_sidecar function
