Akarshan Biswas b1984a452e
Fix: Llama.cpp server hangs on model load (#6030)
* Fix: Llama.cpp server hangs on model load

Resolves an issue where the llama.cpp server would hang indefinitely when loading certain models, as described in the attached ticket. The server's readiness message was not being correctly detected, causing the application to stall.

The previous implementation used a line-buffered reader (BufReader::lines()) to process the stderr stream. This method proved to be unreliable for the specific output of the llama.cpp server.

This commit refactors the stderr handling logic to use a more robust, chunk-based approach (read_until(b'\n', ...)). This ensures that the output is processed as it arrives, reliably capturing critical status messages and preventing the application from hanging during model initialization.

Fixes: #6021

* Handle error gracefully with ServerError

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Revert "Handle error gracefully with ServerError"

This reverts commit 267a8a8a3262fbe36a445a30b8b3ba9a39697643.

* Revert "Fix: Llama.cpp server hangs on model load"

This reverts commit 44e5447f82f0ae32b6db7ffb213025f130d655c4.

* Add more guards, refactor and fix error sending to FE

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-02 21:50:07 +05:30
2025-07-31 18:52:01 +10:00
2025-07-10 21:14:21 +07:00
2025-03-18 13:06:17 +07:00
2023-10-30 23:20:10 +07:00
2025-03-18 13:06:17 +07:00
2023-12-29 11:30:16 +07:00
2024-10-28 23:09:25 +07:00
2025-07-15 22:29:28 +07:00
2025-07-31 15:41:44 +07:00

Jan - Local AI Assistant

Jan banner

GitHub commit activity Github Last Commit Github Contributors GitHub closed issues Discord

Getting Started - Docs - Changelog - Bug reports - Discord

Jan is a ChatGPT-alternative that runs 100% offline on your device. Our goal is to make it easy for a layperson to download and run LLMs and use AI with full control and privacy.

⚠️ Jan is in active development.

Installation

Because clicking a button is still the easiest way to get started:

Platform Stable Beta Nightly
Windows jan.exe jan.exe jan.exe
macOS jan.dmg jan.dmg jan.dmg
Linux (deb) jan.deb jan.deb jan.deb
Linux (AppImage) jan.AppImage jan.AppImage jan.AppImage

Download from jan.ai or GitHub Releases.

Demo

Features

  • Local AI Models: Download and run LLMs (Llama, Gemma, Qwen, etc.) from HuggingFace
  • Cloud Integration: Connect to OpenAI, Anthropic, Mistral, Groq, and others
  • Custom Assistants: Create specialized AI assistants for your tasks
  • OpenAI-Compatible API: Local server at localhost:1337 for other applications
  • Model Context Protocol: MCP integration for enhanced capabilities
  • Privacy First: Everything runs locally when you want it to

Build from Source

For those who enjoy the scenic route:

Prerequisites

  • Node.js ≥ 20.0.0
  • Yarn ≥ 1.22.0
  • Make ≥ 3.81
  • Rust (for Tauri)

Run with Make

git clone https://github.com/menloresearch/jan
cd jan
make dev

This handles everything: installs dependencies, builds core components, and launches the app.

Available make targets:

  • make dev - Full development setup and launch
  • make build - Production build
  • make test - Run tests and linting
  • make clean - Delete everything and start fresh

Run with Mise (easier)

You can also run with mise, which is a bit easier as it ensures Node.js, Rust, and other dependency versions are automatically managed:

git clone https://github.com/menloresearch/jan
cd jan

# Install mise (if not already installed)
curl https://mise.run | sh

# Install tools and start development
mise install    # installs Node.js, Rust, and other tools
mise dev        # runs the full development setup

Available mise commands:

  • mise dev - Full development setup and launch
  • mise build - Production build
  • mise test - Run tests and linting
  • mise clean - Delete everything and start fresh
  • mise tasks - List all available tasks

Manual Commands

yarn install
yarn build:core
yarn build:extensions
yarn dev

System Requirements

Minimum specs for a decent experience:

  • macOS: 13.6+ (8GB RAM for 3B models, 16GB for 7B, 32GB for 13B)
  • Windows: 10+ with GPU support for NVIDIA/AMD/Intel Arc
  • Linux: Most distributions work, GPU acceleration available

For detailed compatibility, check our installation guides.

Troubleshooting

When things go sideways (they will):

  1. Check our troubleshooting docs
  2. Copy your error logs and system specs
  3. Ask for help in our Discord #🆘|jan-help channel

We keep logs for 24 hours, so don't procrastinate on reporting issues.

Contributing

Contributions welcome. See CONTRIBUTING.md for the full spiel.

Contact

Trust & Safety

Friendly reminder: We're not trying to scam you.

  • We won't ask for personal information
  • Jan is completely free (no premium version exists)
  • We don't have a cryptocurrency or ICO
  • We're bootstrapped and not seeking your investment (yet)

License

Apache 2.0 - Because sharing is caring.

Acknowledgements

Built on the shoulders of giants:

Description
Languages
TypeScript 54.9%
JavaScript 34.1%
Rust 8.6%
Python 1.5%
Shell 0.4%
Other 0.5%