0c5fbc102c
This commit introduces significant improvements to the llama.cpp extension, focusing on the 'Flash Attention' setting and refactoring Tauri plugin interactions for better code clarity and maintenance. The backend interaction is streamlined by removing the unnecessary `libraryPath` argument from the Tauri plugin commands for loading models and listing devices. * **Simplified API Calls:** The `loadLlamaModel`, `unloadLlamaModel`, and `get_devices` functions in both the extension and the Tauri plugin now manage the library path internally based on the backend executable's location. * **Decoupled Logic:** The extension (`src/index.ts`) now uses the new, simplified Tauri plugin functions, which enhances modularity and reduces boilerplate code in the extension. * **Type Consistency:** Added `UnloadResult` interface to `guest-js/index.ts` for consistency. * **Updated UI Control:** The 'Flash Attention' setting in `settings.json` is changed from a boolean checkbox to a string-based dropdown, offering **'auto'**, **'on'**, and **'off'** options. * **Improved Logic:** The extension logic in `src/index.ts` is updated to correctly handle the new string-based `flash_attn` configuration. It now passes the string value (`'auto'`, `'on'`, or `'off'`) directly as a command-line argument to the llama.cpp backend, simplifying the version-checking logic previously required for older llama.cpp versions. The old, complex logic tied to specific backend versions is removed. This refactoring cleans up the extension's codebase and moves environment and path setup concerns into the Tauri plugin where they are most relevant.
Jan - Open-source ChatGPT replacement
Getting Started - Community - Changelog - Bug reports
Jan is bringing the best of open-source AI in an easy-to-use product. Download and run LLMs with full control and privacy.
Installation
The easiest way to get started is by downloading one of the following versions for your respective operating system:
| Platform | Download |
| Windows | jan.exe |
| macOS | jan.dmg |
| Linux (deb) | jan.deb |
| Linux (AppImage) | jan.AppImage |
Download from jan.ai or GitHub Releases.
Features
- Local AI Models: Download and run LLMs (Llama, Gemma, Qwen, GPT-oss etc.) from HuggingFace
- Cloud Integration: Connect to GPT models via OpenAI, Claude models via Anthropic, Mistral, Groq, and others
- Custom Assistants: Create specialized AI assistants for your tasks
- OpenAI-Compatible API: Local server at
localhost:1337for other applications - Model Context Protocol: MCP integration for agentic capabilities
- Privacy First: Everything runs locally when you want it to
Build from Source
For those who enjoy the scenic route:
Prerequisites
- Node.js ≥ 20.0.0
- Yarn ≥ 1.22.0
- Make ≥ 3.81
- Rust (for Tauri)
Run with Make
git clone https://github.com/janhq/jan
cd jan
make dev
This handles everything: installs dependencies, builds core components, and launches the app.
Available make targets:
make dev- Full development setup and launchmake build- Production buildmake test- Run tests and lintingmake clean- Delete everything and start fresh
Manual Commands
yarn install
yarn build:tauri:plugin:api
yarn build:core
yarn build:extensions
yarn dev
System Requirements
Minimum specs for a decent experience:
- macOS: 13.6+ (8GB RAM for 3B models, 16GB for 7B, 32GB for 13B)
- Windows: 10+ with GPU support for NVIDIA/AMD/Intel Arc
- Linux: Most distributions work, GPU acceleration available
For detailed compatibility, check our installation guides.
Troubleshooting
If things go sideways:
- Check our troubleshooting docs
- Copy your error logs and system specs
- Ask for help in our Discord
#🆘|jan-helpchannel
Contributing
Contributions welcome. See CONTRIBUTING.md for the full spiel.
Links
- Documentation - The manual you should read
- API Reference - For the technically inclined
- Changelog - What we broke and fixed
- Discord - Where the community lives
Contact
- Bugs: GitHub Issues
- Business: hello@jan.ai
- Jobs: hr@jan.ai
- General Discussion: Discord
License
Apache 2.0 - Because sharing is caring.
Acknowledgements
Built on the shoulders of giants: