Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: Allow N-GPU Layers (NGL) to be set to 0 in llama.cpp (#5907)

Browse Source 
* fix: Allow N-GPU Layers (NGL) to be set to 0 in llama.cpp

The `n_gpu_layers` (NGL) setting in the llama.cpp extension was incorrectly preventing users from disabling GPU layers by automatically defaulting to 100 when set to 0.

This was caused by a condition that only pushed `cfg.n_gpu_layers` if it was greater than 0 (`cfg.n_gpu_layers > 0`).

This commit updates the condition to `cfg.n_gpu_layers >= 0`, allowing 0 to be a valid and accepted value for NGL. This ensures that users can effectively disable GPU offloading when desired.

* fix: default ngl

---------

Co-authored-by: Louis <louis@jan.ai>
This commit is contained in:
Akarshan Biswas 2025-07-25 16:24:53 +05:30 committed by GitHub
parent 0c53ad0e16
commit 3982ed4c6f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
extensions/llamacpp-extension/src/index.ts
View File

@ -1182,7 +1182,9 @@ export default class llamacpp_extension extends AIEngine {
// Add remaining options from the interface
if (cfg.chat_template) args.push('--chat-template', cfg.chat_template)
args.push('-ngl', String(cfg.n_gpu_layers > 0 ? cfg.n_gpu_layers : 100))
const gpu_layers =
parseInt(String(cfg.n_gpu_layers)) >= 0 ? cfg.n_gpu_layers : 100
args.push('-ngl', String(gpu_layers))
if (cfg.threads > 0) args.push('--threads', String(cfg.threads))
if (cfg.threads_batch > 0)
args.push('--threads-batch', String(cfg.threads_batch))