133 Commits

Author SHA1 Message Date
Thien Tran
070d8534c4
add some string validation 2025-07-02 12:27:14 +07:00
Thien Tran
494a47aaa5
fix download condition 2025-07-02 12:27:14 +07:00
Thien Tran
f32ae402d5
fix CUDA version URL 2025-07-02 12:27:14 +07:00
Thien Tran
27146eb5cc
fix feature parsing 2025-07-02 12:27:14 +07:00
Thien Tran
a75d13f42f
fix version compare 2025-07-02 12:27:14 +07:00
Thien Tran
3490299f66
refactor get supported features. check driver version for cu11 and cu12 2025-07-02 12:27:13 +07:00
Akarshan Biswas
07d76dc871
feat: Allow specifying mmproj path during model loading
The `loadOptions` interface in `AIEngine.ts` now includes an optional `mmprojPath` property.  This allows users to provide a path to their MMProject file when loading a model, which is required for certain model types.  The `llamacpp-extension/src/index.ts` has been updated to pass this option to the llamacpp server if provided.
2025-07-02 12:27:13 +07:00
Thien Tran
fbfaaf43c5
download CUDA libs if needed 2025-07-02 12:27:13 +07:00
Thien Tran
40cd7e962a
feat: download backend for llama.cpp extension (#5123)
* wip

* update

* add download logic

* add decompress. support delete file

* download backend upon selecting setting

* add some logging and nootes

* add note on race condition

* remove then catch

* default to none backend. only download if it's not installed

* merge version and backend. fetch version from GH

* restrict scope of output_dir

* add note on unpack
2025-07-02 12:27:13 +07:00
Akarshan Biswas
da23673a44
feat: Add API key generation for Llama.cpp
This commit introduces API key generation for the Llama.cpp extension.  The API key is now generated on the server side using HMAC-SHA256 and a secret key to ensure security and uniqueness.  The frontend now passes the model ID and API secret to the server to generate the key. This addresses the requirement for secure model access and authorization.
2025-07-02 12:27:12 +07:00
Akarshan Biswas
d6edb1e944
If checking for proper ctx_len settings after refactoring 2025-07-02 12:27:12 +07:00
Thien Tran
39bb3f34d6
patch failing calls to cortex 2025-07-02 12:27:12 +07:00
Akarshan Biswas
31971e7821
(WIP)randomly generate api-key hash each session 2025-07-02 12:27:12 +07:00
Akarshan Biswas
1dd762f0cf
remove parseGGUFFileName function as it is not used 2025-07-02 12:27:12 +07:00
Akarshan Biswas
7481fae0df
remove ununsed imports and remove n_ctx key from loadOptions 2025-07-02 12:27:11 +07:00
Akarshan Biswas
77d861f56f
Fixup: change key to ctx_size to align with upstream and remove duplicate key 2025-07-02 12:27:11 +07:00
Thien Tran
d5c07acdb5
feat: add LlamacppConfig for llama.cpp extension to improve settings (#5121)
* add engine settings

* update load options

* rename variable
2025-07-02 12:27:11 +07:00
Thien Tran
9bb4deeb78
update model config (import and list) 2025-07-02 12:27:11 +07:00
Thien Tran
77f6770333
update fileStat() 2025-07-02 12:27:10 +07:00
Akarshan Biswas
742e731e96
Add --reasoning_budget option 2025-07-02 12:27:10 +07:00
Akarshan Biswas
fe457a5368
slight modelbasepath refactoring 2025-07-02 12:27:10 +07:00
Akarshan Biswas
c5a0ee7f6e
refactor unload and implement a destructor to clean up sessions 2025-07-02 12:27:10 +07:00
Thien Tran
cd36b423b6
add basic model list 2025-07-02 12:27:10 +07:00
Thien Tran
d523166b61
implement delete 2025-07-02 12:27:09 +07:00
Thien Tran
ded9ae733a
feat: Model import (download + local import) for llama.cpp extension (#5087)
* add pull and abortPull

* add model import (download only)

* write model.yaml. support local model import

* remove cortex-related command

* add TODO

* remove cortex-related command
2025-07-02 12:27:09 +07:00
Akarshan Biswas
a7a2dcc8d8
refactor load/unload again; move types to core and refactor AIEngine abstract class 2025-07-02 12:27:09 +07:00
Akarshan Biswas
0e9a8a27e5
fixup from refactoring 2025-07-02 12:27:08 +07:00
Akarshan Biswas
bbbf4779df
refactor load/unload 2025-07-02 12:27:08 +07:00
Akarshan Biswas
021f8ae80f
Fixup: llama-server load 2025-07-02 12:27:08 +07:00
Akarshan Biswas
ed6f86d4b1
Change scripts to download only llama.cpp engine 2025-07-02 12:27:07 +07:00
Akarshan Biswas
a8abc9f9aa
Resolved conflicts by keeping HEAD changes 2025-07-02 12:27:07 +07:00
Akarshan Biswas
19274f7e69
update settings 2025-07-02 12:26:39 +07:00
Thien Tran
3f082372fd
add llamacpp-extension. can list some models 2025-07-02 12:26:39 +07:00