Akarshan Biswas
07d76dc871
feat: Allow specifying mmproj path during model loading
...
The `loadOptions` interface in `AIEngine.ts` now includes an optional `mmprojPath` property. This allows users to provide a path to their MMProject file when loading a model, which is required for certain model types. The `llamacpp-extension/src/index.ts` has been updated to pass this option to the llamacpp server if provided.
2025-07-02 12:27:13 +07:00
Thien Tran
fbfaaf43c5
download CUDA libs if needed
2025-07-02 12:27:13 +07:00
Thien Tran
40cd7e962a
feat: download backend for llama.cpp extension ( #5123 )
...
* wip
* update
* add download logic
* add decompress. support delete file
* download backend upon selecting setting
* add some logging and nootes
* add note on race condition
* remove then catch
* default to none backend. only download if it's not installed
* merge version and backend. fetch version from GH
* restrict scope of output_dir
* add note on unpack
2025-07-02 12:27:13 +07:00
Akarshan Biswas
da23673a44
feat: Add API key generation for Llama.cpp
...
This commit introduces API key generation for the Llama.cpp extension. The API key is now generated on the server side using HMAC-SHA256 and a secret key to ensure security and uniqueness. The frontend now passes the model ID and API secret to the server to generate the key. This addresses the requirement for secure model access and authorization.
2025-07-02 12:27:12 +07:00
Akarshan Biswas
d6edb1e944
If checking for proper ctx_len settings after refactoring
2025-07-02 12:27:12 +07:00
Thien Tran
39bb3f34d6
patch failing calls to cortex
2025-07-02 12:27:12 +07:00
Akarshan Biswas
31971e7821
(WIP)randomly generate api-key hash each session
2025-07-02 12:27:12 +07:00
Akarshan Biswas
1dd762f0cf
remove parseGGUFFileName function as it is not used
2025-07-02 12:27:12 +07:00
Akarshan Biswas
7481fae0df
remove ununsed imports and remove n_ctx key from loadOptions
2025-07-02 12:27:11 +07:00
Akarshan Biswas
77d861f56f
Fixup: change key to ctx_size to align with upstream and remove duplicate key
2025-07-02 12:27:11 +07:00
Thien Tran
d5c07acdb5
feat: add LlamacppConfig for llama.cpp extension to improve settings ( #5121 )
...
* add engine settings
* update load options
* rename variable
2025-07-02 12:27:11 +07:00
Thien Tran
9bb4deeb78
update model config (import and list)
2025-07-02 12:27:11 +07:00
Thien Tran
77f6770333
update fileStat()
2025-07-02 12:27:10 +07:00
Akarshan Biswas
742e731e96
Add --reasoning_budget option
2025-07-02 12:27:10 +07:00
Akarshan Biswas
fe457a5368
slight modelbasepath refactoring
2025-07-02 12:27:10 +07:00
Akarshan Biswas
c5a0ee7f6e
refactor unload and implement a destructor to clean up sessions
2025-07-02 12:27:10 +07:00
Thien Tran
cd36b423b6
add basic model list
2025-07-02 12:27:10 +07:00
Thien Tran
d523166b61
implement delete
2025-07-02 12:27:09 +07:00
Thien Tran
ded9ae733a
feat: Model import (download + local import) for llama.cpp extension ( #5087 )
...
* add pull and abortPull
* add model import (download only)
* write model.yaml. support local model import
* remove cortex-related command
* add TODO
* remove cortex-related command
2025-07-02 12:27:09 +07:00
Akarshan Biswas
a7a2dcc8d8
refactor load/unload again; move types to core and refactor AIEngine abstract class
2025-07-02 12:27:09 +07:00
Akarshan Biswas
0e9a8a27e5
fixup from refactoring
2025-07-02 12:27:08 +07:00
Akarshan Biswas
bbbf4779df
refactor load/unload
2025-07-02 12:27:08 +07:00
Akarshan Biswas
021f8ae80f
Fixup: llama-server load
2025-07-02 12:27:08 +07:00
Akarshan Biswas
ed6f86d4b1
Change scripts to download only llama.cpp engine
2025-07-02 12:27:07 +07:00
Akarshan Biswas
a8abc9f9aa
Resolved conflicts by keeping HEAD changes
2025-07-02 12:27:07 +07:00
Akarshan Biswas
19274f7e69
update settings
2025-07-02 12:26:39 +07:00
Thien Tran
3f082372fd
add llamacpp-extension. can list some models
2025-07-02 12:26:39 +07:00
Louis
5caf9cc690
Merge pull request #5607 from menloresearch/dependabot/npm_and_yarn/extensions/download-extension/rimraf-6.0.1
...
chore(deps-dev): bump rimraf from 3.0.2 to 6.0.1 in /extensions/download-extension
2025-07-01 01:09:45 +07:00
Louis
9917716cc9
Merge pull request #5608 from menloresearch/dependabot/npm_and_yarn/extensions/conversational-extension/rimraf-6.0.1
...
chore(deps-dev): bump rimraf from 3.0.2 to 6.0.1 in /extensions/conversational-extension
2025-07-01 01:09:28 +07:00
dependabot[bot]
c358831808
chore(deps-dev): bump rimraf in /extensions/assistant-extension
...
---
updated-dependencies:
- dependency-name: rimraf
dependency-version: 6.0.1
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2025-06-30 08:51:12 +00:00
dependabot[bot]
1c9a64855e
chore(deps-dev): bump rimraf in /extensions/conversational-extension
...
---
updated-dependencies:
- dependency-name: rimraf
dependency-version: 6.0.1
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2025-06-30 08:49:18 +00:00
dependabot[bot]
3e22835049
chore(deps-dev): bump rimraf in /extensions/download-extension
...
---
updated-dependencies:
- dependency-name: rimraf
dependency-version: 6.0.1
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2025-06-30 08:44:31 +00:00
Louis
e1b6690763
Merge branch 'dev' into release/v0.6.3
...
# Conflicts:
# web-app/src/containers/DropdownModelProvider.tsx
# web-app/src/containers/SettingsMenu.tsx
# web-app/src/containers/dialogs/DeleteModel.tsx
# web-app/src/routes/__root.tsx
# web-app/src/routes/settings/general.tsx
# web-app/src/routes/threads/$threadId.tsx
2025-06-26 21:55:35 +07:00
Louis
c5aac9270c
Update extensions/inference-cortex-extension/src/index.ts
...
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-26 19:36:07 +07:00
Louis
d75db73a43
fix: handle context shift setting from old app versions
2025-06-26 19:32:34 +07:00
Faisal Amir
f5cfe8a537
✨ enhancement: Added jan-nano-128k
2025-06-25 15:56:38 +07:00
dependabot[bot]
51b6c0a7de
chore(deps-dev): bump typescript in /extensions/download-extension ( #5405 )
...
Bumps [typescript](https://github.com/microsoft/TypeScript ) from 5.3.3 to 5.8.3.
- [Release notes](https://github.com/microsoft/TypeScript/releases )
- [Changelog](https://github.com/microsoft/TypeScript/blob/main/azure-pipelines.release-publish.yml )
- [Commits](https://github.com/microsoft/TypeScript/compare/v5.3.3...v5.8.3 )
---
updated-dependencies:
- dependency-name: typescript
dependency-version: 5.8.3
dependency-type: direct:development
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-24 14:54:11 +07:00
Louis
52bb54d2d5
fix: glitch model download issue ( #5322 )
2025-06-17 23:01:22 +07:00
Louis
86079074d2
🐛 fix: update default extension settings ( #5315 )
...
* fix: update default extension settings
* chore: hide language setting on Prod
2025-06-17 17:37:31 +07:00
Louis
2899d58ad7
🐛 fix: llama.cpp default NGL setting does not offload all layers to GPU ( #5310 )
...
* 🐛 fix: llama.cpp default NGL setting does not offload all layers to GPU
* chore: cover more cases
* chore: clean up
* fix: should not show GPU section on Mac
2025-06-17 15:30:07 +07:00
Louis
9ab69b157b
🐛 fix: Jan-nano repo name changed ( #5274 )
2025-06-15 11:17:41 +07:00
Louis
2634659366
🐛 fix: default onboarding model should use recommended quantizations ( #5273 )
...
* 🐛 fix: default onboarding model should use recommended quantizations
* ✨ enhancement: show context shift option in provider settings
* 🔧 chore: wording
2025-06-15 02:34:14 +07:00
Louis
a10dbef2c8
✨ enhancement: Model sources should not be hard coded from frontend ( #5270 )
2025-06-14 22:00:43 +07:00
Louis
1e17cc6ec7
✨ enhancement: model run improvement ( #5268 )
...
* fix: mcp tool error handling
* fix: error message
* fix: trigger download from recommend model
* fix: can't scroll hub
* fix: show progress
* ✨ enhancement: prompt users to increase context size
* ✨ enhancement: rearrange action buttons for a better UX
* 🔧 chore: clean up logics
---------
Co-authored-by: Faisal Amir <urmauur@gmail.com>
2025-06-14 16:32:15 +07:00
Louis
23240f958e
Merge pull request #5262 from menloresearch/chore/sync-new-hub-data
...
chore: sync new hub data
2025-06-13 17:05:49 +07:00
Louis
27c4918395
fix: default settings should leave empty ( #5257 )
...
* fix: default settings should leave empty
* fix: default settings
* fix: remove some more default settings
* fix: threads and cont
* fix: data
* fix: default setting
* fix: settings
* chore: bump cortex version
* chore: bump to cortex 1.0.14
* chore: clean up
* typoe
* chore: fix dialog hang
* fix: default parameter
* chore: truncate edit model title
* chore: update default provider settings
* chore: fix typo
---------
Co-authored-by: Faisal Amir <urmauur@gmail.com>
2025-06-13 00:01:25 +07:00
Louis
51a321219d
chore: fix model settings are not applied accordingly on change ( #5231 )
...
* chore: fix model settings are not applied accordingly on change
* chore: handle failed tool call
* chore: stop inference and model on reject
2025-06-10 16:26:42 +07:00
Louis
6d6790d5e0
fix: clean up cortex processes ( #5215 )
...
* fix: clean up cortex processes
* chore: clean up
* chore: does not need output shell
2025-06-09 22:23:07 +07:00
Louis
b85bce8e09
fix: migrate provider settings ( #5205 )
...
* chore: fix migrate api key
* chore: add logs
* chore: deprecate migration step
* chore: clean up
* fix: key apply
2025-06-05 20:25:43 +07:00
Louis
6faca3e732
refactor: remove JS server package ( #5192 )
...
* refactor: remove js server package
* chore: migrate HF token data
2025-06-04 15:33:35 +07:00
