Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,155 Commits 50 Branches 197 Tags
Commit Graph

699 Commits

Author SHA1 Message Date
Akarshan Biswas
fd9e034461
feat: update AIEngine load method and backend path handling 
- Changed load method to accept modelId instead of loadOptions for better clarity and simplicity
- Renamed engineBasePath parameter to backendPath for consistency with the backend's directory structure
- Added getRandomPort method to ensure unique ports for each session to prevent conflicts
- Refactored configuration and model loading logic to improve maintainability and reduce redundancy
 2025-07-02 12:27:15 +07:00
Thien Tran
9e24e28341
add await to config 2025-07-02 12:27:15 +07:00
Thien Tran
070d8534c4
add some string validation 2025-07-02 12:27:14 +07:00
Thien Tran
494a47aaa5
fix download condition 2025-07-02 12:27:14 +07:00
Thien Tran
f32ae402d5
fix CUDA version URL 2025-07-02 12:27:14 +07:00
Thien Tran
27146eb5cc
fix feature parsing 2025-07-02 12:27:14 +07:00
Thien Tran
a75d13f42f
fix version compare 2025-07-02 12:27:14 +07:00
Thien Tran
3490299f66
refactor get supported features. check driver version for cu11 and cu12 2025-07-02 12:27:13 +07:00
Akarshan Biswas
07d76dc871
feat: Allow specifying mmproj path during model loading 
The `loadOptions` interface in `AIEngine.ts` now includes an optional `mmprojPath` property.  This allows users to provide a path to their MMProject file when loading a model, which is required for certain model types.  The `llamacpp-extension/src/index.ts` has been updated to pass this option to the llamacpp server if provided.
 2025-07-02 12:27:13 +07:00
Thien Tran
fbfaaf43c5
download CUDA libs if needed 2025-07-02 12:27:13 +07:00
Thien Tran
40cd7e962a
feat: download backend for llama.cpp extension (#5123) 
* wip

* update

* add download logic

* add decompress. support delete file

* download backend upon selecting setting

* add some logging and nootes

* add note on race condition

* remove then catch

* default to none backend. only download if it's not installed

* merge version and backend. fetch version from GH

* restrict scope of output_dir

* add note on unpack
 2025-07-02 12:27:13 +07:00
Akarshan Biswas
da23673a44
feat: Add API key generation for Llama.cpp 
This commit introduces API key generation for the Llama.cpp extension.  The API key is now generated on the server side using HMAC-SHA256 and a secret key to ensure security and uniqueness.  The frontend now passes the model ID and API secret to the server to generate the key. This addresses the requirement for secure model access and authorization.
 2025-07-02 12:27:12 +07:00
Akarshan Biswas
d6edb1e944
If checking for proper ctx_len settings after refactoring 2025-07-02 12:27:12 +07:00
Thien Tran
39bb3f34d6
patch failing calls to cortex 2025-07-02 12:27:12 +07:00
Akarshan Biswas
31971e7821
(WIP)randomly generate api-key hash each session 2025-07-02 12:27:12 +07:00
Akarshan Biswas
1dd762f0cf
remove parseGGUFFileName function as it is not used 2025-07-02 12:27:12 +07:00
Akarshan Biswas
7481fae0df
remove ununsed imports and remove n_ctx key from loadOptions 2025-07-02 12:27:11 +07:00
Akarshan Biswas
77d861f56f
Fixup: change key to ctx_size to align with upstream and remove duplicate key 2025-07-02 12:27:11 +07:00
Thien Tran
d5c07acdb5
feat: add LlamacppConfig for llama.cpp extension to improve settings (#5121) 
* add engine settings

* update load options

* rename variable
 2025-07-02 12:27:11 +07:00
Thien Tran
9bb4deeb78
update model config (import and list) 2025-07-02 12:27:11 +07:00
Thien Tran
77f6770333
update fileStat() 2025-07-02 12:27:10 +07:00
Akarshan Biswas
742e731e96
Add --reasoning_budget option 2025-07-02 12:27:10 +07:00
Akarshan Biswas
fe457a5368
slight modelbasepath refactoring 2025-07-02 12:27:10 +07:00
Akarshan Biswas
c5a0ee7f6e
refactor unload and implement a destructor to clean up sessions 2025-07-02 12:27:10 +07:00
Thien Tran
cd36b423b6
add basic model list 2025-07-02 12:27:10 +07:00
Thien Tran
d523166b61
implement delete 2025-07-02 12:27:09 +07:00
Thien Tran
ded9ae733a
feat: Model import (download + local import) for llama.cpp extension (#5087) 
* add pull and abortPull

* add model import (download only)

* write model.yaml. support local model import

* remove cortex-related command

* add TODO

* remove cortex-related command
 2025-07-02 12:27:09 +07:00
Akarshan Biswas
a7a2dcc8d8
refactor load/unload again; move types to core and refactor AIEngine abstract class 2025-07-02 12:27:09 +07:00
Akarshan Biswas
0e9a8a27e5
fixup from refactoring 2025-07-02 12:27:08 +07:00
Akarshan Biswas
bbbf4779df
refactor load/unload 2025-07-02 12:27:08 +07:00
Akarshan Biswas
021f8ae80f
Fixup: llama-server load 2025-07-02 12:27:08 +07:00
Akarshan Biswas
ed6f86d4b1
Change scripts to download only llama.cpp engine 2025-07-02 12:27:07 +07:00
Akarshan Biswas
a8abc9f9aa
Resolved conflicts by keeping HEAD changes 2025-07-02 12:27:07 +07:00
Akarshan Biswas
19274f7e69
update settings 2025-07-02 12:26:39 +07:00
Thien Tran
3f082372fd
add llamacpp-extension. can list some models 2025-07-02 12:26:39 +07:00
Louis
5caf9cc690
Merge pull request #5607 from menloresearch/dependabot/npm_and_yarn/extensions/download-extension/rimraf-6.0.1 
chore(deps-dev): bump rimraf from 3.0.2 to 6.0.1 in /extensions/download-extension
 2025-07-01 01:09:45 +07:00
Louis
9917716cc9
Merge pull request #5608 from menloresearch/dependabot/npm_and_yarn/extensions/conversational-extension/rimraf-6.0.1 
chore(deps-dev): bump rimraf from 3.0.2 to 6.0.1 in /extensions/conversational-extension
 2025-07-01 01:09:28 +07:00
dependabot[bot]
c358831808
chore(deps-dev): bump rimraf in /extensions/assistant-extension 
---
updated-dependencies:
- dependency-name: rimraf
  dependency-version: 6.0.1
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-06-30 08:51:12 +00:00
dependabot[bot]
1c9a64855e
chore(deps-dev): bump rimraf in /extensions/conversational-extension 
---
updated-dependencies:
- dependency-name: rimraf
  dependency-version: 6.0.1
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-06-30 08:49:18 +00:00
dependabot[bot]
3e22835049
chore(deps-dev): bump rimraf in /extensions/download-extension 
---
updated-dependencies:
- dependency-name: rimraf
  dependency-version: 6.0.1
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-06-30 08:44:31 +00:00
Louis
e1b6690763
Merge branch 'dev' into release/v0.6.3 
# Conflicts:
#	web-app/src/containers/DropdownModelProvider.tsx
#	web-app/src/containers/SettingsMenu.tsx
#	web-app/src/containers/dialogs/DeleteModel.tsx
#	web-app/src/routes/__root.tsx
#	web-app/src/routes/settings/general.tsx
#	web-app/src/routes/threads/$threadId.tsx
 2025-06-26 21:55:35 +07:00
Louis
c5aac9270c
Update extensions/inference-cortex-extension/src/index.ts 
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
 2025-06-26 19:36:07 +07:00
Louis
d75db73a43
fix: handle context shift setting from old app versions 2025-06-26 19:32:34 +07:00
Faisal Amir
f5cfe8a537 ✨enhancement: Added jan-nano-128k 2025-06-25 15:56:38 +07:00
dependabot[bot]
51b6c0a7de
chore(deps-dev): bump typescript in /extensions/download-extension (#5405) 
Bumps [typescript](https://github.com/microsoft/TypeScript) from 5.3.3 to 5.8.3.
- [Release notes](https://github.com/microsoft/TypeScript/releases)
- [Changelog](https://github.com/microsoft/TypeScript/blob/main/azure-pipelines.release-publish.yml)
- [Commits](https://github.com/microsoft/TypeScript/compare/v5.3.3...v5.8.3)

---
updated-dependencies:
- dependency-name: typescript
  dependency-version: 5.8.3
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-06-24 14:54:11 +07:00
Louis
52bb54d2d5
fix: glitch model download issue (#5322) 2025-06-17 23:01:22 +07:00
Louis
86079074d2
🐛fix: update default extension settings (#5315) 
* fix: update default extension settings

* chore: hide language setting on Prod
 2025-06-17 17:37:31 +07:00
Louis
2899d58ad7
🐛fix: llama.cpp default NGL setting does not offload all layers to GPU (#5310) 
* 🐛fix: llama.cpp default NGL setting does not offload all layers to GPU

* chore: cover more cases

* chore: clean up

* fix: should not show GPU section on Mac
 2025-06-17 15:30:07 +07:00
Louis
9ab69b157b
🐛fix: Jan-nano repo name changed (#5274) 2025-06-15 11:17:41 +07:00
Louis
2634659366
🐛fix: default onboarding model should use recommended quantizations (#5273) 
* 🐛fix: default onboarding model should use recommended quantizations

* ✨enhancement: show context shift option in provider settings

* 🔧chore: wording
 2025-06-15 02:34:14 +07:00