31 Commits

Author SHA1 Message Date
hiento09
bd0e525d66
chore: upgrade to turbo v2 and reduce ci quality gate runtime (#4324)
* chore: upgrade to turbo v2 and reduce ci quality gate runtime

* chore: upgrade to yarn v4 and deprecate turborepo

* chore: reconfigure modules and fix tests

* chore: switch to github-hosted runner

* fix: dependency resolving

* chore: clean redundant step

* chore: headless test

* chore: remove headed param

* fix: resolve dependency version

* fix: ubuntu no-sandbox test

---------

Co-authored-by: Hien To <tominhhien97@gmail.com>
Co-authored-by: Louis <louis@jan.ai>
2024-12-29 17:46:15 +07:00
Louis
4080dc4b65
feat: model and cortex extensions update 2024-11-04 15:37:12 +07:00
Faisal Amir
b37d4a5c7e
fix: types issue (#internalTypeOnlyBrand) in the @types/node package (#3921) 2024-10-31 21:17:25 +07:00
Louis
8334076047
fix: #3491 - Unable to use tensorrt-llm (#3741)
* fix: #3491 - Unable to use tensorrt-llm

* fix: abortModelDownload input type
2024-09-30 11:58:46 +07:00
Louis
8e603bd5db
fix: #3476 - Mismatch id between model json and path (#3645)
* fix: mismatch between model json and path

* chore: revert preserve model settings

* test: add tests
2024-09-17 16:43:47 +07:00
Louis
98bef7b7cf
test: add model parameter validation rules and persistence tests (#3618)
* test: add model parameter validation rules and persistence tests

* chore: fix CI cov step

* fix: invalid model settings should fallback to origin value

* test: support fallback integer settings
2024-09-17 08:34:58 +07:00
Louis
a699f8f32f
Revert "Jan integrates Cortex"
This reverts commit ad6fbea22df6deaba31e146dddb456e4a5d5dd75

Revert "chore: add engine logo from local instead of metadata logo (#3363)"

This reverts commit ad6fbea22df6deaba31e146dddb456e4a5d5dd75.

Revert "fix: LaTex formula render issue (#3353)"

This reverts commit 3b2c84c4fee61b886c883c68801be3bc5a8584ad.

Revert "chore: minor ui improvement (#3352)"

This reverts commit 6dd387db2b5b9890f19d0c3505cf9cb770fd492f.

Revert "fix: failed to relaunch app to update (#3351)"

This reverts commit fcaf98a2fa4e674799602e8093914bcc04ced153.

Revert "chore: add back GPU information to system monitoring bar (#3350)"

This reverts commit 03455a91807c7af6c6325901997c6d7231d2cd0d.

Revert "fix: empty model page not shown when delete all threads and models (#3343)"

This reverts commit 9e29fcd69eb9085843896686806fd453a1285723.

Revert "feat: allow user configure remote model from my model (#3348)"

This reverts commit fdab8af057f80cf1ccaae0dc42c4e5161925f51e.

Revert "chore: ui fix button outline for configure cloud model (#3347)"

This reverts commit fe8ed1f26dc86ead92ffea4f36e2989caf7dad88.

Revert "feat: move icon create new thread into top panel (#3346)"

This reverts commit 46cb1b45b997181e2188f8dafb2fc0d0cc12ddcd.

Revert "chore(UI): update experience model dropdown (#3342)"

This reverts commit 8b44613015a907dc491113aeb99c963080424892.

Revert "Chore/simple bug template and correct a copy (#3344)"

This reverts commit 23cd5fd3979e7529811045da5c4912369bcc7532.

Revert "chore(ui): fix alignment loader starter screen (#3338)"

This reverts commit e9f5d2f837ce323b0851ea04cded913ab433388c.

Revert "Increase retry upload to R2 to 5 times (#3337)"

This reverts commit dcfb497934edc795955d971b6d391ee1e6309a03.

Revert "fix: broken jan build - add log trace (jan.log) (#3336)"

This reverts commit 77422c3a7ed240909942ac0d8c4b259af8d87a28.

Revert "chore: disable quick ask (#3334)"

This reverts commit 6e4b6b09ae009149f262d86d5b19bb8096267c19.

Revert "fix: update legacy path (#3328)"

This reverts commit 5eb112142c6431cfe0cdf11ce28810ca650a5427.

Revert "chore: add cortex version (#3318)"

This reverts commit 60587649c56a1f24272e763f25aa5b4042f7719a.

Revert "fix: broken app due to incorrect api path (#3316)"

This reverts commit 3de4eab2a0dfbf9f593d73b9dde6bca1d9df2279.

Revert "feat: modal waiting cortex (#3306)"

This reverts commit 1f5168d4af9080b867c19d334c398bf32e4f54b8.

Revert "fix: refresh should not create new thread (#3314)"

This reverts commit 624d07703c50ea332ed4eeac9dc3a26bc8190d08.

Revert "fix: avoid lose title threads (#3307)"

This reverts commit a4f5fda104c2d1e01ea72798f055e5b4e3cfd616.

Revert "feat: change data folder (#3309)"

This reverts commit b43242b9b24352c7f90995eccab753dede679616.

Revert "feat: embed cortex into jan as a js module (#3305)"

This reverts commit b348110fb73bd5f13c69f1b915168687dea776d0.

Revert "fix: migration item in setting detail omit buttons (#3298)"

This reverts commit 709204b2bc9d9ed08e2245cbb084482f5908ab3a.

Revert "fix: merge gpu arch and os tensorrt models (#3299)"

This reverts commit aa7dbdc9fa701debeee28d9c7eb4af6258685321.

Revert "chore: update cortex new version (#3300)"

This reverts commit 602097909d38b4874db8b9f19a729c65a0ac9619.

Revert "fix: engine logo on model dropdown (#3291)"

This reverts commit 8eb8611c28f6c4cdf1ab142a6e18c82bcc4c2073.

Revert "fix: icon setting can close and open right panel (#3295)"

This reverts commit be31e9315e2df5c483de3f46bd37740d277cfccd.

Revert "fix: error while importing local model is not shown (#3294)"

This reverts commit 26be941e8426462e1e3a28e5b9bf1f834f462f82.

Revert "fix: add lower case quantization support (#3293)"

This reverts commit 3135ccc27e894a4056f882cd25f0bf7e10e56f49.

Revert "fix: onnx can't be selected in download model modal (#3283)"

This reverts commit 2521e1db518e9e01493e89dcc98c181ccd2b48a2.

Revert "feat: add chunk count (#3290)"

This reverts commit bad481bf05aa38edcf553e1273f5d692a65c9225.

Revert "fix: RAM always show 0% (#3287)"

This reverts commit 2201e6c5f87538b953503937fe6b135fe1aa2d94.

Revert "fix: remote engine should not allow reinit (#3284)"

This reverts commit 98abff0da3467c090618233db12a25bfa4c1db69.

Revert "chore": update minor UI (#3281)"

This reverts commit 105a9aa1a1830648a32ae285f751b4078c8ac2b2.

Revert "chore: update z-index tooltip (#3280)"

This reverts commit 5a81865508c205ed8c54df209092553a0c40054f.

Revert "feat: add nvidia engine (#3279)"

This reverts commit 8372f30f0ee99606b123351e7bb62636c62c8b23.

Revert "fix: migration wrong directory (#3278)"

This reverts commit 7fb1354287677f577070ccb065ed3a5f9e5b9882.

Revert "fix: clearer app loading prompt (#3275)"

This reverts commit 44a6401000334b79b225ab6fd6afb79f9da4bd51.

Revert "fix: allow user to reinit engine from settings page (#3277)"

This reverts commit 57cf3c7b3d5bface785763d06813906ba6eab7c9.

Revert "feat: enable copy over instructions (#3266)"

This reverts commit 2074511067201f0addb9d274cc90d1e782f2bc1d.

Revert "chore: toast message on model import fail with reason (#3276)"

This reverts commit 3bebdfe67e1571c7414065a36d16eb5941115ee0.

Revert "fix: should not let second instance terminate cortex (#3274)"

This reverts commit d074a5a445b73ca195a49814a935300f9e895aaa.

Revert "chore: remnove focus button (#3272)"

This reverts commit 07fa79e71a401becdbc0f474c27b860654a8bd62.

Revert "chore: update hub search result (#3273)"

This reverts commit 10b4a9087af709d147b34f6c3ee63d2d3b75c77a.

Revert "chore: temporary hidden import model (#3270)"

This reverts commit db5d8aba454fd4cc1e07253ca4805d4b1b3e7fb2.

Revert "fix: set cortex data folder path when starting jan (#3252)"

This reverts commit 91c77eda78ecd251d480e58b853fe7b261f6de50.

Revert "fix: remote model added manually does not shown in model drop down (#3261)"

This reverts commit 224ca3f7cc25b2577ab123829907964b78b78aa8.

Revert "feat: add more options for cortex popup (#3236)"

This reverts commit 5e06ed8a122aaed9d68fbd04ce42b65bf8987e58.

Revert "feat: manage cloud models from threads screen (#3223)"

This reverts commit 37a3c4f844419e66cfe3f2a9ff79ba688538241f.

Revert "chore: check the legacy incompatible message type (#3248)"

This reverts commit c10caf8d7f1f9cf68551e41de5d54cd4450cf44a.

Revert "chore: minor copy for grammar (#3235)"

This reverts commit f0f23078f31f58e01ba27787d6926f5c1eb2ff0b.

Revert "fix: add back normalize message function (#3234)"

This reverts commit 83579df3a40ff61eac25975da8295fceaec679dc.

Revert "chore: update conditional starter screen after cortex load (#3227)"

This reverts commit 4d3a97f1dca9e6c3ea746586e8607541f2d1c0b3.

Revert "fix: broken status parse due to empty category (#3233)"

This reverts commit 68714eeaf9212a6fdacd5c6a48d8691db9cc99eb.

Revert "feat: make scroll area type auto for make default visible scrollbar (#3220)"

This reverts commit 13428d60e7d3ea6a24c0df8871ea13e2dec0d5fd.

Revert "fix: update new api from cortex to support 0.5.0 (#3221)"

This reverts commit ec9b5bf682a8676e132a08075b6ae03cf9e23132.

Revert "feat: new starter screen (#3217)"

This reverts commit e8ee694abd33b34112d2c7d09f8c03370c2d22cc.

Revert "bump-cortex-0.5.0-1 (#3218)"

This reverts commit 5369da78f5b83b1c8761cb48820ccf3111728a90.

Revert "Deprecate Docker and K8s (#3219)"

This reverts commit 7611a05c44982d07465bec57658d5bf965f30ad5.

Revert "chore: set container max width for chat message and new hub screen (#3213)"

This reverts commit 007daa71616268b0e741e7a890b319401e49a81e.

Revert "feat: integrating cortex (#3001)"

This reverts commit 101268f6f36df96b62982a9eeb8581ebe103a909.
2024-08-15 10:44:47 +07:00
NamH
101268f6f3
feat: integrating cortex (#3001)
* feat: integrating cortex

* Temporary prevent crash

Signed-off-by: James <namnh0122@gmail.com>

* fix yarn lint

Signed-off-by: James <namnh0122@gmail.com>

* refactor: remove core node module - fs - extensions and so on (#3151)

* add migration script for threads, messages and models

Signed-off-by: James <namnh0122@gmail.com>

* remove freq_penalty and presence_penalty if model not supported

Signed-off-by: James <namnh0122@gmail.com>

* add back models in my models

Signed-off-by: James <namnh0122@gmail.com>

* fix api-url for setup API key popup

Signed-off-by: James <namnh0122@gmail.com>

* fix using model name for dropdown model

Signed-off-by: James <namnh0122@gmail.com>

* fix can't click to hotkey

Signed-off-by: James <namnh0122@gmail.com>

* fix: disable some UIs

Signed-off-by: James <namnh0122@gmail.com>

* fix build

Signed-off-by: James <namnh0122@gmail.com>

* reduce calling HF api

Signed-off-by: James <namnh0122@gmail.com>

* some ui update

Signed-off-by: James <namnh0122@gmail.com>

* feat: modal migration UI  (#3153)

* feat: handle popup migration

* chore: update loader

* chore: integrate script migration

* chore: cleanup import

* chore: moving out spinner loader

* chore: update check thread message success migrate

* chore: add handle script into retry button

* remove warning from joi

Signed-off-by: James <namnh0122@gmail.com>

* chore: fix duplicate children

* fix: path after migrating model

Signed-off-by: James <namnh0122@gmail.com>

* chore: apply mutation for config

* chore: prevent calling too many create assistant api

Signed-off-by: James <namnh0122@gmail.com>

* using cortexso

Signed-off-by: James <namnh0122@gmail.com>

* update download api

Signed-off-by: James <namnh0122@gmail.com>

* fix use on slider item

Signed-off-by: James <namnh0122@gmail.com>

* fix: ui no download model or simple onboarding (#3166)

* fix download huggingface model match with slider item

Signed-off-by: James <namnh0122@gmail.com>

* update owner_logo to logo and author

Signed-off-by: James <namnh0122@gmail.com>

* update new cortexso

Signed-off-by: James <namnh0122@gmail.com>

* Add install python step for macos

* add engine table

Signed-off-by: James <namnh0122@gmail.com>

* fix local icons

Signed-off-by: James <namnh0122@gmail.com>

* feat: add search feature for model hub

Signed-off-by: James <namnh0122@gmail.com>

* fix misalign switch

Signed-off-by: James <namnh0122@gmail.com>

* fix: delete thread not focus on other thread

Signed-off-by: James <namnh0122@gmail.com>

* add get model from hugging face

Signed-off-by: James <namnh0122@gmail.com>

* fix download from hugging face

Signed-off-by: James <namnh0122@gmail.com>

* small update

Signed-off-by: James <namnh0122@gmail.com>

* update

Signed-off-by: James <namnh0122@gmail.com>

* fix system monitor rounded only on the left

Signed-off-by: James <namnh0122@gmail.com>

* chore: update ui new hub screen (#3174)

* chore: update ui new hub screen

* chore: update layout centerpanel thread and hub screen

* chore: update detail model by group

* update cortexso 0.1.13

Signed-off-by: James <namnh0122@gmail.com>

* chore: add file size

Signed-off-by: James <namnh0122@gmail.com>

* chore: put engine to experimental feature

Signed-off-by: James <namnh0122@gmail.com>

* chore: open cortex folder

Signed-off-by: James <namnh0122@gmail.com>

* chore: add back user avatar

Signed-off-by: James <namnh0122@gmail.com>

* chore: minor UI hub (#3182)

* chore: add back right click thread list and update 3 dots are overlapping with the text

* chore: update position dropdown list my models

* chore: make on-device tab showing 6 items instead of 4

* chore: update style description modals detail model

* chore: update isGeneration loader and author name on modal

* feat: integrate cortex single executable

Signed-off-by: James <namnh0122@gmail.com>

* fix build

Signed-off-by: James <namnh0122@gmail.com>

* chore: added blank state

* chore: update ui component blank state

* bump cortex binary version

* fix: logic show modal migration (#3165)

* fix: logic show modal migration

* chore: fixed logic

* chore: read contain format gguf local models

* chore: change return hasLocalModel

* chore: intiial skipmigration state

* chore: filter embedding model

* fix: delete top thread not focus on any other thread

* chore: added UI no result component search models group (#3188)

* fix: remote model should show all when user config that engine

Signed-off-by: James <namnh0122@gmail.com>

* chore: set state thread and models migration using getOnInit (#3189)

* chore: set state thread and models migration using getOnInit

* chore: add state as dependecies hooks

* chore: system monitor panel show engine model (#3192)

* fix: remove config api, replace with engine

Signed-off-by: James <namnh0122@gmail.com>

* update

Signed-off-by: James <namnh0122@gmail.com>

* update reactquery

Signed-off-by: James <namnh0122@gmail.com>

* bump cortex 0.4.35

* feat: add waiting for cortex popup

Signed-off-by: James <namnh0122@gmail.com>

* chore: add loader detail model popup (#3195)

* chore: model start loader (#3197)

* chore: added model loader when user starting chat without model active

* chore: update copies loader

* fix: select min file size if recommended quant does not exist

Signed-off-by: James <namnh0122@gmail.com>

* chore: temporary hide gpu config

* fix: tensorrt not shown

Signed-off-by: James <namnh0122@gmail.com>

* fix lint

Signed-off-by: James <namnh0122@gmail.com>

* fix tests

Signed-off-by: James <namnh0122@gmail.com>

* fix e2e tests (wip)

Signed-off-by: James <namnh0122@gmail.com>

* update

Signed-off-by: James <namnh0122@gmail.com>

* fix: adding element and correct test to adapt new UI

* fix: temp skip unstable part

* fix: only show models which can be supported

Signed-off-by: James <namnh0122@gmail.com>

* Update version.txt

* update send message

Signed-off-by: James <namnh0122@gmail.com>

* fix: not allow user send message when is generating

Signed-off-by: James <namnh0122@gmail.com>

* chore: temp skip Playwright test due to env issue

* chore: temp skip Playwright test due to env issue

* update

Signed-off-by: James <namnh0122@gmail.com>

* chore: minor-ui-feedback (#3202)

---------

Signed-off-by: James <namnh0122@gmail.com>
Co-authored-by: Louis <louis@jan.ai>
Co-authored-by: Faisal Amir <urmauur@gmail.com>
Co-authored-by: Hien To <tominhhien97@gmail.com>
Co-authored-by: Van Pham <64197333+Van-QA@users.noreply.github.com>
Co-authored-by: Van-QA <van@jan.ai>
2024-07-26 17:52:43 +07:00
Louis
537ef20a54
chore: replace nitro by cortex-cpp (#2912) 2024-05-16 17:46:49 +07:00
NamH
31397de2d1
Refactor/deprecate hugging face ext (#2620)
* refactor: deprecate huggingface extension

Signed-off-by: James <james@jan.ai>
2024-04-16 17:23:45 +07:00
Louis
02c49e796d
fix: race condition issue - reading settings.json file (#2683)
* fix: race condition issue - reading settings.json file

* fix: cannot reset data while starting model

* chore: remove extension suffix
2024-04-11 15:37:46 +07:00
Louis
c0949b2d7e
fix: better kill process tensorrt-llm (#2681) 2024-04-11 12:47:41 +07:00
Louis
065ed03099
fix: wrong monitoring system information type (#2679) 2024-04-11 11:07:31 +07:00
Louis
b19234ed71
chore: Extension should have product name in manifest (#2675)
* chore: Extension should have product name in manifest

* chore: typo
2024-04-11 09:50:58 +07:00
Louis
d93d74c86b
feat: nitro additional dependencies (#2674) 2024-04-11 09:13:02 +07:00
NamH
e0d6049d66
chore: extension should register its own models (#2601)
* chore: extension should register its own models

Signed-off-by: James <james@jan.ai>

---------

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
2024-04-05 14:18:58 +07:00
hiento09
a6cbc0b86f
Change release download url to cloudflare worker proxy and update download model tensorrt llm to aws s3 endpoint (#2576)
Co-authored-by: Hien To <tominhhien97@gmail.com>
2024-04-02 17:08:53 +07:00
NamH
fa35aa6e14
feat: dynamically register extension settings (#2494)
* feat: add extesion settings

Signed-off-by: James <james@jan.ai>

---------

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: Louis <louis@jan.ai>
2024-03-29 15:44:46 +07:00
NamH
67e285fa96
chore: remove rmdirsync from core api since it is deprecated (#2459)
* chore: remove rmdirsync from core api since it is deprecated

Signed-off-by: James <james@jan.ai>

* chore: remove mkdirsync

Signed-off-by: James <james@jan.ai>

---------

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
2024-03-22 17:57:16 +07:00
Louis
3c0383f6d8
fix: app raises port not available error (#2466) 2024-03-22 17:53:33 +07:00
Louis
b8cee875b1
fix: app shows wrong toast on stopping inference (#2460) 2024-03-22 14:40:15 +07:00
Louis
acbec78dbf
fix: refactor inference engines to extends AIEngine (#2347)
* fix: refactor nitro to extends localoaiengine

* fix: refactor openai extension

* chore: refactor groq extension

* chore: refactor triton tensorrt extension

* chore: add tests

* chore: refactor engines
2024-03-22 09:35:14 +07:00
Louis
ff7ec39915
fix: incompatible browser dependency (#2439)
* fix: incompatible browser dependency

* fix: update model extension to use rollup

* fix: test timeout
2024-03-21 16:54:42 +07:00
Louis
489e8aab24
Sync release 0.4.9 to dev (#2407)
* fix: move tensorrt executable to engine (#2400)

* fix: move tensorrt executable to engine

Signed-off-by: James <james@jan.ai>

* some update

Signed-off-by: hiro <hiro@jan.ai>

* chore: bump tensorrt version

* fix: wrong destroy path

* fix: install extensions in parallel

* chore: update path for tensorrt engine (#2404)

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>

---------

Signed-off-by: James <james@jan.ai>
Signed-off-by: hiro <hiro@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: hiro <hiro@jan.ai>
Co-authored-by: Louis <louis@jan.ai>

* Release/v0.4.9 (#2421)

* fix: turn off experimental settings should also turn off quick ask (#2411)

* fix: app glitches 1s generating response before starting model (#2412)

* fix: disable experimental feature should also disable vulkan (#2414)

* fix: model load stuck on windows when can't get CPU core count (#2413)

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>

* feat: TensorRT-LLM engine update support (#2415)

* fix: engine update

* chore: add remove prepopulated models

Signed-off-by: James <james@jan.ai>

* update tinyjensen url

Signed-off-by: James <james@jan.ai>

* update llamacorn

Signed-off-by: James <james@jan.ai>

* update Mistral 7B Instruct v0.1 int4

Signed-off-by: James <james@jan.ai>

* update tensorrt

Signed-off-by: James <james@jan.ai>

* update

Signed-off-by: hiro <hiro@jan.ai>

* update

Signed-off-by: James <james@jan.ai>

* prettier

Signed-off-by: James <james@jan.ai>

* update mistral config

Signed-off-by: James <james@jan.ai>

* fix some lint

Signed-off-by: James <james@jan.ai>

---------

Signed-off-by: James <james@jan.ai>
Signed-off-by: hiro <hiro@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: hiro <hiro@jan.ai>

* Tensorrt LLM disable turing support (#2418)

Co-authored-by: Hien To <tominhhien97@gmail.com>

* chore: add prompt template tensorrtllm (#2375)

* chore: add prompt template tensorrtllm

* Add Prompt template for mistral and correct model metadata

---------

Co-authored-by: Hien To <tominhhien97@gmail.com>

* fix: correct tensorrt mistral model.json (#2419)

---------

Signed-off-by: James <james@jan.ai>
Signed-off-by: hiro <hiro@jan.ai>
Co-authored-by: Louis <louis@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: hiro <hiro@jan.ai>
Co-authored-by: hiento09 <136591877+hiento09@users.noreply.github.com>
Co-authored-by: Hien To <tominhhien97@gmail.com>

---------

Signed-off-by: James <james@jan.ai>
Signed-off-by: hiro <hiro@jan.ai>
Co-authored-by: NamH <NamNh0122@gmail.com>
Co-authored-by: James <james@jan.ai>
Co-authored-by: hiro <hiro@jan.ai>
Co-authored-by: hiento09 <136591877+hiento09@users.noreply.github.com>
Co-authored-by: Hien To <tominhhien97@gmail.com>
2024-03-19 12:20:09 +07:00
NamH
ed6bd14e02
chore: temporary remove linux from tensorrt support (#2386)
Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
2024-03-15 23:02:42 +07:00
NamH
5f19983de1
fix: some regressions for tensorrt nightly build (#2380)
* fix: some regressions for tensorrt nightly build

Signed-off-by: James <james@jan.ai>
---------

Signed-off-by: hiro <hiro@jan.ai>
Signed-off-by: James <james@jan.ai>
Co-authored-by: hiro <hiro@jan.ai>
Co-authored-by: James <james@jan.ai>
2024-03-15 17:45:56 +07:00
Louis
2d622614bf
Update models.json (#2382) 2024-03-15 16:37:00 +07:00
Louis
58e12f35c9
fix: wrong engine handling (#2363) 2024-03-14 23:59:42 +07:00
Louis
758afdbeb4
fix: incompatible GPU error message (#2357)
* fix: incompatible GPU error message

* fix: change port
2024-03-14 22:11:55 +07:00
Louis
aab8ee84af
fix: disable rag & stream settings from tensorrt model.json (#2351) 2024-03-14 19:04:31 +07:00
Louis
d85d02693b
feat: Nitro-Tensorrt-LLM Extension (#2280)
* feat: tensorrt-llm-extension

* fix: loading

* feat: add download tensorrt llm runner

Signed-off-by: James <james@jan.ai>

* feat: update to rollupjs instead of webpack for monitoring extension

Signed-off-by: James <james@jan.ai>

* feat: move update nvidia info to monitor extension

Signed-off-by: James <james@jan.ai>

* allow download tensorrt

Signed-off-by: James <james@jan.ai>

* update

Signed-off-by: James <james@jan.ai>

* allow download tensor rt based on gpu setting

Signed-off-by: James <james@jan.ai>

* update downloaded models

Signed-off-by: James <james@jan.ai>

* feat: add extension compatibility

* dynamic tensor rt engines

Signed-off-by: James <james@jan.ai>

* update models

Signed-off-by: James <james@jan.ai>

* chore: remove ts-ignore

* feat: getting installation state from extension

Signed-off-by: James <james@jan.ai>

* chore: adding type for decompress

Signed-off-by: James <james@jan.ai>

* feat: update according Louis's comment

Signed-off-by: James <james@jan.ai>

* feat: add progress for installing extension

Signed-off-by: James <james@jan.ai>

* chore: remove args from extension installation

* fix: model download does not work properly

* fix: do not allow user to stop tensorrtllm inference

* fix: extension installed style

* fix: download tensorrt does not update state

Signed-off-by: James <james@jan.ai>

* chore: replace int4 by fl16

* feat: modal for installing extension

Signed-off-by: James <james@jan.ai>

* fix: start download immediately after press install

Signed-off-by: James <james@jan.ai>

* fix: error switching between engines

* feat: rename inference provider to ai engine and refactor to core

* fix: missing ulid

* fix: core bundler

* feat: add cancel extension installing

Signed-off-by: James <james@jan.ai>

* remove mocking for mac

Signed-off-by: James <james@jan.ai>

* fix: show models only when extension is ready

* add tensorrt badge for model

Signed-off-by: James <james@jan.ai>

* fix: copy

* fix: add compatible check (#2342)

* fix: add compatible check

Signed-off-by: James <james@jan.ai>

* fix: copy

* fix: font

* fix: copy

* fix: broken monitoring extension

* chore: bump engine

* fix: copy

* fix: model copy

* fix: copy

* fix: model json

---------

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: Louis <louis@jan.ai>

* fix: vulkan support

* fix: installation button padding

* fix: empty script

* fix: remove hard code string

---------

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: NamH <NamNh0122@gmail.com>
2024-03-14 14:07:22 +07:00