hiento09
7c0b45f0c5
Fix switch thread crash nitro windows linux ( #1214 )
...
* Add fix bug #1204 on windows
* nitro gpu exit on kill
* correct bat script syntax
* Remove wait 5 second before start nitro
---------
Co-authored-by: Hien To <tominhhien97@gmail.com>
2023-12-26 17:33:20 +07:00
hiento09
4f93e14d16
Fix token speed slow in machine has multi gpus ( #1157 )
...
* Update bat script windows choose GPU has highest ram to start nitro
* Update bash script for linux to choose gpu has highest vram
---------
Co-authored-by: Hien To <tominhhien97@gmail.com>
2023-12-21 15:38:21 +07:00
Louis
e0370210a7
fix : #1096 yield error message upon thread switching ( #1109 )
2023-12-20 14:58:47 +07:00
NamH
84fb5ef346
fix(Thread): #1042 allow create new thread by clicking Use in Jan Hub ( #1103 )
...
Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
2023-12-19 17:06:57 +07:00
Louis
4653030bc1
fix : #1097 streaming response is replaced by error message ( #1099 )
2023-12-19 16:42:13 +07:00
Louis
e3c1787d57
fix: app failed to load model sometime due to race condition ( #1071 )
2023-12-19 11:10:07 +07:00
hiento09
9398b734a1
Posthog disable click event and increase timeout for nitro load model request ( #1060 )
...
Co-authored-by: Service Account <service@jan.ai>
2023-12-18 21:10:36 +07:00
hiento09
fde176955a
bump nitro version to 0.1.30 ( #1036 )
2023-12-15 17:39:49 +07:00
0xSage
c556427905
more copy nits
2023-12-15 16:33:49 +08:00
Faisal Amir
4f41dab9b5
feat: move stop inference button into the send button
2023-12-14 16:40:51 +07:00
hiro
257011309f
fix: Kill nitro many times app onDispose
2023-12-13 16:47:34 +07:00
hiro
96fa392fba
Update extensions/inference-nitro-extension/src/module.ts
...
Co-authored-by: Louis <louis@jan.ai>
2023-12-13 16:40:56 +07:00
hiro
539f11e583
feat: Add api to kill nitro
2023-12-13 16:35:59 +07:00
hiro
7f60265b3e
chore: Bump nitro to 0.1.27 to support api to kill process
2023-12-13 16:35:37 +07:00
hiro
5f404e2c3f
feat: Add prompt template resolver feature to system_prompt, ai_prompt, user_prompt
2023-12-13 14:29:34 +07:00
hiro
8f5c5e1e42
chore: Bump nitro to 0.1.26
2023-12-12 19:41:48 +07:00
hiro
14f83ddb70
fix: Revert drogon thread to 1 instead of CPU num
2023-12-12 07:27:25 +07:00
hiro
16c66e968c
chore: Update n_threads to cpu_threads
2023-12-12 06:15:22 +07:00
hiro
0e63689eae
fix: inference engine nitro stopModel undefined in module.ts
2023-12-12 06:15:22 +07:00
hiro
f528e9ea77
fix: Update inference nitro with n_threads equals to physcial core num
2023-12-12 06:15:22 +07:00
hiro
cafdaaaccd
feat: Nitro sensing hardware init
2023-12-12 06:15:22 +07:00
NamH
121dc119f1
feat: adding model params ( #886 )
...
* feat: adding model params
Signed-off-by: James <james@jan.ai>
* chore: inference request parameter
* Improve ui right panel model params
* Remove unused import
* Update slider track for darkmode
---------
Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: Louis <louis@jan.ai>
Co-authored-by: Faisal Amir <urmauur@gmail.com>
2023-12-11 23:58:24 +07:00
hiro
f2eb8635da
chore: Bumpt nitro bin version to version 0.1.23
2023-12-11 20:53:53 +07:00
hiro
3903c10c5b
fix: inference nitro accepts model parameters
2023-12-10 20:16:05 +07:00
hiro
7e3e648e2a
fix: inference extensions small syntax fix
2023-12-09 00:56:24 +07:00
hiro
c32ad0aff7
fix: small change in nitro bin location
2023-12-09 00:42:48 +07:00
hiro
0ef9a581d3
fix: BAT for nitro
2023-12-09 00:36:55 +07:00
Louis
b939692187
chore: stop inference event
2023-12-08 23:07:30 +07:00
hiro
1177007ad6
fix: Update code based on comments from @james
2023-12-08 23:07:30 +07:00
hiro
9daee14167
fix: Add hack waiting for model loading
2023-12-08 23:06:49 +07:00
hiro
5f3cf2bcce
fix: Temporary disable model parameters spreading
2023-12-08 23:06:49 +07:00
hiro
3987fdc95b
feat: Add nitro inference engine stop model handler
2023-12-08 23:06:49 +07:00
hiro
44bfcaabd9
fix: Add type def global for nitro extension - inference
2023-12-08 23:06:49 +07:00
hiro
a2cf42ac76
fix: Spreading model.parameters
2023-12-08 23:06:49 +07:00
hiro
0c3e23665b
fix: Add dynamic values from engine settings and model params
2023-12-08 23:06:49 +07:00
hiro
4f2a3b7b27
fix: Add dynamic values from engine settings and model params to inference request
2023-12-08 23:06:49 +07:00
hiro
b970e97869
chore: Move interface definition location
2023-12-08 23:06:49 +07:00
hiro
2c648caa5f
fix: Update nitro with read/ write for engine.json
2023-12-08 23:06:49 +07:00
hiro
56b778675f
feat: Add nitro engine json reader and writer
2023-12-08 23:06:08 +07:00
hiro
68ee6693ba
chore: Remove default nitro.json file
2023-12-08 23:06:08 +07:00
hiro
6d3bf24d5c
chore: remove gitkeep
2023-12-08 23:06:08 +07:00
hiro
337da50840
fix: Add fs to read and write nitro engine settings
2023-12-08 23:06:08 +07:00
hiro
19637c40bf
feat: Add nitro engine settings
2023-12-08 23:06:08 +07:00
hiro
c01737ff69
refactor: Change inference-extension to inference-nitro-extension
2023-12-08 23:06:08 +07:00
