hiro
539f11e583
feat: Add api to kill nitro
2023-12-13 16:35:59 +07:00
hiro
7f60265b3e
chore: Bump nitro to 0.1.27 to support api to kill process
2023-12-13 16:35:37 +07:00
Louis
3266014b29
fix: 963 can not run openai models on windows ( #974 )
2023-12-13 16:26:26 +07:00
hiro
5f404e2c3f
feat: Add prompt template resolver feature to system_prompt, ai_prompt, user_prompt
2023-12-13 14:29:34 +07:00
hiro
587f5addfa
fix: Fix issues based on Louis comments
2023-12-13 01:27:18 +07:00
hiro
f2688771ff
feat: Add triton trtllm for engine for remote models
2023-12-13 01:24:36 +07:00
hiro
8f5c5e1e42
chore: Bump nitro to 0.1.26
2023-12-12 19:41:48 +07:00
hiro
14f83ddb70
fix: Revert drogon thread to 1 instead of CPU num
2023-12-12 07:27:25 +07:00
hiro
16c66e968c
chore: Update n_threads to cpu_threads
2023-12-12 06:15:22 +07:00
hiro
0e63689eae
fix: inference engine nitro stopModel undefined in module.ts
2023-12-12 06:15:22 +07:00
hiro
f528e9ea77
fix: Update inference nitro with n_threads equals to physcial core num
2023-12-12 06:15:22 +07:00
hiro
cafdaaaccd
feat: Nitro sensing hardware init
2023-12-12 06:15:22 +07:00
NamH
121dc119f1
feat: adding model params ( #886 )
...
* feat: adding model params
Signed-off-by: James <james@jan.ai>
* chore: inference request parameter
* Improve ui right panel model params
* Remove unused import
* Update slider track for darkmode
---------
Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: Louis <louis@jan.ai>
Co-authored-by: Faisal Amir <urmauur@gmail.com>
2023-12-11 23:58:24 +07:00
hiro
f2eb8635da
chore: Bumpt nitro bin version to version 0.1.23
2023-12-11 20:53:53 +07:00
hiro
0522d2fe4d
fix: inference openai compatible accepts parameters
2023-12-10 20:16:19 +07:00
hiro
3903c10c5b
fix: inference nitro accepts model parameters
2023-12-10 20:16:05 +07:00
hiro
bbffaafcdc
feat: Added support for Azure OpenAI API
2023-12-09 00:56:39 +07:00
hiro
7e3e648e2a
fix: inference extensions small syntax fix
2023-12-09 00:56:24 +07:00
hiro
c32ad0aff7
fix: small change in nitro bin location
2023-12-09 00:42:48 +07:00
hiro
0ef9a581d3
fix: BAT for nitro
2023-12-09 00:36:55 +07:00
hiro
f9e73b0bbd
fix: Change base_url to full_url
2023-12-08 23:07:30 +07:00
Louis
b939692187
chore: stop inference event
2023-12-08 23:07:30 +07:00
hiro
1177007ad6
fix: Update code based on comments from @james
2023-12-08 23:07:30 +07:00
hiro
9daee14167
fix: Add hack waiting for model loading
2023-12-08 23:06:49 +07:00
hiro
e1190ec41c
fix: Enforce openai inference to load settings from openai.json onLoad and onModelInit
2023-12-08 23:06:49 +07:00
hiro
5f3cf2bcce
fix: Temporary disable model parameters spreading
2023-12-08 23:06:49 +07:00
hiro
3987fdc95b
feat: Add nitro inference engine stop model handler
2023-12-08 23:06:49 +07:00
hiro
44bfcaabd9
fix: Add type def global for nitro extension - inference
2023-12-08 23:06:49 +07:00
hiro
a2cf42ac76
fix: Spreading model.parameters
2023-12-08 23:06:49 +07:00
hiro
f34024a329
fix: spreading model.paramters object and update chatCompletion route
2023-12-08 23:06:49 +07:00
hiro
7ed8c31629
fix: Add dynamic values from engine settings and model params
2023-12-08 23:06:49 +07:00
hiro
0c3e23665b
fix: Add dynamic values from engine settings and model params
2023-12-08 23:06:49 +07:00
hiro
16f2ffe9b4
fix: Add dynamic values from engine settings and model params to inference request
2023-12-08 23:06:49 +07:00
hiro
4f2a3b7b27
fix: Add dynamic values from engine settings and model params to inference request
2023-12-08 23:06:49 +07:00
hiro
516e226290
chore: Move interface definition location
2023-12-08 23:06:49 +07:00
hiro
b970e97869
chore: Move interface definition location
2023-12-08 23:06:49 +07:00
hiro
486c5b8ca0
fix: Update openai with read/ write for engine.json
2023-12-08 23:06:49 +07:00
hiro
2c648caa5f
fix: Update nitro with read/ write for engine.json
2023-12-08 23:06:49 +07:00
hiro
56b778675f
feat: Add nitro engine json reader and writer
2023-12-08 23:06:08 +07:00
hiro
8ab36d7cb2
feat: Add openai engine json reader and writer
2023-12-08 23:06:08 +07:00
hiro
9a18d3133c
chore: refactor openai file structure
2023-12-08 23:06:08 +07:00
hiro
68ee6693ba
chore: Remove default nitro.json file
2023-12-08 23:06:08 +07:00
hiro
6d3bf24d5c
chore: remove gitkeep
2023-12-08 23:06:08 +07:00
hiro
d69f0e3321
chore: Update openai engine
2023-12-08 23:06:08 +07:00
hiro
337da50840
fix: Add fs to read and write nitro engine settings
2023-12-08 23:06:08 +07:00
hiro
19637c40bf
feat: Add nitro engine settings
2023-12-08 23:06:08 +07:00
hiro
48fd8de30c
feat: Init commit for inference engine for openai extension
2023-12-08 23:06:08 +07:00
hiro
c01737ff69
refactor: Change inference-extension to inference-nitro-extension
2023-12-08 23:06:08 +07:00
hiro
52d61ddc9f
fix(nitro): Use nitro at 127.0.0.1 instead of 0.0.0.0 to prevent warning issue
2023-12-06 16:04:06 +07:00
Louis
f3fbade975
fix: delete message break the entire thread ( #869 )
2023-12-06 11:41:16 +07:00
