jan/extensions/huggingface-extension
Louis d85d02693b
feat: Nitro-Tensorrt-LLM Extension (#2280)
* feat: tensorrt-llm-extension

* fix: loading

* feat: add download tensorrt llm runner

Signed-off-by: James <james@jan.ai>

* feat: update to rollupjs instead of webpack for monitoring extension

Signed-off-by: James <james@jan.ai>

* feat: move update nvidia info to monitor extension

Signed-off-by: James <james@jan.ai>

* allow download tensorrt

Signed-off-by: James <james@jan.ai>

* update

Signed-off-by: James <james@jan.ai>

* allow download tensor rt based on gpu setting

Signed-off-by: James <james@jan.ai>

* update downloaded models

Signed-off-by: James <james@jan.ai>

* feat: add extension compatibility

* dynamic tensor rt engines

Signed-off-by: James <james@jan.ai>

* update models

Signed-off-by: James <james@jan.ai>

* chore: remove ts-ignore

* feat: getting installation state from extension

Signed-off-by: James <james@jan.ai>

* chore: adding type for decompress

Signed-off-by: James <james@jan.ai>

* feat: update according Louis's comment

Signed-off-by: James <james@jan.ai>

* feat: add progress for installing extension

Signed-off-by: James <james@jan.ai>

* chore: remove args from extension installation

* fix: model download does not work properly

* fix: do not allow user to stop tensorrtllm inference

* fix: extension installed style

* fix: download tensorrt does not update state

Signed-off-by: James <james@jan.ai>

* chore: replace int4 by fl16

* feat: modal for installing extension

Signed-off-by: James <james@jan.ai>

* fix: start download immediately after press install

Signed-off-by: James <james@jan.ai>

* fix: error switching between engines

* feat: rename inference provider to ai engine and refactor to core

* fix: missing ulid

* fix: core bundler

* feat: add cancel extension installing

Signed-off-by: James <james@jan.ai>

* remove mocking for mac

Signed-off-by: James <james@jan.ai>

* fix: show models only when extension is ready

* add tensorrt badge for model

Signed-off-by: James <james@jan.ai>

* fix: copy

* fix: add compatible check (#2342)

* fix: add compatible check

Signed-off-by: James <james@jan.ai>

* fix: copy

* fix: font

* fix: copy

* fix: broken monitoring extension

* chore: bump engine

* fix: copy

* fix: model copy

* fix: copy

* fix: model json

---------

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: Louis <louis@jan.ai>

* fix: vulkan support

* fix: installation button padding

* fix: empty script

* fix: remove hard code string

---------

Signed-off-by: James <james@jan.ai>
Co-authored-by: James <james@jan.ai>
Co-authored-by: NamH <NamNh0122@gmail.com>
2024-03-14 14:07:22 +07:00
..

Create a Jan Plugin using Typescript

Use this template to bootstrap the creation of a TypeScript Jan plugin. 🚀

Create Your Own Plugin

To create your own plugin, you can use this repository as a template! Just follow the below instructions:

  1. Click the Use this template button at the top of the repository
  2. Select Create a new repository
  3. Select an owner and name for your new repository
  4. Click Create repository
  5. Clone your new repository

Initial Setup

After you've cloned the repository to your local machine or codespace, you'll need to perform some initial setup steps before you can develop your plugin.

Note

You'll need to have a reasonably modern version of Node.js handy. If you are using a version manager like nodenv or nvm, you can run nodenv install in the root of your repository to install the version specified in package.json. Otherwise, 20.x or later should work!

  1. 🛠️ Install the dependencies

    npm install
    
  2. 🏗️ Package the TypeScript for distribution

    npm run bundle
    
  3. Check your artifact

    There will be a tgz file in your plugin directory now

Update the Plugin Metadata

The package.json file defines metadata about your plugin, such as plugin name, main entry, description and version.

When you copy this repository, update package.json with the name, description for your plugin.

Update the Plugin Code

The src/ directory is the heart of your plugin! This contains the source code that will be run when your plugin extension functions are invoked. You can replace the contents of this directory with your own code.

There are a few things to keep in mind when writing your plugin code:

  • Most Jan Plugin Extension functions are processed asynchronously. In index.ts, you will see that the extension function will return a Promise<any>.

    import { core } from "@janhq/core";
    
    function onStart(): Promise<any> {
      return core.invokePluginFunc(MODULE_PATH, "run", 0);
    }
    

    For more information about the Jan Plugin Core module, see the documentation.

So, what are you waiting for? Go ahead and start customizing your plugin!