From baa0398fb4a0325515fd90f23b116a5cd2fa1811 Mon Sep 17 00:00:00 2001 From: nam-john-ho <146448941+nam-john-ho@users.noreply.github.com> Date: Tue, 3 Oct 2023 17:18:50 +0700 Subject: [PATCH] Update adr-001-jan-deployable-cloud-native.md --- adr/adr-001-jan-deployable-cloud-native.md | 50 ++++++++++++++-------- 1 file changed, 31 insertions(+), 19 deletions(-) diff --git a/adr/adr-001-jan-deployable-cloud-native.md b/adr/adr-001-jan-deployable-cloud-native.md index 74be7a078..83c18adc0 100644 --- a/adr/adr-001-jan-deployable-cloud-native.md +++ b/adr/adr-001-jan-deployable-cloud-native.md @@ -7,41 +7,53 @@ ## Authors - @nam-john-ho +- @louis ## Context ### Status Quo -User doesn't have a local GPU machine but wants to run Jan on a rented server -User wants a quick, fast way to experiment with Jan on a rented GPU -https://github.com/janhq/jan/issues/255 +* User doesn't have a local GPU machine but wants to run Jan on a rented server +* User wants a quick, fast way to experiment with Jan on a rented GPU +* https://github.com/janhq/jan/issues/255 ## Decision -This ADR aims to outline design decisions for deploying Jan in cloud native environments such as: Runpod, AWS, Azure, GCP in a fast and simple way. -The current code-base should not change too much. -The current plugins should be reusable across enviroments (Desktop, Cloud-native). -Simple authentication (username/password) should be supported. +* This ADR aims to outline design decisions for deploying Jan in cloud native environments such as: Runpod, AWS, Azure, GCP in a fast and simple way. +* The current code-base should not change too much. +* The current plugins must be reusable across enviroments (Desktop, Cloud-native). +* Simple authentication (username/password) must be supported. ### Key Design Decisions -![alt text](images/adr-001-01.png "Title") - +![Key Design](images/adr-001-01.png "Key Design") +Introduce 2 components in Jan: +- Middleware: responsible for routing the user interface to the appropriate platform (Electron/WebApp) that is built when packaged. +- Http server: a http server on cloud environment which interacts with plugin directly. ### Detailed Design - - +#### FE +- Middleware: TBD +- Httpserver: TBD +- Custom build for httpweb/electron: TBD +#### Devops: +* Allow to pass username/password as environment variables +* Assign a Public IP to the instance +* Customize instance types on-demand. +- Runpod: TBD +- AWS: TBD +- Azure: TBD +- GCP: TBD ## Alternative Approaches - - +Separated server process runs along side with electron. https://github.com/janhq/jan/pull/184/commits/6005409a945bb0e80a61132b9eb77f47f19d0aa6 ## Considerations +## References - -https://www.runpod.io/console/templates -https://repost.aws/articles/ARQ0Tz9eorSL6EAus7XPMG-Q/how-to-install-textgen-webui-on-aws -https://www.youtube.com/watch?v=_59AsSyMERQ -https://gpus.llm-utils.org/running-llama-2-on-runpod-with-oobaboogas-text-generation-webui/ -https://medium.com/@jarimh1984/installing-oobabooga-and-oobabooga-api-to-runpod-cloud-step-by-step-tutorial-47457974dfa5 +- https://www.runpod.io/console/templates +- https://repost.aws/articles/ARQ0Tz9eorSL6EAus7XPMG-Q/how-to-install-textgen-webui-on-aws +- https://www.youtube.com/watch?v=_59AsSyMERQ +- https://gpus.llm-utils.org/running-llama-2-on-runpod-with-oobaboogas-text-generation-webui/ +- https://medium.com/@jarimh1984/installing-oobabooga-and-oobabooga-api-to-runpod-cloud-step-by-step-tutorial-47457974dfa5