Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update adr-001-jan-deployable-cloud-native.md

Browse Source
This commit is contained in:
nam-john-ho 2023-10-03 17:18:50 +07:00 committed by GitHub
parent 7257fe95ee
commit baa0398fb4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 31 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
adr/adr-001-jan-deployable-cloud-native.md
View File

@ -7,41 +7,53 @@
## Authors ## Authors
- @nam-john-ho - @nam-john-ho
- @louis
## Context ## Context
### Status Quo ### Status Quo
User doesn't have a local GPU machine but wants to run Jan on a rented server * User doesn't have a local GPU machine but wants to run Jan on a rented server
User wants a quick, fast way to experiment with Jan on a rented GPU * User wants a quick, fast way to experiment with Jan on a rented GPU
https://github.com/janhq/jan/issues/255 * https://github.com/janhq/jan/issues/255
## Decision ## Decision
This ADR aims to outline design decisions for deploying Jan in cloud native environments such as: Runpod, AWS, Azure, GCP in a fast and simple way. * This ADR aims to outline design decisions for deploying Jan in cloud native environments such as: Runpod, AWS, Azure, GCP in a fast and simple way.
The current code-base should not change too much. * The current code-base should not change too much.
The current plugins should be reusable across enviroments (Desktop, Cloud-native). * The current plugins must be reusable across enviroments (Desktop, Cloud-native).
Simple authentication (username/password) should be supported. * Simple authentication (username/password) must be supported.
### Key Design Decisions ### Key Design Decisions
![alt text](images/adr-001-01.png "Title") ![Key Design](images/adr-001-01.png "Key Design")
Introduce 2 components in Jan:
- Middleware: responsible for routing the user interface to the appropriate platform (Electron/WebApp) that is built when packaged.
- Http server: a http server on cloud environment which interacts with plugin directly.
### Detailed Design ### Detailed Design
#### FE
- Middleware: TBD
- Httpserver: TBD
- Custom build for httpweb/electron: TBD
#### Devops:
* Allow to pass username/password as environment variables
* Assign a Public IP to the instance
* Customize instance types on-demand.
- Runpod: TBD
- AWS: TBD
- Azure: TBD
- GCP: TBD
## Alternative Approaches ## Alternative Approaches
Separated server process runs along side with electron. https://github.com/janhq/jan/pull/184/commits/6005409a945bb0e80a61132b9eb77f47f19d0aa6
## Considerations ## Considerations
## References
- https://www.runpod.io/console/templates
https://www.runpod.io/console/templates - https://repost.aws/articles/ARQ0Tz9eorSL6EAus7XPMG-Q/how-to-install-textgen-webui-on-aws
https://repost.aws/articles/ARQ0Tz9eorSL6EAus7XPMG-Q/how-to-install-textgen-webui-on-aws - https://www.youtube.com/watch?v=_59AsSyMERQ
https://www.youtube.com/watch?v=_59AsSyMERQ - https://gpus.llm-utils.org/running-llama-2-on-runpod-with-oobaboogas-text-generation-webui/
https://gpus.llm-utils.org/running-llama-2-on-runpod-with-oobaboogas-text-generation-webui/ - https://medium.com/@jarimh1984/installing-oobabooga-and-oobabooga-api-to-runpod-cloud-step-by-step-tutorial-47457974dfa5
https://medium.com/@jarimh1984/installing-oobabooga-and-oobabooga-api-to-runpod-cloud-step-by-step-tutorial-47457974dfa5