Nicholai/jan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'dev' into chore/get-to-3.5-performance

Browse Source
This commit is contained in:
hieu-jan 2024-03-22 15:02:28 +09:00
commit 1858ab1fec
362 changed files with 8626 additions and 4823 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

39
.github/workflows/jan-electron-linter-and-test.yml vendored
View File

@ -22,6 +22,7 @@ on:
branches:
- main
- dev
- release/**
paths:
- "electron/**"
- .github/workflows/jan-electron-linter-and-test.yml
@ -66,17 +67,51 @@ jobs:
CSC_IDENTITY_AUTO_DISCOVERY: "false"
test-on-windows:
if: github.event_name == 'push'
strategy:
fail-fast: false
matrix:
antivirus-tools: ['mcafee', 'default-windows-security','bit-defender']
runs-on: windows-desktop-${{ matrix.antivirus-tools }}
steps:
- name: Clean workspace
run: |
Remove-Item -Path .\* -Force -Recurse
Remove-Item -Path "\\?\$(Get-Location)\*" -Force -Recurse
$path = "$Env:APPDATA\jan"
if (Test-Path $path) {
Remove-Item $path -Recurse -Force
Remove-Item "\\?\$path" -Recurse -Force
} else {
Write-Output "Folder does not exist."
}
- name: Getting the repo
uses: actions/checkout@v3
- name: Installing node
uses: actions/setup-node@v1
with:
node-version: 20
# Clean cache, continue on error
- name: "Cleanup cache"
shell: powershell
continue-on-error: true
run: |
make clean
- name: Linter and test
shell: powershell
run: |
make test
test-on-windows-pr:
if: github.event_name == 'pull_request'
runs-on: windows-desktop-default-windows-security
steps:
- name: Clean workspace
run: |
Remove-Item -Path "\\?\$(Get-Location)\*" -Force -Recurse
$path = "$Env:APPDATA\jan"
if (Test-Path $path) {
Remove-Item "\\?\$path" -Recurse -Force
} else {
Write-Output "Folder does not exist."
}

5
Makefile
View File

@ -53,15 +53,17 @@ build: check-file-counts
clean:
ifeq ($(OS),Windows_NT)
powershell -Command "Get-ChildItem -Path . -Include node_modules, .next, dist, build, out -Recurse -Directory | Remove-Item -Recurse -Force"
powershell -Command "Get-ChildItem -Path . -Include package-lock.json -Recurse -File | Remove-Item -Recurse -Force"
powershell -Command "Remove-Item -Recurse -Force ./pre-install/*.tgz"
powershell -Command "Remove-Item -Recurse -Force ./electron/pre-install/*.tgz"
rmdir /s /q "%USERPROFILE%\jan\extensions"
powershell -Command "if (Test-Path \"$($env:USERPROFILE)\jan\extensions\") { Remove-Item -Path \"$($env:USERPROFILE)\jan\extensions\" -Recurse -Force }"
else ifeq ($(shell uname -s),Linux)
find . -name "node_modules" -type d -prune -exec rm -rf '{}' +
find . -name ".next" -type d -exec rm -rf '{}' +
find . -name "dist" -type d -exec rm -rf '{}' +
find . -name "build" -type d -exec rm -rf '{}' +
find . -name "out" -type d -exec rm -rf '{}' +
find . -name "packake-lock.json" -type f -exec rm -rf '{}' +
rm -rf ./pre-install/*.tgz
rm -rf ./electron/pre-install/*.tgz
rm -rf "~/jan/extensions"
@ -72,6 +74,7 @@ else
find . -name "dist" -type d -exec rm -rf '{}' +
find . -name "build" -type d -exec rm -rf '{}' +
find . -name "out" -type d -exec rm -rf '{}' +
find . -name "packake-lock.json" -type f -exec rm -rf '{}' +
rm -rf ./pre-install/*.tgz
rm -rf ./electron/pre-install/*.tgz
rm -rf ~/jan/extensions

21
README.md
View File

@ -43,31 +43,31 @@ Jan is an open-source ChatGPT alternative that runs 100% offline on your compute
<tr style="text-align:center">
<td style="text-align:center"><b>Stable (Recommended)</b></td>
<td style="text-align:center">
<a href='https://github.com/janhq/jan/releases/download/v0.4.8/jan-win-x64-0.4.8.exe'>
<a href='https://github.com/janhq/jan/releases/download/v0.4.9/jan-win-x64-0.4.9.exe'>
<img src='./docs/static/img/windows.png' style="height:14px; width: 14px" />
<b>jan.exe</b>
</a>
</td>
<td style="text-align:center">
<a href='https://github.com/janhq/jan/releases/download/v0.4.8/jan-mac-x64-0.4.8.dmg'>
<a href='https://github.com/janhq/jan/releases/download/v0.4.9/jan-mac-x64-0.4.9.dmg'>
<img src='./docs/static/img/mac.png' style="height:15px; width: 15px" />
<b>Intel</b>
</a>
</td>
<td style="text-align:center">
<a href='https://github.com/janhq/jan/releases/download/v0.4.8/jan-mac-arm64-0.4.8.dmg'>
<a href='https://github.com/janhq/jan/releases/download/v0.4.9/jan-mac-arm64-0.4.9.dmg'>
<img src='./docs/static/img/mac.png' style="height:15px; width: 15px" />
<b>M1/M2</b>
</a>
</td>
<td style="text-align:center">
<a href='https://github.com/janhq/jan/releases/download/v0.4.8/jan-linux-amd64-0.4.8.deb'>
<a href='https://github.com/janhq/jan/releases/download/v0.4.9/jan-linux-amd64-0.4.9.deb'>
<img src='./docs/static/img/linux.png' style="height:14px; width: 14px" />
<b>jan.deb</b>
</a>
</td>
<td style="text-align:center">
<a href='https://github.com/janhq/jan/releases/download/v0.4.8/jan-linux-x86_64-0.4.8.AppImage'>
<a href='https://github.com/janhq/jan/releases/download/v0.4.9/jan-linux-x86_64-0.4.9.AppImage'>
<img src='./docs/static/img/linux.png' style="height:14px; width: 14px" />
<b>jan.AppImage</b>
</a>
@ -76,31 +76,31 @@ Jan is an open-source ChatGPT alternative that runs 100% offline on your compute
<tr style="text-align:center">
<td style="text-align:center"><b>Experimental (Nightly Build)</b></td>
<td style="text-align:center">
<a href='https://delta.jan.ai/latest/jan-win-x64-0.4.8-324.exe'>
<a href='https://delta.jan.ai/latest/jan-win-x64-0.4.9-337.exe'>
<img src='./docs/static/img/windows.png' style="height:14px; width: 14px" />
<b>jan.exe</b>
</a>
</td>
<td style="text-align:center">
<a href='https://delta.jan.ai/latest/jan-mac-x64-0.4.8-324.dmg'>
<a href='https://delta.jan.ai/latest/jan-mac-x64-0.4.9-337.dmg'>
<img src='./docs/static/img/mac.png' style="height:15px; width: 15px" />
<b>Intel</b>
</a>
</td>
<td style="text-align:center">
<a href='https://delta.jan.ai/latest/jan-mac-arm64-0.4.8-324.dmg'>
<a href='https://delta.jan.ai/latest/jan-mac-arm64-0.4.9-337.dmg'>
<img src='./docs/static/img/mac.png' style="height:15px; width: 15px" />
<b>M1/M2</b>
</a>
</td>
<td style="text-align:center">
<a href='https://delta.jan.ai/latest/jan-linux-amd64-0.4.8-324.deb'>
<a href='https://delta.jan.ai/latest/jan-linux-amd64-0.4.9-337.deb'>
<img src='./docs/static/img/linux.png' style="height:14px; width: 14px" />
<b>jan.deb</b>
</a>
</td>
<td style="text-align:center">
<a href='https://delta.jan.ai/latest/jan-linux-x86_64-0.4.8-324.AppImage'>
<a href='https://delta.jan.ai/latest/jan-linux-x86_64-0.4.9-337.AppImage'>
<img src='./docs/static/img/linux.png' style="height:14px; width: 14px" />
<b>jan.AppImage</b>
</a>
@ -327,6 +327,7 @@ Jan builds on top of other open-source projects:
- [llama.cpp](https://github.com/ggerganov/llama.cpp)
- [LangChain](https://github.com/langchain-ai)
- [TensorRT](https://github.com/NVIDIA/TensorRT)
- [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM)
## Contact

16
core/package.json
View File

@ -8,8 +8,8 @@
],
"homepage": "https://jan.ai",
"license": "AGPL-3.0",
"main": "dist/core.umd.js",
"module": "dist/core.es5.js",
"main": "dist/core.es5.js",
"module": "dist/core.cjs.js",
"typings": "dist/types/index.d.ts",
"files": [
"dist",
@ -17,8 +17,7 @@
],
"author": "Jan <service@jan.ai>",
"exports": {
".": "./dist/core.umd.js",
"./sdk": "./dist/core.umd.js",
".": "./dist/core.es5.js",
"./node": "./dist/node/index.cjs.js"
},
"typesVersions": {
@ -27,10 +26,6 @@
"./dist/core.es5.js.map",
"./dist/types/index.d.ts"
],
"sdk": [
"./dist/core.es5.js.map",
"./dist/types/index.d.ts"
],
"node": [
"./dist/node/index.cjs.js.map",
"./dist/types/node/index.d.ts"
@ -45,8 +40,9 @@
"start": "rollup -c rollup.config.ts -w"
},
"devDependencies": {
"@rollup/plugin-replace": "^5.0.5",
"@types/jest": "^29.5.12",
"@types/node": "^12.0.2",
"@types/node": "^20.11.4",
"eslint": "8.57.0",
"eslint-plugin-jest": "^27.9.0",
"jest": "^29.7.0",
@ -63,6 +59,6 @@
},
"dependencies": {
"rxjs": "^7.8.1",
"ulid": "^2.3.0"
"ulidx": "^2.3.0"
}
}

19
core/rollup.config.ts
View File

@ -3,17 +3,16 @@ import commonjs from 'rollup-plugin-commonjs'
import sourceMaps from 'rollup-plugin-sourcemaps'
import typescript from 'rollup-plugin-typescript2'
import json from 'rollup-plugin-json'
import replace from '@rollup/plugin-replace'
const pkg = require('./package.json')
const libraryName = 'core'
export default [
{
input: `src/index.ts`,
output: [
{ file: pkg.main, name: libraryName, format: 'umd', sourcemap: true },
{ file: pkg.module, format: 'es', sourcemap: true },
// { file: pkg.main, name: libraryName, format: 'umd', sourcemap: true },
{ file: pkg.main, format: 'es', sourcemap: true },
],
// Indicate here external modules you don't wanna include in your bundle (i.e.: 'lodash')
external: ['path'],
@ -30,7 +29,13 @@ export default [
// Allow node_modules resolution, so you can use 'external' to control
// which external modules to include in the bundle
// https://github.com/rollup/rollup-plugin-node-resolve#usage
resolve(),
replace({
'node:crypto': 'crypto',
'delimiters': ['"', '"'],
}),
resolve({
browser: true,
}),
// Resolve source maps to the original source
sourceMaps(),
@ -46,7 +51,7 @@ export default [
'pacote',
'@types/pacote',
'@npmcli/arborist',
'ulid',
'ulidx',
'node-fetch',
'fs',
'request',
@ -64,7 +69,7 @@ export default [
// Allow json resolution
json(),
// Compile TypeScript files
typescript({ useTsconfigDeclarationDir: true, exclude: ['src/*.ts', 'src/extensions/**'] }),
typescript({ useTsconfigDeclarationDir: true }),
// Allow bundling cjs modules (unlike webpack, rollup doesn't understand cjs)
commonjs(),
// Allow node_modules resolution, so you can use 'external' to control

4
core/src/api/index.ts
View File

@ -33,7 +33,7 @@ export enum AppRoute {
stopServer = 'stopServer',
log = 'log',
logServer = 'logServer',
systemInformations = 'systemInformations',
systemInformation = 'systemInformation',
showToast = 'showToast',
}
@ -95,6 +95,8 @@ export enum FileManagerRoute {
getUserHomePath = 'getUserHomePath',
fileStat = 'fileStat',
writeBlob = 'writeBlob',
mkdir = 'mkdir',
rm = 'rm',
}
export type ApiFunction = (...args: any[]) => any

34
core/src/core.ts
View File

@ -1,4 +1,4 @@
import { DownloadRequest, FileStat, NetworkConfig } from './types'
import { DownloadRequest, FileStat, NetworkConfig, SystemInformation } from './types'
/**
* Execute a extension module function in main process
@ -13,7 +13,7 @@ const executeOnMain: (extension: string, method: string, ...args: any[]) => Prom
extension,
method,
...args
) => global.core?.api?.invokeExtensionFunc(extension, method, ...args)
) => globalThis.core?.api?.invokeExtensionFunc(extension, method, ...args)
/**
* Downloads a file from a URL and saves it to the local file system.
@ -26,7 +26,7 @@ const executeOnMain: (extension: string, method: string, ...args: any[]) => Prom
const downloadFile: (downloadRequest: DownloadRequest, network?: NetworkConfig) => Promise<any> = (
downloadRequest,
network
) => global.core?.api?.downloadFile(downloadRequest, network)
) => globalThis.core?.api?.downloadFile(downloadRequest, network)
/**
* Aborts the download of a specific file.
@ -34,14 +34,14 @@ const downloadFile: (downloadRequest: DownloadRequest, network?: NetworkConfig)
* @returns {Promise<any>} A promise that resolves when the download has been aborted.
*/
const abortDownload: (fileName: string) => Promise<any> = (fileName) =>
global.core.api?.abortDownload(fileName)
globalThis.core.api?.abortDownload(fileName)
/**
* Gets Jan's data folder path.
*
* @returns {Promise<string>} A Promise that resolves with Jan's data folder path.
*/
const getJanDataFolderPath = (): Promise<string> => global.core.api?.getJanDataFolderPath()
const getJanDataFolderPath = (): Promise<string> => globalThis.core.api?.getJanDataFolderPath()
/**
* Opens the file explorer at a specific path.
@ -49,21 +49,22 @@ const getJanDataFolderPath = (): Promise<string> => global.core.api?.getJanDataF
* @returns {Promise<any>} A promise that resolves when the file explorer is opened.
*/
const openFileExplorer: (path: string) => Promise<any> = (path) =>
global.core.api?.openFileExplorer(path)
globalThis.core.api?.openFileExplorer(path)
/**
* Joins multiple paths together.
* @param paths - The paths to join.
* @returns {Promise<string>} A promise that resolves with the joined path.
*/
const joinPath: (paths: string[]) => Promise<string> = (paths) => global.core.api?.joinPath(paths)
const joinPath: (paths: string[]) => Promise<string> = (paths) =>
globalThis.core.api?.joinPath(paths)
/**
* Retrive the basename from an url.
* @param path - The path to retrieve.
* @returns {Promise<string>} A promise that resolves with the basename.
*/
const baseName: (paths: string) => Promise<string> = (path) => global.core.api?.baseName(path)
const baseName: (paths: string) => Promise<string> = (path) => globalThis.core.api?.baseName(path)
/**
* Opens an external URL in the default web browser.
@ -72,20 +73,20 @@ const baseName: (paths: string) => Promise<string> = (path) => global.core.api?.
* @returns {Promise<any>} - A promise that resolves when the URL has been successfully opened.
*/
const openExternalUrl: (url: string) => Promise<any> = (url) =>
global.core.api?.openExternalUrl(url)
globalThis.core.api?.openExternalUrl(url)
/**
* Gets the resource path of the application.
*
* @returns {Promise<string>} - A promise that resolves with the resource path.
*/
const getResourcePath: () => Promise<string> = () => global.core.api?.getResourcePath()
const getResourcePath: () => Promise<string> = () => globalThis.core.api?.getResourcePath()
/**
* Gets the user's home path.
* @returns return user's home path
*/
const getUserHomePath = (): Promise<string> => global.core.api?.getUserHomePath()
const getUserHomePath = (): Promise<string> => globalThis.core.api?.getUserHomePath()
/**
* Log to file from browser processes.
@ -93,7 +94,7 @@ const getUserHomePath = (): Promise<string> => global.core.api?.getUserHomePath(
* @param message - Message to log.
*/
const log: (message: string, fileName?: string) => void = (message, fileName) =>
global.core.api?.log(message, fileName)
globalThis.core.api?.log(message, fileName)
/**
* Check whether the path is a subdirectory of another path.
@ -104,13 +105,14 @@ const log: (message: string, fileName?: string) => void = (message, fileName) =>
* @returns {Promise<boolean>} - A promise that resolves with a boolean indicating whether the path is a subdirectory.
*/
const isSubdirectory: (from: string, to: string) => Promise<boolean> = (from: string, to: string) =>
global.core.api?.isSubdirectory(from, to)
globalThis.core.api?.isSubdirectory(from, to)
/**
* Get system information
* @returns {Promise<any>} - A promise that resolves with the system information.
*/
const systemInformations: () => Promise<any> = () => global.core.api?.systemInformations()
const systemInformation: () => Promise<SystemInformation> = () =>
globalThis.core.api?.systemInformation()
/**
* Show toast message from browser processes.
@ -119,7 +121,7 @@ const systemInformations: () => Promise<any> = () => global.core.api?.systemInfo
* @returns
*/
const showToast: (title: string, message: string) => void = (title, message) =>
global.core.api?.showToast(title, message)
globalThis.core.api?.showToast(title, message)
/**
* Register extension point function type definition
*/
@ -146,7 +148,7 @@ export {
log,
isSubdirectory,
getUserHomePath,
systemInformations,
systemInformation,
showToast,
FileStat,
}

6
core/src/events.ts
View File

@ -5,7 +5,7 @@
* @param handler The handler function to call when the event is observed.
*/
const on: (eventName: string, handler: Function) => void = (eventName, handler) => {
global.core?.events?.on(eventName, handler)
globalThis.core?.events?.on(eventName, handler)
}
/**
@ -15,7 +15,7 @@ const on: (eventName: string, handler: Function) => void = (eventName, handler)
* @param handler The handler function to call when the event is observed.
*/
const off: (eventName: string, handler: Function) => void = (eventName, handler) => {
global.core?.events?.off(eventName, handler)
globalThis.core?.events?.off(eventName, handler)
}
/**
@ -25,7 +25,7 @@ const off: (eventName: string, handler: Function) => void = (eventName, handler)
* @param object The object to pass to the event callback.
*/
const emit: (eventName: string, object: any) => void = (eventName, object) => {
global.core?.events?.emit(eventName, object)
globalThis.core?.events?.emit(eventName, object)
}
export const events = {

8
core/src/extension.ts
View File

@ -19,6 +19,7 @@ export interface Compatibility {
const ALL_INSTALLATION_STATE = [
'NotRequired', // not required.
'Installed', // require and installed. Good to go.
'Updatable', // require and installed but need to be updated.
'NotInstalled', // require to be installed.
'Corrupted', // require but corrupted. Need to redownload.
] as const
@ -59,6 +60,13 @@ export abstract class BaseExtension implements ExtensionType {
return undefined
}
/**
* Determine if the extension is updatable.
*/
updatable(): boolean {
return false
}
/**
* Determine if the prerequisites for the extension are installed.
*

4
core/src/extensions/ai-engines/AIEngine.ts
View File

@ -14,7 +14,9 @@ export abstract class AIEngine extends BaseExtension {
// The model folder
modelFolder: string = 'models'
abstract models(): Promise<Model[]>
models(): Promise<Model[]> {
return Promise.resolve([])
}
/**
* On extension load, subscribe to events.

29
core/src/extensions/ai-engines/LocalOAIEngine.ts
View File

@ -1,4 +1,4 @@
import { executeOnMain, getJanDataFolderPath, joinPath } from '../../core'
import { executeOnMain, getJanDataFolderPath, joinPath, systemInformation } from '../../core'
import { events } from '../../events'
import { Model, ModelEvent } from '../../types'
import { OAIEngine } from './OAIEngine'
@ -9,9 +9,9 @@ import { OAIEngine } from './OAIEngine'
*/
export abstract class LocalOAIEngine extends OAIEngine {
// The inference engine
abstract nodeModule: string
loadModelFunctionName: string = 'loadModel'
unloadModelFunctionName: string = 'unloadModel'
isRunning: boolean = false
/**
* On extension load, subscribe to events.
@ -19,22 +19,27 @@ export abstract class LocalOAIEngine extends OAIEngine {
onLoad() {
super.onLoad()
// These events are applicable to local inference providers
events.on(ModelEvent.OnModelInit, (model: Model) => this.onModelInit(model))
events.on(ModelEvent.OnModelStop, (model: Model) => this.onModelStop(model))
events.on(ModelEvent.OnModelInit, (model: Model) => this.loadModel(model))
events.on(ModelEvent.OnModelStop, (model: Model) => this.unloadModel(model))
}
/**
* Load the model.
*/
async onModelInit(model: Model) {
async loadModel(model: Model) {
if (model.engine.toString() !== this.provider) return
const modelFolder = await joinPath([await getJanDataFolderPath(), this.modelFolder, model.id])
const res = await executeOnMain(this.nodeModule, this.loadModelFunctionName, {
const systemInfo = await systemInformation()
const res = await executeOnMain(
this.nodeModule,
this.loadModelFunctionName,
{
modelFolder,
model,
})
},
systemInfo
)
if (res?.error) {
events.emit(ModelEvent.OnModelFail, {
@ -45,16 +50,14 @@ export abstract class LocalOAIEngine extends OAIEngine {
} else {
this.loadedModel = model
events.emit(ModelEvent.OnModelReady, model)
this.isRunning = true
}
}
/**
* Stops the model.
*/
onModelStop(model: Model) {
if (model.engine?.toString() !== this.provider) return
this.isRunning = false
unloadModel(model: Model) {
if (model.engine && model.engine?.toString() !== this.provider) return
this.loadedModel = undefined
executeOnMain(this.nodeModule, this.unloadModelFunctionName).then(() => {
events.emit(ModelEvent.OnModelStopped, {})

22
core/src/extensions/ai-engines/OAIEngine.ts
View File

@ -1,5 +1,5 @@
import { requestInference } from './helpers/sse'
import { ulid } from 'ulid'
import { ulid } from 'ulidx'
import { AIEngine } from './AIEngine'
import {
ChatCompletionRole,
@ -23,7 +23,6 @@ import { events } from '../../events'
export abstract class OAIEngine extends AIEngine {
// The inference engine
abstract inferenceUrl: string
abstract nodeModule: string
// Controller to handle stop requests
controller = new AbortController()
@ -38,7 +37,7 @@ export abstract class OAIEngine extends AIEngine {
onLoad() {
super.onLoad()
events.on(MessageEvent.OnMessageSent, (data: MessageRequest) => this.inference(data))
events.on(InferenceEvent.OnInferenceStopped, () => this.onInferenceStopped())
events.on(InferenceEvent.OnInferenceStopped, () => this.stopInference())
}
/**
@ -78,7 +77,13 @@ export abstract class OAIEngine extends AIEngine {
...data.model,
}
requestInference(this.inferenceUrl, data.messages ?? [], model, this.controller).subscribe({
requestInference(
this.inferenceUrl,
data.messages ?? [],
model,
this.controller,
this.headers()
).subscribe({
next: (content: any) => {
const messageContent: ThreadContent = {
type: ContentType.Text,
@ -109,8 +114,15 @@ export abstract class OAIEngine extends AIEngine {
/**
* Stops the inference.
*/
onInferenceStopped() {
stopInference() {
this.isCancelled = true
this.controller?.abort()
}
/**
* Headers for the inference request
*/
headers(): HeadersInit {
return {}
}
}

46
core/src/extensions/ai-engines/RemoteOAIEngine.ts Normal file
View File

@ -0,0 +1,46 @@
import { events } from '../../events'
import { Model, ModelEvent } from '../../types'
import { OAIEngine } from './OAIEngine'
/**
* Base OAI Remote Inference Provider
* Added the implementation of loading and unloading model (applicable to local inference providers)
*/
export abstract class RemoteOAIEngine extends OAIEngine {
// The inference engine
abstract apiKey: string
/**
* On extension load, subscribe to events.
*/
onLoad() {
super.onLoad()
// These events are applicable to local inference providers
events.on(ModelEvent.OnModelInit, (model: Model) => this.loadModel(model))
events.on(ModelEvent.OnModelStop, (model: Model) => this.unloadModel(model))
}
/**
* Load the model.
*/
async loadModel(model: Model) {
if (model.engine.toString() !== this.provider) return
events.emit(ModelEvent.OnModelReady, model)
}
/**
* Stops the model.
*/
unloadModel(model: Model) {
if (model.engine && model.engine.toString() !== this.provider) return
events.emit(ModelEvent.OnModelStopped, {})
}
/**
* Headers for the inference request
*/
override headers(): HeadersInit {
return {
'Authorization': `Bearer ${this.apiKey}`,
'api-key': `${this.apiKey}`,
}
}
}

4
core/src/extensions/ai-engines/helpers/sse.ts
View File

@ -12,7 +12,8 @@ export function requestInference(
id: string
parameters: ModelRuntimeParams
},
controller?: AbortController
controller?: AbortController,
headers?: HeadersInit
): Observable<string> {
return new Observable((subscriber) => {
const requestBody = JSON.stringify({
@ -27,6 +28,7 @@ export function requestInference(
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*',
'Accept': model.parameters.stream ? 'text/event-stream' : 'application/json',
...headers,
},
body: requestBody,
signal: controller?.signal,

1
core/src/extensions/ai-engines/index.ts
View File

@ -1,3 +1,4 @@
export * from './AIEngine'
export * from './OAIEngine'
export * from './LocalOAIEngine'
export * from './RemoteOAIEngine'

3
core/src/extensions/monitoring.ts
View File

@ -1,5 +1,5 @@
import { BaseExtension, ExtensionTypeEnum } from '../extension'
import { GpuSetting, MonitoringInterface } from '../index'
import { GpuSetting, MonitoringInterface, OperatingSystemInfo } from '../index'
/**
* Monitoring extension for system monitoring.
@ -16,4 +16,5 @@ export abstract class MonitoringExtension extends BaseExtension implements Monit
abstract getGpuSetting(): Promise<GpuSetting>
abstract getResourcesInfo(): Promise<any>
abstract getCurrentLoad(): Promise<any>
abstract getOsInfo(): Promise<OperatingSystemInfo>
}

33
core/src/fs.ts
View File

@ -4,7 +4,7 @@ import { FileStat } from './types'
* Writes data to a file at the specified path.
* @returns {Promise<any>} A Promise that resolves when the file is written successfully.
*/
const writeFileSync = (...args: any[]) => global.core.api?.writeFileSync(...args)
const writeFileSync = (...args: any[]) => globalThis.core.api?.writeFileSync(...args)
/**
* Writes blob data to a file at the specified path.
@ -13,47 +13,52 @@ const writeFileSync = (...args: any[]) => global.core.api?.writeFileSync(...args
* @returns
*/
const writeBlob: (path: string, data: string) => Promise<any> = (path, data) =>
global.core.api?.writeBlob(path, data)
globalThis.core.api?.writeBlob(path, data)
/**
* Reads the contents of a file at the specified path.
* @returns {Promise<any>} A Promise that resolves with the contents of the file.
*/
const readFileSync = (...args: any[]) => global.core.api?.readFileSync(...args)
const readFileSync = (...args: any[]) => globalThis.core.api?.readFileSync(...args)
/**
* Check whether the file exists
* @param {string} path
* @returns {boolean} A boolean indicating whether the path is a file.
*/
const existsSync = (...args: any[]) => global.core.api?.existsSync(...args)
const existsSync = (...args: any[]) => globalThis.core.api?.existsSync(...args)
/**
* List the directory files
* @returns {Promise<any>} A Promise that resolves with the contents of the directory.
*/
const readdirSync = (...args: any[]) => global.core.api?.readdirSync(...args)
const readdirSync = (...args: any[]) => globalThis.core.api?.readdirSync(...args)
/**
* Creates a directory at the specified path.
* @returns {Promise<any>} A Promise that resolves when the directory is created successfully.
*/
const mkdirSync = (...args: any[]) => global.core.api?.mkdirSync(...args)
const mkdirSync = (...args: any[]) => globalThis.core.api?.mkdirSync(...args)
const mkdir = (...args: any[]) => globalThis.core.api?.mkdir(...args)
/**
* Removes a directory at the specified path.
* @returns {Promise<any>} A Promise that resolves when the directory is removed successfully.
*/
const rmdirSync = (...args: any[]) =>
global.core.api?.rmdirSync(...args, { recursive: true, force: true })
globalThis.core.api?.rmdirSync(...args, { recursive: true, force: true })
const rm = (path: string) => globalThis.core.api?.rm(path)
/**
* Deletes a file from the local file system.
* @param {string} path - The path of the file to delete.
* @returns {Promise<any>} A Promise that resolves when the file is deleted.
*/
const unlinkSync = (...args: any[]) => global.core.api?.unlinkSync(...args)
const unlinkSync = (...args: any[]) => globalThis.core.api?.unlinkSync(...args)
/**
* Appends data to a file at the specified path.
*/
const appendFileSync = (...args: any[]) => global.core.api?.appendFileSync(...args)
const appendFileSync = (...args: any[]) => globalThis.core.api?.appendFileSync(...args)
/**
* Synchronizes a file from a source path to a destination path.
@ -62,15 +67,15 @@ const appendFileSync = (...args: any[]) => global.core.api?.appendFileSync(...ar
* @returns {Promise<any>} - A promise that resolves when the file has been successfully synchronized.
*/
const syncFile: (src: string, dest: string) => Promise<any> = (src, dest) =>
global.core.api?.syncFile(src, dest)
globalThis.core.api?.syncFile(src, dest)
/**
* Copy file sync.
*/
const copyFileSync = (...args: any[]) => global.core.api?.copyFileSync(...args)
const copyFileSync = (...args: any[]) => globalThis.core.api?.copyFileSync(...args)
const copyFile: (src: string, dest: string) => Promise<void> = (src, dest) =>
global.core.api?.copyFile(src, dest)
globalThis.core.api?.copyFile(src, dest)
/**
* Gets the file's stats.
@ -82,7 +87,7 @@ const copyFile: (src: string, dest: string) => Promise<void> = (src, dest) =>
const fileStat: (path: string, outsideJanDataFolder?: boolean) => Promise<FileStat | undefined> = (
path,
outsideJanDataFolder
) => global.core.api?.fileStat(path, outsideJanDataFolder)
) => globalThis.core.api?.fileStat(path, outsideJanDataFolder)
// TODO: Export `dummy` fs functions automatically
// Currently adding these manually
@ -92,7 +97,9 @@ export const fs = {
existsSync,
readdirSync,
mkdirSync,
mkdir,
rmdirSync,
rm,
unlinkSync,
appendFileSync,
copyFileSync,

24
core/src/node/api/processors/fsExt.ts
View File

@ -88,4 +88,28 @@ export class FSExt implements Processor {
})
})
}
mkdir(path: string): Promise<void> {
return new Promise((resolve, reject) => {
fs.mkdir(path, { recursive: true }, (err) => {
if (err) {
reject(err)
} else {
resolve()
}
})
})
}
rm(path: string): Promise<void> {
return new Promise((resolve, reject) => {
fs.rm(path, { recursive: true }, (err) => {
if (err) {
reject(err)
} else {
resolve()
}
})
})
}
}

2
core/src/node/api/restful/common.ts
View File

@ -40,7 +40,7 @@ export const commonRouter = async (app: HttpServer) => {
})
// Threads
app.post(`/threads/`, async (req, res) => createThread(req.body))
app.post(`/threads`, async (req, res) => createThread(req.body))
app.get(`/threads/:threadId/messages`, async (req, res) =>
getMessages(req.params.threadId).then(normalizeData)

2
core/src/node/api/restful/helper/builder.ts
View File

@ -216,7 +216,7 @@ export const createMessage = async (threadId: string, message: any) => {
const threadMessagesFileName = 'messages.jsonl'
try {
const { ulid } = require('ulid')
const { ulid } = require('ulidx')
const msgId = ulid()
const createdAt = Date.now()
const threadMessage: ThreadMessage = {

13
core/src/node/extension/store.ts
View File

@ -93,8 +93,7 @@ export function persistExtensions() {
*/
export async function installExtensions(extensions: any) {
const installed: Extension[] = []
for (const ext of extensions) {
// Set install options and activation based on input type
const installations = extensions.map((ext: any): Promise<void> => {
const isObject = typeof ext === 'object'
const spec = isObject ? [ext.specifier, ext] : [ext]
const activate = isObject ? ext.activate !== false : true
@ -102,15 +101,17 @@ export async function installExtensions(extensions: any) {
// Install and possibly activate extension
const extension = new Extension(...spec)
if (!extension.origin) {
continue
return Promise.resolve()
}
await extension._install()
return extension._install().then(() => {
if (activate) extension.setActive(true)
// Add extension to store if needed
addExtension(extension)
installed.push(extension)
}
})
})
await Promise.all(installations)
// Return list of all installed extensions
return installed

10
core/src/node/helper/config.ts
View File

@ -82,6 +82,7 @@ export const getJanExtensionsPath = (): string => {
*/
export const physicalCpuCount = async (): Promise<number> => {
const platform = os.platform()
try {
if (platform === 'linux') {
const output = await exec('lscpu -p | egrep -v "^#" | sort -u -t, -k 2,4 | wc -l')
return parseInt(output.trim(), 10)
@ -103,6 +104,13 @@ export const physicalCpuCount = async (): Promise<number> => {
})
return cores.length
}
} catch (err) {
console.warn('Failed to get physical CPU count', err)
// Divide by 2 to get rid of hyper threading
const coreCount = Math.ceil(os.cpus().length / 2)
console.debug('Using node API to get physical CPU count:', coreCount)
return coreCount
}
}
const exec = async (command: string): Promise<string> => {
@ -118,7 +126,7 @@ const exec = async (command: string): Promise<string> => {
}
export const getEngineConfiguration = async (engineId: string) => {
if (engineId !== 'openai') {
if (engineId !== 'openai' && engineId !== 'groq') {
return undefined
}
const directoryPath = join(getJanDataFolderPath(), 'engines')

2
core/src/node/helper/resource.ts
View File

@ -1,6 +1,6 @@
import { SystemResourceInfo } from '../../types'
import { physicalCpuCount } from './config'
import { log, logServer } from './log'
import { log } from './log'
export const getSystemResourceInfo = async (): Promise<SystemResourceInfo> => {
const cpu = await physicalCpuCount()

2
core/src/node/index.ts
View File

@ -4,3 +4,5 @@ export * from './extension/manager'
export * from './extension/store'
export * from './api'
export * from './helper'
export * from './../types'
export * from './../api'

24
core/src/types/miscellaneous/systemResourceInfo.ts
View File

@ -30,3 +30,27 @@ export type GpuSettingInfo = {
name: string
arch?: string
}
export type SystemInformation = {
gpuSetting: GpuSetting
osInfo?: OperatingSystemInfo
}
export const SupportedPlatforms = ['win32', 'linux', 'darwin'] as const
export type SupportedPlatformTuple = typeof SupportedPlatforms
export type SupportedPlatform = SupportedPlatformTuple[number]
export type OperatingSystemInfo = {
platform: SupportedPlatform | 'unknown'
arch: string
release: string
machine: string
version: string
totalMem: number
freeMem: number
}
export type CpuCoreInfo = {
model: string
speed: number
}

1
core/src/types/model/modelEntity.ts
View File

@ -18,6 +18,7 @@ export type ModelInfo = {
export enum InferenceEngine {
nitro = 'nitro',
openai = 'openai',
groq = 'groq',
triton_trtllm = 'triton_trtllm',
nitro_tensorrt_llm = 'nitro-tensorrt-llm',

23
docs/blog/01-january-10-2024-bitdefender-false-positive-flag.mdx → docs/blog/2024-01-10-bitdefender-false-positive-flag.mdx
View File

@ -1,10 +1,27 @@
---
title: "Post Mortem: Bitdefender False Positive Flag"
title: 'Post Mortem: Bitdefender False Positive Flag'
description: "10th January 2024, Jan's 0.4.4 Release on Windows triggered Bitdefender to incorrectly flag it as infected with Gen:Variant.Tedy.258323, leading to automatic quarantine warnings on users' computers."
slug: /postmortems/january-10-2024-bitdefender-false-positive-flag
tags: [Postmortem]
authors: [hientominh, Van-QA]
date: 2024-01-10
keywords:
[
postmortem,
bitdefender,
false positive,
antivirus,
jan,
nitro,
incident,
incident response,
supply chain security,
user communication,
documentation,
antivirus compatibility,
cross-platform testing,
proactive incident response,
user education,
lessons learned,
]
---
Following the recent incident related to Jan version 0.4.4 triggering Bitdefender on Windows with Gen:Variant.Tedy.258323 on January 10, 2024, we wanted to provide a comprehensive postmortem and outline the necessary follow-up actions.

145
docs/blog/2024-03-19-TensorRT-LLM.md Normal file
View File

@ -0,0 +1,145 @@
---
title: Benchmarking TensorRT-LLM vs. llama.cpp
description: Jan has added support for the TensorRT-LLM Inference Engine, as an alternative to llama.cpp. We provide a performance benchmark that shows the head-to-head comparison of the two Inference Engine and model formats, with TensorRT-LLM providing better performance but consumes significantly more VRAM and RAM.
tags: [Nvidia, TensorRT-LLM, llama.cpp, 3090, 4090, "inference engine"]
unlisted: true
---
Jan has added support [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) as an alternative to the default [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. TensorRT-LLM allows Nvidia GPU owners to run blazing fast LLM inference as a hardware-optimized LLM inference engine that compiles models to [run extremely fast on Nvidia GPUs](https://blogs.nvidia.com/blog/tensorrt-llm-windows-stable-diffusion-rtx/).
You can follow our [TensorRT-LLM Guide](/guides/providers/tensorrt-llm) to try it out today. We've also added a few TensorRT-LLM models to Jan's Model Hub for download:
- Mistral 7b
- TinyLlama-1.1b
- TinyJensen-1.1b 😂
:::tip
TensorRT-LLM support is available in [v0.4.9](https://github.com/janhq/jan/releases/tag/v0.4.9), but should be considered an experimental feature.
Please report bugs on [Github](https://github.com/janhq/jan) or on our Discord's [#tensorrt-llm](https://discord.com/channels/1107178041848909847/1201832734704795688) channel.
:::
## Performance Benchmarks
We were really curious to see how TensorRT-LLM would perform vs. llama.cpp on consumer-grade GPUs. TensorRT-LLM has previously been shown by Nvidia to reach performance of up to [10,000 tokens/s](https://nvidia.github.io/TensorRT-LLM/blogs/H100vsA100.html) on datacenter-grade GPUs. As most of Jan's users are proud card carrying members of the [GPU Poor](https://www.semianalysis.com/p/google-gemini-eats-the-world-gemini#the-gpu-poor), we wanted to see how the two inference engine performed on the same hardware.
:::info
An interesting aside: Jan actually started out in June 2023 building on [FastTransformer](https://github.com/NVIDIA/FasterTransformer), the precursor library to TensorRT-LLM. TensorRT-LLM was released in September 2023, making it a very young library. We're excited to see it's roadmap develop!
:::
### Test Setup
We picked 3 hardware platforms to run the test on, based on Jan's userbase's self-reported common hardware platforms.
| NVIDIA GPU | VRAM Used (GB) | CUDA Cores | Tensor Cores | Memory Bus Width (bit) | Memory Bandwidth (GB/s) |
| ------------------------- | -------------- | ---------- | ------------ | ---------------------- | ----------------------- |
| RTX 4090 Desktop (Ada) | 24 | 16,384 | 512 | 384 | ~1000 |
| RTX 3090 Desktop (Ampere) | 24 | 10,496 | 328 | 384 | 935.8 |
| RTX 4060 Laptop (Ada) | 8 | 3,072 | 96 | 128 | 272 |
:::warning[Low-spec Machines?]
We didn't bother including low-spec machines: TensorRT-LLM is meant for performance, and simply doesn't work on lower grade Nvidia GPUs, or computers without GPUs.
TensorRT-LLM provides blazing fast performance at the cost of [memory usage](https://nvidia.github.io/TensorRT-LLM/memory.html). This means that the performance improvements only show up in higher-range GPUs with larger VRAMs.
We've found that [llama.cpp](https://github.com/ggerganov/llama.cpp) does an incredible job of democratizing inference to the [GPU Poor](https://www.semianalysis.com/p/google-gemini-eats-the-world-gemini#the-gpu-poor) with CPU-only or lower-range GPUs. Huge shout outs to the [llama.cpp maintainers](https://github.com/ggerganov/llama.cpp/graphs/contributors) and the [ggml.ai](https://ggml.ai/) team.
:::
We chose the popular Mistral 7b model to run on both GGUF and TensorRT-LLM, picking comparable quantizations.
#### llama.cpp Setup
- For llama.cpp, we used `Mistral-7b-q4_k_m`
- [ ] Fill in `ngl` params, GPU offload etc
#### TensorRT-LLM Setup
- For TensorRT-LLM, we used `Mistral-7b-int4 AWQ`
- We ran TensorRT-LLM with `free_gpu_memory_fraction` to test it with the lowest VRAM consumption (performance may be affected)
- Note: We picked AWQ for TensorRT-LLM as a handicap as AWQ supposedly sacrifices performance for quality
#### Experiment Setup
We ran the experiment using a standardized inference request in a sandboxed environment on the same machine:
- We ran tests 5 times for each inference engine, on a baremetal PC with no other applications open
- Each inference request was of `batch_size` 1 and `input_len` 2048, `output_len` 512 as a realistic test case
- CPU and Memory usage were obtained from.... Windows Task Manager 😱
- GPU usage was obtained from `nvtop`, `htop`, and `nvidia-smi`
## Results
Our biggest takeaway: TensorRT-LLM is faster than llama.cpp on 4090s and 3090s with larger VRAMs. However, on smaller GPUs (e.g. Laptop 4060 GPUs),
| | 4090 Desktop | 3090 Desktop | 4060 Laptop |
| ------------ | ------------ | ------------ | ----------- |
| TensorRT-LLM | ✅ 159t/s | ✅ 140.27t/s | ❌ 19t/s |
| llama.cpp | 101.3t/s | 90t/s | 22t/s |
### RTX-4090 Desktop
:::info[Hardware Details]
- CPU: Intel 13th series
- GPU: NVIDIA GPU 4090 (Ampere - sm 86)
- RAM: 32GB
- OS: Windows 11 Pro on Proxmox
:::
Nvidia's RTX-4090 is their top-of-the-line consumer GPU, and retails for [approximately $2,000](https://www.amazon.com/rtx-4090/s?k=rtx+4090).
#### Mistral-7b int4
| Metrics | GGUF (using GPU) | TensorRT-LLM | Difference |
| -------------------- | -------------------- | ------------ | -------------- |
| Throughput (token/s) | 101.3 | 159 | ✅ 57% faster |
| VRAM Used (GB) | 5.5 | 6.3 | 🤔 14% more |
| RAM Used (GB) | 0.54 | 0.42 | 🤯 20% less |
| Disk Size (GB) | 4.07 | 3.66 | 🤯 10% smaller |
### RTX-3090 Desktop
:::info[Hardware Details]
- CPU: Intel 13th series
- GPU: NVIDIA GPU 3090 (Ampere - sm 86)
- RAM: 64GB
- OS: Windows
:::
#### Mistral-7b int4
| Metrics | GGUF (using GPU) | TensorRT-LLM | Difference |
| -------------------- | -------------------- | ------------ | ------------ |
| Throughput (token/s) | 90 | ✅ 140.27 | ✅ 55% faster |
| VRAM Used (GB) | 6.0 | 6.8 | 🤔 13% more |
| RAM Used (GB) | 0.54 | 0.42 | 🤯 22% less |
| Disk Size (GB) | 4.07 | 3.66 | 🤯 10% less |
### RTX-4060 Laptop
- [ ] Dan to re-run perf tests and fill in details
:::info[Hardware Details]
- Manufacturer: Acer Nitro 16 Phenix
- CPU: Ryzen 7000
- RAM: 16GB
- GPU: NVIDIA Laptop GPU 4060 (Ada)
:::
#### Mistral-7b int4
| Metrics | GGUF (using the GPU) | TensorRT-LLM | Difference |
| -------------------- | -------------------- | ------------ | ---------- |
| Throughput (token/s) | 22 | ❌ 19 | |
| VRAM Used (GB) | 2.1 | 7.7 | |
| RAM Used (GB) | 0.3 | 13.5 | |
| Disk Size (GB) | 4.07 | 4.07 | |

9
docs/docs/about/about.md
View File

@ -4,14 +4,15 @@ slug: /about
description: Jan is a desktop application that turns computers into thinking machines.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
about Jan,
desktop application,
thinking machine,

BIN
docs/docs/about/assets/solar-punk.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 198 KiB

BIN
docs/docs/about/assets/vision-1.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 351 KiB

4
docs/docs/about/faq.md
View File

@ -1,3 +1,7 @@
---
title: Frequently Asked Questions (FAQ) - Jan
---
# Frequently Asked Questions (FAQ)
## What is Jan?

74
docs/docs/about/vision.md Normal file
View File

@ -0,0 +1,74 @@
---
title: Jan's Vision
slug: /vision
description: Jan is a desktop application that turns computers into thinking machines.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
about Jan,
desktop application,
thinking machine,
jan vision,
]
---
## Jan's vision is to shape a future where humans and machines collaborate, continuing our legacy as toolmakers
Throughout history, humanity has thrived by mastering tools, from [controlling fire](https://en.wikipedia.org/wiki/Control_of_fire_by_early_humans) to [inventing the wheel](https://en.wikipedia.org/wiki/Wheel). These leaps weren't just about survival, they were foundational to our civilization.
Today, we stand on the brink of a new frontier with artificial intelligence. AI is not merely another tool, it represents a new form of collaboration between humans and machines - promising to enhance our creativity, augment our lives, and deepen our understanding of the world.
![jan ai shapes the future](./assets/vision-1.webp)
In the future we envision, AI will be as integral to our lives as fire and the wheel once were, with each individual having their own machines/robots. Mastering AI, like mastering fire, will require understanding its potential, respecting its power, and learning to control it for the betterment of humanity.
### Inspired by Science Fiction, Grounded in Optimism
Our vision is influenced by the harmonious coexistence of humans and machines in science fiction. From the helpful companionship of [C3PO](https://tr.wikipedia.org/wiki/C-3PO) and [Jarvis](https://en.wikipedia.org/wiki/J.A.R.V.I.S.) to the strategic alliances in [Halo](https://www.imdb.com/title/tt2934286/), these stories showcase a future where technology amplifies human potential.
### Jan's Role in Shaping the Future
Jan is our contribution to this future - a tool designed to augment human capabilities, not replace them. We are committed to developing AI that works for humanity, enhancing our creativity, productivity, and well-being. With Jan, we aim to empower individuals and communities to achieve more, together.
Our vision is not just a dream, it's a blueprint for a future where technology and humanity harmonize to unlock unprecedented possibilities.
## How we imagine the world in the future
We are fundamentally optimistic about the future. Jan aligns with the [Solarpunk movement](https://en.wikipedia.org/wiki/Solarpunk), which envisions a world where technology and nature coexist and flourish together. We reject the notion of climate doomerism and instead, focus on the positive impact we can make with AI.
![solarpunk and jan](./assets/solar-punk.webp)
Imagine a world where every individual is empowered by their own robots, where machines are not just tools but partners in our journey. This is the future Jan is striving to create.
Now, let's take a glimpse into this future through a day in the life of Emre, a reflection of how Jan's vision manifests in everyday life.
## A Day in the Life of Emre in 2050
> In 2050, Emre wakes up to the gentle sound of birds chirping, a soothing alarm created by **his own AI robot, Jan**. As he gets ready for the day, **Jan has already prepared** his schedule, factoring in his preferences and the day's weather.
>
> At breakfast, Emre discusses his upcoming project with **Jan, who offers insights and suggestions**, enhancing Emre's creativity. As he heads to work, his self-driving car, **integrated with Jan**, takes the most scenic and efficient route, allowing Emre to enjoy a moment of tranquility.
>
> In the office, Emre collaborates with colleagues from around the globe in a virtual workspace. **Jan assists** by translating languages in real-time and organizing ideas, making collaboration seamless and productive.
>
> During lunch, Emre decides to explore a new hobby. **Jan quickly curates** a list of resources and connects Emre with a virtual mentor, making learning accessible and enjoyable.
>
> In the afternoon, Emre takes a break to connect with nature. His smart garden, **managed by Jan**, is thriving, blending technology with the natural world in perfect harmony.
>
> As the day winds down, Emre reflects on his accomplishments. **With Jan's help**, he's been able to focus on what truly matters, achieving a balance between work, personal growth, and well-being.
>
> In 2050, Jan is more than just a tool, it's an integral part of Emre's life, **augmenting his abilities** and enabling him to live a more fulfilling life.
What a day, hah!
---
Jan's vision commits to developing thinking machines that work alongside humans - learning, adapting, and contributing to a broader, smarter society. This journey isn't just about technology. It's about creating a future where humans and machines collaborate.
Let's build the future together - join the journey!

10
docs/docs/acknowledgements.md
View File

@ -4,14 +4,15 @@ description: Jan is a ChatGPT-alternative that runs on your own computer, with a
slug: /acknowledgements
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
acknowledgements,
third-party libraries,
]
@ -24,3 +25,4 @@ We would like to express our gratitude to the following third-party libraries th
- [llama.cpp](https://github.com/ggerganov/llama.cpp/blob/master/LICENSE)
- [LangChain.js](https://github.com/langchain-ai/langchainjs/blob/main/LICENSE)
- [TensorRT](https://github.com/NVIDIA/TensorRT/blob/main/LICENSE)
- [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/blob/main/LICENSE)

14
docs/docs/community/community.mdx
View File

@ -4,14 +4,20 @@ slug: /community
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
about Jan,
desktop application,
thinking machine,
community,
socials,
]
---

10
docs/docs/developer/01-overview/01-architecture.md
View File

@ -4,14 +4,16 @@ slug: /developer/architecture
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
architecture,
]
---

10
docs/docs/developer/01-overview/02-file-based.md
View File

@ -4,14 +4,16 @@ slug: /developer/file-based
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
file based approach,
]
---

10
docs/docs/developer/01-overview/03-user-interface.md
View File

@ -4,14 +4,16 @@ slug: /developer/ui
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
UI kit,
]
---

9
docs/docs/developer/01-overview/04-install-and-prerequisites.md
View File

@ -4,14 +4,15 @@ slug: /developer/prereq
description: Guide to install and setup Jan for development.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
installation,
prerequisites,
developer setup,

9
docs/docs/developer/01-overview/README.md
View File

@ -4,14 +4,15 @@ slug: /developer
description: Jan Docs | Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

12
docs/docs/developer/02-build-assistant/01-your-first-assistant.md
View File

@ -1,17 +1,18 @@
---
title: Your First Assistant
slug: /developer/build-assistant/your-first-assistant/
slug: /developer/assistant/your-first-assistant/
description: A quick start on how to build an assistant.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
quick start,
build assistant,
]
@ -20,4 +21,3 @@ keywords:
:::caution
This is currently under development.
:::

11
docs/docs/developer/02-build-assistant/02-assistant-anatomy.md
View File

@ -1,17 +1,18 @@
---
title: Anatomy of an Assistant
slug: /developer/build-assistant/assistant-anatomy/
slug: /developer/assistant/assistant-anatomy/
description: An overview of assistant.json
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
build assistant,
assistant anatomy,
]

11
docs/docs/developer/02-build-assistant/03-package-your-assistant.md
View File

@ -1,17 +1,18 @@
---
title: Package your Assistant
slug: /developer/build-assistant/package-your-assistant/
slug: /developer/assistant/package-your-assistant/
description: Package your assistant for sharing and publishing.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
quick start,
build assistant,
]

11
docs/docs/developer/02-build-assistant/README.mdx
View File

@ -1,17 +1,10 @@
---
title: Build an Assistant
slug: /developer/build-assistant
slug: /developer/assistant
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
Jan, Rethink the Computer, local AI, privacy focus, free and open source, private and offline, conversational AI, no-subscription fee, large language models,
build assistant,
]
---

11
docs/docs/developer/03-build-engine/01-your-first-engine.md
View File

@ -1,17 +1,18 @@
---
title: Your First Engine
slug: /developer/build-engine/your-first-engine/
slug: /developer/engine/your-first-engine/
description: A quick start on how to build your first engine
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
quick start,
build engine,
]

11
docs/docs/developer/03-build-engine/02-engine-anatomy.md
View File

@ -1,17 +1,18 @@
---
title: Anatomy of an Engine
slug: /developer/build-engine/engine-anatomy
slug: /developer/engine/engine-anatomy
description: An overview of engine.json
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
build engine,
engine anatomy,
]

11
docs/docs/developer/03-build-engine/03-package-your-engine.md
View File

@ -1,17 +1,18 @@
---
title: Package your Engine
slug: /developer/build-engine/package-your-engine/
slug: /developer/engine/package-your-engine/
description: Package your engine for sharing and publishing.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
build engine,
engine anatomy,
]

11
docs/docs/developer/03-build-engine/README.mdx
View File

@ -1,17 +1,18 @@
---
title: Build an Inference Engine
slug: /developer/build-engine/
slug: /developer/engine/
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
build assistant,
]
---

15
docs/docs/developer/04-build-extension/01-your-first-extension.md
View File

@ -1,17 +1,18 @@
---
title: Your First Extension
slug: /developer/build-extension/your-first-extension/
slug: /developer/extension/your-first-extension/
description: A quick start on how to build your first extension
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
quick start,
build extension,
]
@ -76,10 +77,10 @@ There are a few things to keep in mind when writing your extension code:
In `index.ts`, you will see that the extension function will return a `Promise<any>`.
```typescript
import { core } from "@janhq/core";
import { core } from '@janhq/core'
function onStart(): Promise<any> {
return core.invokePluginFunc(MODULE_PATH, "run", 0);
return core.invokePluginFunc(MODULE_PATH, 'run', 0)
}
```

11
docs/docs/developer/04-build-extension/02-extension-anatomy.md
View File

@ -1,17 +1,18 @@
---
title: Anatomy of an Extension
slug: /developer/build-extension/extension-anatomy
slug: /developer/extension/extension-anatomy
description: An overview of extensions.json
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
build extension,
extension anatomy,
]

11
docs/docs/developer/04-build-extension/03-package-your-extension.md
View File

@ -1,17 +1,18 @@
---
title: Package your Engine
slug: /developer/build-extension/package-your-extension/
slug: /developer/extension/package-your-extension/
description: Package your extension for sharing and publishing.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
build extension,
extension anatomy,
]

11
docs/docs/developer/04-build-extension/README.mdx
View File

@ -1,17 +1,10 @@
---
title: Build an Extension
slug: /developer/build-extension/
slug: /developer/extension/
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
Jan, Rethink the Computer, local AI, privacy focus, free and open source, private and offline, conversational AI, no-subscription fee, large language models,
build extension,
]
---

9
docs/docs/developer/05-framework/03-engineering/README.mdx
View File

@ -4,14 +4,15 @@ slug: /developer/engineering
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
spec,
engineering,
]

11
docs/docs/developer/05-framework/03-engineering/assistants.md
View File

@ -1,16 +1,17 @@
---
title: "Assistants"
title: 'Assistants'
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-engineering/chats.md
View File

@ -3,14 +3,15 @@ title: Chats
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

11
docs/docs/developer/05-framework/03-engineering/files.md
View File

@ -1,16 +1,17 @@
---
title: "Files"
title: 'Files'
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

11
docs/docs/developer/05-framework/03-engineering/fine-tuning.md
View File

@ -1,16 +1,17 @@
---
title: "Fine-tuning"
title: 'Fine-tuning'
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-engineering/messages.md
View File

@ -3,14 +3,15 @@ title: Messages
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-engineering/models.md
View File

@ -3,14 +3,15 @@ title: Models
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-engineering/prompts.md
View File

@ -3,14 +3,15 @@ title: Prompts
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-engineering/threads.md
View File

@ -3,14 +3,15 @@ title: Threads
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-product/README.mdx
View File

@ -4,14 +4,15 @@ slug: /developer/product
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
spec,
product,
]

9
docs/docs/developer/05-framework/03-product/chat.md
View File

@ -3,14 +3,15 @@ title: Chat
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-product/hub.md
View File

@ -3,14 +3,15 @@ title: Hub
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-product/jan.md
View File

@ -3,14 +3,15 @@ title: Jan (The Default Assistant)
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-product/settings.md
View File

@ -3,14 +3,15 @@ title: Settings
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/03-product/system-monitor.md
View File

@ -3,14 +3,15 @@ title: System Monitor
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

9
docs/docs/developer/05-framework/README.md
View File

@ -4,14 +4,15 @@ slug: /developer/framework/
description: Jan Docs | Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
large language models,
]
---

3
docs/docs/events/hcmc-oct23.md
View File

@ -1,8 +1,9 @@
---
title: "Jan's AI Hacker House (Ho Chi Minh City)"
description: "24-27 Oct 2023, District 3, HCMC. AI-focused talks, workshops and social events. Hosted by Jan.ai"
description: '24-27 Oct 2023, District 3, HCMC. AI-focused talks, workshops and social events. Hosted by Jan.ai'
slug: /events/hcmc-oct23
image: /img/hcmc-launch-party.png
keywords: [AI, Hacker House, Ho Chi Minh City, HCMC, Jan.ai]
---
![](/img/hcmc-launch-party.png)

3
docs/docs/events/nvidia-llm-day-nov-23.md
View File

@ -1,5 +1,5 @@
---
title: "Nov 23: Nvidia GenAI Day"
title: 'Nov 23: Nvidia GenAI Day'
description: Nvidia's LLM Day
---
@ -18,4 +18,3 @@ Jan will be at Nvidia's GenAI Innovation Day in Nov '23, focusing on Enterprise
### Programme
![](/img/nvidia-llm-day.png)

8
docs/docs/guides/advanced-settings/_category_.json
View File

@ -1,8 +0,0 @@
{
"label": "Advanced Settings",
"position": 11,
"link": {
"type": "doc",
"id": "guides/advanced-settings/advanced-settings"
}
}

150
docs/docs/guides/advanced-settings/advanced-settings.mdx
View File

@ -1,150 +0,0 @@
---
title: Advanced Settings
sidebar_position: 1
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
advanced-settings,
]
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This guide will show you how to use the advanced settings in Jan.
## Access the Advanced Settings
To access the Jan's advanced settings, follow the steps below:
1. Navigate to the main dashboard.
2. Click the **gear icon (⚙️)** on the bottom left of your screen.
3. Under the **Settings screen**, click the **Advanced Settings**.
4. You can configure the following settings:
| Feature | Description |
|---------------------------|-----------------------------------------------------------------------------------------------------------------------|
| **Keyboard Shortcuts** | Keyboard shortcuts speed up your workflow. For a quick overview of useful keyboard shortcuts, refer to the list [below](advanced-settings.mdx#keyboard-shortcuts). |
| **Experimental Mode** | Enables experimental features that may be unstable. |
| **GPU Acceleration** | Enables the boosting of your model performance by using your GPU devices for acceleration. |
| **Jan Data Folder** | Location for messages, model configurations, and user data. Changeable to a different location. |
| **HTTPS Proxy & Ignore SSL Certificate** | Use a proxy server for internet connections and ignore SSL certificates for self-signed certificates. Please check out the guide on how to set up your own HTTPS proxy server [here](http-proxy.mdx). |
| **Clear Logs** | Removes all logs from the Jan application. |
| **Reset To Factory Default** | Resets the application to its original state, deleting all data including model customizations and conversation history. |
## Keyboard Shortcuts
Here are some of the keyboard shortcuts that you can use in Jan.
<Tabs>
<TabItem value="mac" label = "Mac" default>
| Combination | Description |
| --------------- | -------------------------------------------------- |
| `⌘ E` | Show list your models |
| `⌘ K` | Show list navigation pages |
| `⌘ B` | Toggle collapsible left panel |
| `⌘ ,` | Navigate to setting page |
| `Enter` | Send a message |
| `Shift + Enter` | Insert new line in input box |
| `Arrow Up` | Navigate to the previous option (within the search dialog) |
| `Arrow Down` | Navigate to the next option (within the search dialog) |
</TabItem>
<TabItem value="windows" label = "Windows" default>
| Combination | Description |
| --------------- | -------------------------------------------------- |
| `Ctrl E` | Show list your models |
| `Ctrl K` | Show list navigation pages |
| `Ctrl B` | Toggle collapsible left panel |
| `Ctrl ,` | Navigate to setting page |
| `Enter` | Send a message |
| `Shift + Enter` | Insert new line in input box |
| `Arrow Up` | Navigate to the previous option (within the search dialog) |
| `Arrow Down` | Navigate to the next option (within the search dialog) |
</TabItem>
<TabItem value="linux" label = "Linux" default>
| Combination | Description |
| --------------- | -------------------------------------------------- |
| `Ctrl E` | Show list your models |
| `Ctrl K` | Show list navigation pages |
| `Ctrl B` | Toggle collapsible left panel |
| `Ctrl ,` | Navigate to setting page |
| `Enter` | Send a message |
| `Shift + Enter` | Insert new line in input box |
| `Arrow Up` | Navigate to the previous option (within the search dialog) |
| `Arrow Down` | Navigate to the next option (within the search dialog) |
</TabItem>
</Tabs>
:::note
The keyboard shortcuts are customizable.
:::
## Enable the Experimental Mode
To try out new fetures that are still in testing phase, follow the steps below:
1. Navigate to the main dashboard.
2. Click the **gear icon (⚙️)** on the bottom left of your screen.
3. Under the **Settings screen**, click the **Advanced Settings**.
4. On the **Experimental Mode** click the slider to enable.
## Enable the GPU Acceleration
To enhance your model performance, follow the steps below:
:::warning
Ensure that you have read the [troubleshooting guide](/docs/guides/common-error/not-using-gpu.mdx) here for further assistance.
:::
1. Navigate to the main dashboard.
2. Click the **gear icon (⚙️)** on the bottom left of your screen.
3. Under the **Settings screen**, click the **Advanced Settings**.
4. On the **GPU Acceleration** click the slider to enable.
## Access the Jan Data Folder
To access the folder where messages, model configurations and user data are stored, follow the steps below:
1. Navigate to the main dashboard.
2. Click the **gear icon (⚙️)** on the bottom left of your screen.
3. Under the **Settings screen**, click the **Advanced Settings**.
4. On the **Jan Data Folder** click the **folder icon (📂)** to access the data or the **pencil icon (✏️)** to change the folder where you keep your data.
## Enable the HTTPS Proxy
To enable the HTTPS Proxy feature, follow the steps below:
1. Make sure to set up your HTTPS Proxy. Check out this [guide](http-proxy.mdx) for instructions on how to do it.
2. Navigate to the main dashboard.
3. Click the **gear icon (⚙️)** on the bottom left of your screen.
4. Under the **Settings screen**, click the **Advanced Settings**.
5. On the **HTTPS Proxy** click the slider to enable.
6. Input your domain in the blank field.
## Ignore SSL Certificate
To Allow self-signed or unverified certificates, follow the steps below:
1. Navigate to the main dashboard.
2. Click the **gear icon (⚙️)** on the bottom left of your screen.
3. Under the **Settings screen**, click the **Advanced Settings**.
4. On the **Ignore SSL Certificates** click the slider to enable.
## Clear Logs
To clear all logs on your Jan app, follow the steps below:
:::warning
This feature clears all the data in your **Jan Data Folder**.
:::
1. Navigate to the main dashboard.
2. Click the **gear icon (⚙️)** on the bottom left of your screen.
3. Under the **Settings screen**, click the **Advanced Settings**.
4. On the **Clear Logs** click the the **Clear** button.
## Reset To Factory Default
To reset the Jan app to its original state, follow the steps below:
:::danger[Remember!]
This irreversible action is only recommended if the application is corrupted.
:::
1. Navigate to the main dashboard.
2. Click the **gear icon (⚙️)** on the bottom left of your screen.
3. Under the **Settings screen**, click the **Advanced Settings**.
4. On the **Reset To Factory Default** click the the **Reset** button.

118
docs/docs/guides/advanced-settings/http-proxy.mdx
View File

@ -1,118 +0,0 @@
---
title: HTTPS Proxy
sidebar_position: 2
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
advanced-settings,
https-proxy,
]
---
## Why HTTPS Proxy?
HTTPS Proxy encrypts data between your browser and the internet, making it hard for outsiders to intercept or read. It also helps you to maintain your privacy and security while being able to bypass regional restrictions on internet.
:::note
- When configuring Jan using an HTTPS proxy, the speed of the downloading model may be affected due to the encryption and decryption process. It also depends on the networking of the cloud service provider.
- HTTPS Proxy does not affect the remote model usage.
:::
## Setting Up Your Own HTTPS Proxy Server
This guide provides a simple overview of setting up an HTTPS proxy server using **Squid**, a widely used open-source proxy software.
:::note
Other software options are also available depending on your requirements.
:::
### Step 1: Choosing a Server
1. Firstly, you need to choose a server to host your proxy server.
:::note
We recommend using a well-known cloud provider service like:
- Amazon AWS
- Google Cloud
- Microsoft Azure
- Digital Ocean
:::
2. Ensure that your server has a public IP address and is accessible from the internet.
### Step 2: Installing Squid
Instal **Squid** using the following command:
```bash
sudo apt-get update
sudo apt-get install squid
```
### Step 3: Configure Squid for HTTPS
To enable HTTPS, you will need to configure Squid with SSL support.
1. Squid requires an SSL certificate to be able to handle HTTPS traffic. You can generate a self-signed certificate or obtain one from a Certificate Authority (CA). For a self-signed certificate, you can use OpenSSL:
```bash
openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout squid-proxy.pem -out squid-proxy.pem
```
2. Edit the Squid configuration file `/etc/squid/squid.conf` to include the path to your SSL certificate and enable the HTTPS port:
```bash
http_port 3128 ssl-bump cert=/path/to/your/squid-proxy.pem
ssl_bump server-first all
ssl_bump bump all
```
3. To intercept HTTPS traffic, Squid uses a process called SSL Bumping. This process allows Squid to decrypt and re-encrypt HTTPS traffic. To enable SSL Bumping, ensure the `ssl_bump` directives are configured correctly in your `squid.conf` file.
### Step 4 (Optional): Configure ACLs and Authentication
1. You can define rules to control who can access your proxy. This is done by editing the squid.conf file and defining ACLs:
```bash
acl allowed_ips src "/etc/squid/allowed_ips.txt"
http_access allow allowed_ips
```
2. If you want to add an authentication layer, Squid supports several authentication schemes. Basic authentication setup might look like this:
```bash
auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/passwords
acl authenticated proxy_auth REQUIRED
http_access allow authenticated
```
### Step 5: Restart and Test Your Proxy
1. After configuring, restart Squid to apply the changes:
```bash
sudo systemctl restart squid
```
2. To test, configure your browser or another client to use the proxy server with its IP address and port (default is 3128).
3. Check if you can access the internet through your proxy.
:::tip
Tips for Secure Your Proxy:
- **Firewall rules**: Ensure that only intended users or IP addresses can connect to your proxy server. This can be achieved by setting up appropriate firewall rules.
- **Regular updates**: Keep your server and proxy software updated to ensure that you are protected against known vulnerabilities.
- **Monitoring and logging**: Monitor your proxy server for unusual activity and enable logging to keep track of the traffic passing through your proxy.
:::
## Setting Up Jan to Use Your HTTPS Proxy
Once you have your HTTPS proxy server set up, you can configure Jan to use it.
1. Navigate to `Settings` > `Advanced Settings` and specify the HTTPS proxy (proxy auto-configuration and SOCKS not supported).
2. You can turn on the feature `Ignore SSL Certificates` if you are using a self-signed certificate. This feature allows self-signed or unverified certificates.

BIN
docs/docs/guides/assets/jan-ai-download.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 95 KiB

BIN
docs/docs/guides/assets/jan-ai-quickstart.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 83 KiB

0
docs/docs/guides/common-error/assets/janOpenAppDirectory.gif → docs/docs/guides/assets/janOpenAppDirectory.gif
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 479 KiB

After

Width:  |  Height:  |  Size: 479 KiB

0
docs/docs/guides/common-error/assets/janSwitchCPUtoGPU.gif → docs/docs/guides/assets/janSwitchCPUtoGPU.gif
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 472 KiB

After

Width:  |  Height:  |  Size: 472 KiB

BIN
docs/docs/guides/assets/quick.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 107 KiB

48
docs/docs/guides/best-practices.mdx
View File

@ -1,48 +0,0 @@
---
title: Best Practices
sidebar_position: 3
description: Comprehensive set of best practices.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
acknowledgements,
third-party libraries,
]
---
Jan is a versatile platform offering solutions for integrating AI locally across various platforms. This guide outlines best practices for developers, analysts, and AI enthusiasts to enhance their experience with Jan when adding AI locally to their computers. Implementing these practices will optimize the performance of AI models.
## Follow the Quickstart Guide
The [quickstart guide](quickstart.mdx) is designed to facilitate a quick setup process. It provides a clear instruction and simple steps to get you up and running with Jan.ai quickly. Even, if you are inexperienced in AI, the quickstart can offer valuable insights and tips to help you get started quickly.
## Setting up the Right Models
Jan offers a range of pre-configured AI models that are tailored to different tasks and industries. You should identify which on that aligns with your objectives. There are factors to be considered:
- Capabilities
- Accuracy
- Processing Speed
:::note
- Some of these factors also depend on your hardware, please see Hardware Requirement.
- Choosing the right model is important to achieve the best performance.
:::
## Setting up Jan
Ensure that you familiarize yourself with the Jan application. Jan offers advanced settings that you can adjust. These settings may influence how your AI behaves locally. Please see the [Advanced Settings](./advanced-settings/advanced-settings.mdx) article for a complete list of Jan's configurations and instructions on how to configure them.
## Integrations
One of Jan's key features is its ability to integrate with many systems. Whether you are incorporating Jan.ai with any open-source LLM provider or other tools, it is important to understand the integration capabilities and limitations.
## Mastering the Prompt Engineering
Prompt engineering is an important aspect when dealing with AI models to generate the desired outputs. Mastering this skill can significantly enhance the performance and the responses of the AI. Below are some tips that you can do for prompt engineering:
- Ask the model to adopt a persona
- Be specific and details get a more specific answers
- Provide examples or preference text or context at the beginning
- Use a clear and concise language
- Use certain keywords and phrases

159
docs/docs/guides/common-error/broken-build.mdx
View File

@ -1,159 +0,0 @@
---
title: Broken Build
sidebar_position: 1
hide_table_of_contents: true
description: A step-by-step guide to fix errors that prevent the project from compiling or running successfully.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
]
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This guide provides you steps to troubleshoot and to resolve the issue where your Jan is stuck in a broken build after installation.
<Tabs>
<TabItem value="mac" label="Mac" default>
### 1. Uninstall Jan
Delete Jan from your `/Applications` folder.
### 2. Delete Application Data, Cache, and User Data
```zsh
# Step 1: Delete the application data
## Newer versions
rm -rf ~/Library/Application\ Support/jan
## Versions 0.2.0 and older
rm -rf ~/Library/Application\ Support/jan-electron
# Step 2: Clear application cache
rm -rf ~/Library/Caches/jan*
# Step 3: Remove all user data
rm -rf ~/jan
```
### 3. Additional Step for Versions Before 0.4.2
If you are using a version before `0.4.2`, you need to run the following commands:
```zsh
ps aux | grep nitro
# Looks for processes like `nitro` and `nitro_arm_64`, and kill them one by one by process ID
kill -9 <PID>
```
### 4. Download the Latest Version
Download the latest version of Jan from our [homepage](https://jan.ai/).
</TabItem>
<TabItem value="windows" label="Windows">
### 1. Uninstall Jan
To uninstall Jan on Windows, use the [Windows Control Panel](https://support.microsoft.com/en-us/windows/uninstall-or-remove-apps-and-programs-in-windows-4b55f974-2cc6-2d2b-d092-5905080eaf98).
### 2. Delete Application Data, Cache, and User Data
```sh
# You can delete the `/Jan` directory in Windows's AppData Directory by visiting the following path `%APPDATA%\Jan`
cd C:\Users\YOUR_USERNAME\AppData\Roaming
rm -r ./Jan
```
### 3. Additional Step for Versions Before 0.4.2
If you are using a version before `0.4.2`, you need to run the following commands:
```sh
# Find the process ID (PID) of the nitro process by filtering the list by process name
tasklist | findstr "nitro"
# Once you have the PID of the process you want to terminate, run the `taskkill`
taskkill /F /PID <PID>
```
### 4. Download the Latest Version
Download the latest version of Jan from our [homepage](https://jan.ai/).
</TabItem>
<TabItem value="linux" label="Linux">
### 1. Uninstall Jan
<Tabs groupId = "linux_type">
<TabItem value="linux_main" label = "Linux">
To uninstall Jan, you should use your package manager's uninstall or remove option.
This will return your system to its state before the installation of Jan.
This method can also reset all settings if you are experiencing any issues with Jan.
</TabItem>
<TabItem value = "deb_ub" label = "Debian / Ubuntu">
To uninstall Jan, run the following command.MDXContent
```sh
sudo apt-get remove jan
# where jan is the name of Jan package
```
This will return your system to its state before the installation of Jan.
This method can also be used to reset all settings if you are experiencing any issues with Jan.
</TabItem>
<TabItem value = "other" label = "Others">
To uninstall Jan, you can uninstall Jan by deleting the `.AppImage` file.
If you wish to completely remove all user data associated with Jan after uninstallation, you can delete the user data at `~/jan`.
This method can also reset all settings if you are experiencing any issues with Jan.
</TabItem>
</Tabs>
### 2. Delete Application Data, Cache, and User Data
```sh
# You can delete the user data folders located at the following `~/jan`
rm -rf ~/jan
```
### 3. Additional Step for Versions Before 0.4.2
If you are using a version before `0.4.2`, you need to run the following commands:
```zsh
ps aux | grep nitro
# Looks for processes like `nitro` and `nitro_arm_64`, and kill them one by one by process ID
kill -9 <PID>
```
### 4. Download the Latest Version
Download the latest version of Jan from our [homepage](https://jan.ai/).
</TabItem>
</Tabs>
By following these steps, you can cleanly uninstall and reinstall Jan, ensuring a smooth and error-free experience with the latest version.
:::note
Before reinstalling Jan, ensure it's completely removed from all shared spaces if it's installed on multiple user accounts on your device.
:::

161
docs/docs/guides/common-error/not-using-gpu.mdx
View File

@ -1,161 +0,0 @@
---
title: Troubleshooting NVIDIA GPU
sidebar_position: 2
description: A step-by-step guide to enable Jan to properly leverage NVIDIA GPU resources, avoiding performance issues.
keywords: [
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
convZ
ersational AI,
no-subscription fee,
large language model,
troubleshooting,
using GPU,
]
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
This guide provides steps to troubleshoot and resolve issues when the Jan app does not utilize the NVIDIA GPU on Windows and Linux systems.
### 1. Ensure GPU Mode Requirements
<Tabs>
<TabItem value="windows" label="Windows">
#### NVIDIA Driver
- Install an [NVIDIA Driver](https://www.nvidia.com/Download/index.aspx) supporting CUDA 11.7 or higher.
- Use the following command to verify the installation:
```sh
nvidia-smi
```
#### CUDA Toolkit
- Install a [CUDA toolkit](https://developer.nvidia.com/cuda-downloads) compatible with your NVIDIA driver.
- Use the following command to verify the installation:
```sh
nvcc --version
```
</TabItem>
<TabItem value="linux" label="Linux">
#### NVIDIA Driver
- Install an [NVIDIA Driver](https://www.nvidia.com/Download/index.aspx) supporting CUDA 11.7 or higher.
- Use the following command to verify the installation:
```sh
nvidia-smi
```
#### CUDA Toolkit
- Install a [CUDA toolkit](https://developer.nvidia.com/cuda-downloads) compatible with your NVIDIA driver.
- Use the following command to verify the installation:
```sh
nvcc --version
```
#### Linux Specifics
- Ensure that `gcc-11`, `g++-11`, `cpp-11`, or higher is installed.
- See [instructions](https://gcc.gnu.org/projects/cxx-status.html#cxx17) for Ubuntu installation.
- **Post-Installation Actions**: Add CUDA libraries to `LD_LIBRARY_PATH`.
- Follow the [Post-installation Actions](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions) instructions.
</TabItem>
</Tabs>
### 2. Switch to GPU Mode
Jan defaults to CPU mode but automatically switches to GPU mode if your system supports it, selecting the GPU with the highest VRAM. Check this setting in `Settings` > `Advanced Settings`.
#### Troubleshooting Tips
If GPU mode isn't enabled by default:
1. Confirm that you have installed an NVIDIA driver supporting CUDA 11.7 or higher. Refer to [CUDA compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver).
2. Ensure compatibility of the CUDA toolkit with your NVIDIA driver. Refer to [CUDA compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver).
3. For Linux, add CUDA's `.so` libraries to the `LD_LIBRARY_PATH`. For Windows, ensure that CUDA's `.dll` libraries are in the PATH. Refer to [Windows setup](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html#environment-setup).
### 3. Check GPU Settings
1. Navigate to `Settings` > `Advanced Settings` > `Jan Data Folder` to access GPU settings.
2. Open the `settings.json` file in the `settings` folder. Here's an example:
```json title="~/jan/settings/settings.json"
{
"notify": true,
"run_mode": "gpu",
"nvidia_driver": {
"exist": true,
"version": "531.18"
},
"cuda": {
"exist": true,
"version": "12"
},
"gpus": [
{
"id": "0",
"vram": "12282"
},
{
"id": "1",
"vram": "6144"
},
{
"id": "2",
"vram": "6144"
}
],
"gpu_highest_vram": "0"
}
```
### 4. Restart Jan
Restart Jan application to make sure it works.
#### Troubleshooting Tips
- Ensure `nvidia_driver` and `cuda` fields indicate installed software.
- If `gpus` field is empty or lacks your GPU, check NVIDIA driver and CUDA toolkit installations.
- For further assistance, share the `settings.json` file.
### Tested Configurations
- **Windows 11 Pro 64-bit:**
- GPU: NVIDIA GeForce RTX 4070ti
- CUDA: 12.2
- NVIDIA driver: 531.18 (Bare metal)
- **Ubuntu 22.04 LTS:**
- GPU: NVIDIA GeForce RTX 4070ti
- CUDA: 12.2
- NVIDIA driver: 545 (Bare metal)
- **Ubuntu 20.04 LTS:**
- GPU: NVIDIA GeForce GTX 1660ti
- CUDA: 12.1
- NVIDIA driver: 535 (Proxmox VM passthrough GPU)
- **Ubuntu 18.04 LTS:**
- GPU: NVIDIA GeForce GTX 1660ti
- CUDA: 12.1
- NVIDIA driver: 535 (Proxmox VM passthrough GPU)
### Common Issues and Solutions
1. If the issue persists, try installing the [Nightly version](https://jan.ai/install/nightly/).
2. Ensure your (V)RAM is accessible; some users with virtual RAM may require additional configuration.
3. Seek assistance in [Jan Discord](https://discord.gg/mY69SZaMaC).

49
docs/docs/guides/error-codes/how-to-get-error-logs.mdx
View File

@ -1,49 +0,0 @@
---
title: How to Get Error Logs
sidebar_position: 5
description: A step-by-step guide to get the Jan app error logs.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
permission denied,
]
---
To get the error logs of your Jan application, follow the steps below:
### Jan Application
1. Navigate to the main dashboard.
2. Click the **gear icon (⚙️)** on the bottom left of your screen.
3. Under the **Settings screen**, click the **Advanced Settings**.
4. On the **Jan Data Folder** click the **folder icon (📂)** to access the data.
5. Click the **logs** folder.
### Jan UI
1. Open your Unix or Linux terminal.
2. Use the following commands to get the recent 50 lines of log files:
```bash
tail -n 50 ~/jan/logs/app.log
```
### Jan API Server
1. Open your Unix or Linux terminal.
2. Use the following commands to get the recent 50 lines of log files:
```bash
tail -n 50 ~/jan/logs/server.log
```
:::warning
Ensure to redact any private or sensitive information when sharing logs or error details.
:::
:::note
If you have any questions or are looking for support, please don't hesitate to contact us via our [Discord community](https://discord.gg/Dt7MxDyNNZ) or create a new issue in our [GitHub repository](https://github.com/janhq/jan/issues/new/choose).
:::

31
docs/docs/guides/error-codes/no-assistant-available.mdx
View File

@ -1,31 +0,0 @@
---
title: No Assistant Available
sidebar_position: 7
description: Troubleshooting steps to resolve issues no assistant available.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
no assistant available,
]
---
When you encounter the following error message:
```
No assistant available.
```
This issue arises when a new, unintentional file appears in `/jan/assistants`.
It can be resolved through the following steps:
1. Access the `/jan/assistants` directory using a file manager or terminal.
2. Within `/jan/assistants`, this directory should only contain a folder named `jan`. Identify any file outside of this folder and remove it.

39
docs/docs/guides/error-codes/permission-denied.mdx
View File

@ -1,39 +0,0 @@
---
title: Permission Denied
sidebar_position: 1
description: A step-by-step guide to fix the issue when access is denied due to insufficient permissions.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
permission denied,
]
---
When running Jan, you might encounter the following error message:
```
Uncaught (in promise) Error: Error invoking layout-480796bff433a3a3.js:538 remote method 'installExtension':
Error Package /Applications/Jan.app/Contents/Resources/app.asar.unpacked/pre-install/janhq-assistant-extension-1.0.0.tgz does not contain a valid manifest:
Error EACCES: permission denied, mkdtemp '/Users/username/.npm/_cacache/tmp/ueCMn4'
```
This error mainly caused by permission problem during installation. To resolve this issue, follow these steps:
1. Open your terminal.
2. Execute the following command to change ownership of the `~/.npm` directory to the current user:
```sh
sudo chown -R $(whoami) ~/.npm
```
:::note
This command ensures that the necessary permissions are granted for Jan installation, resolving the encountered error.
:::

53
docs/docs/guides/error-codes/something-amiss.mdx
View File

@ -1,53 +0,0 @@
---
title: Something's Amiss
sidebar_position: 4
description: A step-by-step guide to resolve an unspecified or general error.
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
When you start a chat with a model and encounter with a Something's Amiss error, here's how to resolve it:
1. Ensure your OS is up to date.
2. Choose a model smaller than 80% of your hardware's V/RAM. For example, on an 8GB machine, opt for models smaller than 6GB.
3. Install the latest [Nightly release](https://jan.ai/install/nightly/) or [clear the application cache](https://jan.ai/troubleshooting/stuck-on-broken-build/) when reinstalling Jan.
4. Confirm your V/RAM accessibility, particularly if using virtual RAM.
5. Nvidia GPU users should download [CUDA](https://developer.nvidia.com/cuda-downloads).
6. Linux users, ensure your system meets the requirements of gcc 11, g++ 11, cpp 11, or higher. Refer to this [link](https://jan.ai/guides/troubleshooting/gpu-not-used/#specific-requirements-for-linux) for details.
7. You might use the wrong port when you [check the app logs](https://jan.ai/troubleshooting/how-to-get-error-logs/) and encounter the Bind address failed at 127.0.0.1:3928 error. To check the port status, try use the `netstat` command, like the following:
<Tabs>
<TabItem value="mac" label="MacOS" default>
```sh
netstat -an | grep 3928
```
</TabItem>
<TabItem value="windows" label="Windows" default>
```sh
netstat -ano | find "3928"
tasklist /fi "PID eq 3928"
```
</TabItem>
<TabItem value="linux" label="Linux" default>
```sh
netstat -anpe | grep "3928"
```
</TabItem>
</Tabs>
:::note
`Netstat` displays the contents of various network-related data structures for active connections
:::
:::tip
Jan uses the following ports:
- Nitro: `3928`
- Jan API Server: `1337`
- Jan Documentation: `3001`
:::

62
docs/docs/guides/error-codes/stuck-on-loading-model.mdx
View File

@ -1,62 +0,0 @@
---
title: Stuck on Loading Model
sidebar_position: 8
description: Troubleshooting steps to resolve issues related to the loading model.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
stuck on loading model,
]
---
## 1. Issue: Model Loading Stuck Due To Missing Windows Management Instrumentation Command-line (WMIC)
Encountering a stuck-on-loading model issue in Jan is caused by errors related to the `Windows Management Instrumentation Command-line (WMIC)` path not being included in the system's PATH environment variable.
Error message:
```
index.js:47 Uncaught (in promise) Error: Error invoking remote method 'invokeExtensionFunc': Error: Command failed: WMIC CPU Get NumberOfCores
```
It can be resolved through the following steps:
1. **Open System Properties:**
- Press `Windows key + R`.
- Type `sysdm.cpl` and press `Enter`.
2. **Access Environment Variables:**
- Go to the "Advanced" tab.
- Click the "Environment Variables" button.
3. **Edit System PATH:**
- Under "System Variables" find and select `Path`.
- Click "Edit."
4. **Add WMIC Path:**
- Click "New" and enter `C:\Windows\System32\Wbem`.
5. **Save Changes:**
- Click "OK" to close and save your changes.
6. **Verify Installation:**
- Restart any command prompts or terminals.
- Run `where wmic` to verify. Expected output: `C:\Windows\System32\wbem\WMIC.exe`.
## 2. Issue: Model Loading Stuck Due To CPU Without AVX
Encountering an issue with models stuck on loading in Jan can be due to the use of older generation CPUs that do not support Advanced Vector Extensions (AVX).
To check if your CPU supports AVX, visit the following link: [CPUs with AVX](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX)
:::warning [Please use this with caution]
As a workaround, consider using an [emulator](https://www.intel.com/content/www/us/en/developer/articles/tool/software-development-emulator.html) to simulate AVX support.
:::

26
docs/docs/guides/error-codes/thread-disappreance.mdx
View File

@ -1,26 +0,0 @@
---
title: Thread Disappearance
sidebar_position: 6
description: Troubleshooting steps to resolve issues threads suddenly disappearance.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
thread disappearance,
]
---
When you encounter the error of old threads suddenly disappear. This can happen when a new, unintentional file is created in `/jan/threads`.
It can be resolved through the following steps:
1. Go to `/jan/threads`.
2. The `/jan/threads` directory contains many folders named with the prefix `jan_` followed by an ID (e.g., `jan_123`). Look for any file not conforming to this naming pattern and remove it.

26
docs/docs/guides/error-codes/undefined-issue.mdx
View File

@ -1,26 +0,0 @@
---
title: Undefined Issue
sidebar_position: 3
description: A step-by-step guide to resolve errors when a variable or object is not defined.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
undefined issue,
]
---
Encountering an `undefined issue` in Jan is caused by errors related to the Nitro tool or other internal processes. It can be resolved through the following steps:
1. Clearing the Jan folder and then reopen the application to determine if the problem persists
2. Manually run the nitro tool located at `~/jan/extensions/@janhq/inference-nitro-extensions/dist/bin/(your-os)/nitro` to check for error messages.
3. Address any nitro error messages that are identified and reassess the persistence of the issue.
4. Reopen Jan to determine if the problem has been resolved after addressing any identified errors.
5. If the issue persists, please share the [app logs](https://jan.ai/troubleshooting/how-to-get-error-logs/) via [Jan Discord](https://discord.gg/mY69SZaMaC) for further assistance and troubleshooting.

24
docs/docs/guides/error-codes/unexpected-token.mdx
View File

@ -1,24 +0,0 @@
---
title: Unexpected Token
sidebar_position: 2
description: A step-by-step guide to correct syntax errors caused by invalid JSON in the code.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
troubleshooting,
unexpected token,
]
---
Encountering the `Unexpected token` error when initiating a chat with OpenAI models mainly caused by either your OpenAI key or where you access your OpenAI from. This issue can be solved through the following steps:
1. Obtain an OpenAI API key from [OpenAI's developer platform](https://platform.openai.com/) and integrate it into your application.
2. Trying a VPN could potentially solve the issue, especially if it's related to region locking for accessing OpenAI services. By connecting through a VPN, you may bypass such restrictions and successfully initiate chats with OpenAI models.

22
docs/docs/guides/extensions/README.mdx
View File

@ -1,22 +0,0 @@
---
title: Extensions
slug: /guides/extensions/
sidebar_position: 5
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
build extension,
]
---
import DocCardList from "@theme/DocCardList";
<DocCardList />

BIN
docs/docs/guides/extensions/assets/extension-setup.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 83 KiB

BIN
docs/docs/guides/extensions/assets/jan-ai-extensions.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 88 KiB

46
docs/docs/guides/extensions/setup-ext.mdx → docs/docs/guides/extensions/extensions.mdx
View File

@ -1,22 +1,24 @@
---
title: Extension Setup
title: What are Jan Extensions?
slug: /extensions
description: Jan Docs | Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
sidebar_position: 1
description: Dive into the available extensions and configure them.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
Rethink the Computer,
local AI,
private AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language model,
extension settings,
large language models,
Jan Extensions,
Extensions,
]
---
The current Jan Desktop Client has some default extensions built on top of this framework to enhance the user experience. In this guide, we will show you the list of default extensions and how to configure extension settings.
## Default Extensions
@ -26,8 +28,8 @@ You can find the default extensions in the `Settings` > `Extensions`.
## List of Default Extensions
| Extension Name | Version | Description | Source Code Link |
| -------------- | ------- | ----------- | ---------------- |
| Assistant Extension | `v1.0.0` | This extension enables assistants, including Jan, a default assistant that can call all downloaded models. | [Link to Source](https://github.com/janhq/jan/tree/dev/extensions/assistant-extension ) |
| ---------------------------------- | --------- | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| Assistant Extension | `v1.0.0` | This extension enables assistants, including Jan, a default assistant that can call all downloaded models. | [Link to Source](https://github.com/janhq/jan/tree/dev/extensions/assistant-extension) |
| Conversational Extension | `v1.0.0` | This extension enables conversations and state persistence via your filesystem. | [Link to Source](https://github.com/janhq/jan/tree/dev/extensions/conversational-extension) |
| Inference Nitro Extension | `v1.0.0` | This extension embeds Nitro, a lightweight (3 MB) inference engine in C++. See nitro.jan.ai. | [Link to Source](https://github.com/janhq/jan/tree/dev/extensions/inference-nitro-extension) |
| Inference Openai Extension | `v1.0.0` | This extension enables OpenAI chat completion API calls. | [Link to Source](https://github.com/janhq/jan/tree/dev/extensions/inference-openai-extension) |
@ -36,24 +38,25 @@ You can find the default extensions in the `Settings` > `Extensions`.
| Monitoring Extension | `v1.0.9` | This extension offers system health and OS-level data. | [Link to Source](https://github.com/janhq/jan/tree/dev/extensions/monitoring-extension) |
## Configure Extension Settings
To configure extension settings:
1. Navigate to the `~/jan/extensions`.
2. Open the `extensions.json` file
3. Edit the file with options including:
| Option | Description |
|-----------------|-------------------------------------------------|
| ---------------- | ----------------------------------- |
| `_active` | Enable/disable the extension. |
| `listeners` | Default listener setting. |
| `origin` | Extension file path. |
| `installOptions`| Version and metadata configuration. |
| `installOptions` | Version and metadata configuration. |
| `name` | Extension name. |
| `version` | Extension version. |
| `main` | Main file path. |
| `description` | Extension description. |
| `url` | Extension URL. |
```json title="~/jan/extensions/extensions.json"
{
"@janhq/assistant-extension": {
@ -136,6 +139,23 @@ To configure extension settings:
}
```
## Import Custom Extension
:::note
Currently, Jan only supports official extensions, which can be directly downloaded in Extension Settings. We plan to support 3rd party Extensions in the future.
:::
For now you can always import a third party extension at your own risk by following the steps below:
1. Navigate to **Settings** > **Extensions** > Click Select under **Manual Installation**.
2. Then, the ~/jan/extensions/extensions.json file will be updated automatically.
:::caution
You need to prepare the extension file in .tgz format to install the **non-default** extension.
:::
:::info[Assistance and Support]
If you have questions, please join our [Discord community](https://discord.gg/Dt7MxDyNNZ) for support, updates, and discussions.

36
docs/docs/guides/extensions/import-ext.mdx
View File

@ -1,36 +0,0 @@
---
title: Import Extensions
sidebar_position: 2
description: A step-by-step guide on how to import extensions.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
import extensions,
]
---
Besides default extensions, you can import extensions into Jan by following the steps below:
1. Navigate to **Settings** > **Extensions** > Click Select under **Manual Installation**.
2. Then, the ~/jan/extensions/extensions.json file will be updated automatically.
:::caution
You need to prepare the extension file in .tgz format to install the **non-default** extension.
:::
:::info[Assistance and Support]
If you have questions, please join our [Discord community](https://discord.gg/Dt7MxDyNNZ) for support, updates, and discussions.
:::

99
docs/docs/guides/faq.mdx
View File

@ -1,99 +0,0 @@
---
title: FAQs
slug: /guides/faqs
sidebar_position: 12
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
acknowledgements,
third-party libraries,
]
---
## General Issues
- **Why can't I download models like Pandora 11B Q4 and Solar Instruct 10.7B Q4?**
- These models might have been removed or taken down. Please check the [Pre-configured Models](models-list.mdx) for the latest updates on model availability.
- **Why does Jan display "Apologies, something's amiss" when I try to run it?**
- This issue may arise if you're using an older Intel chip that does not fully support AVX instructions required for running AI models. Upgrading your hardware may resolve this issue.
- **How can I use Jan in Russia?**
- To use Jan in Russia, a VPN or [HTTPS - Proxy](./advanced-settings/http-proxy.mdx) is recommended to bypass any regional restrictions that might be in place.
- **I'm experiencing an error on startup from Nitro. What should I do?**
- If you encounter errors with Nitro, try switching the path to use the Nitro executable for the version 12-0. This adjustment can help resolve path-related issues.
## Download and Installation Issues
- **What does "Error occurred: Unexpected token" mean?**
- This error usually indicates a problem with your internet connection or that your access to certain resources is being blocked. Using a VPN or [HTTPS - Proxy](./advanced-settings/http-proxy.mdx) can help avoid these issues by providing a secure and unrestricted internet connection.
- **Why aren't my downloads working?**
- If you're having trouble downloading directly through Jan, you might want to download the model separately and then import it into Jan. Detailed instructions are available on [here](install.mdx).
- **Jan AI doesn't open on my Mac with an Intel processor. What can I do?**
- Granting the `.npm` folder permission for the user can resolve issues related to permissions on macOS, especially for users with Intel processors.
- **What should I do if the model download freezes?**
- If a model download freezes, consider importing the models manually. You can find more detailed guidance on how to do this at [Manual Import](./models/import-models.mdx) article.
- **I received a message that the model GPT4 does not exist or I do not have access. What should I do?**
- This message typically means you need to top up your credit with OpenAI or check your access permissions for the model.
- **I can't download models from "Explore the Hub." What's the solution?**
- Uninstalling Jan, clearing the cache, and reinstalling it following the guide provided [here](install.mdx) may help. Also, consider downloading the `.gguf` model via a browser as an alternative approach.
## Technical Issues and Solutions
- **How can I download models with a socks5 proxy or import a local model file?**
- Nightly builds of Jan offer support for downloading models with socks5 proxies or importing local model files.
- **My device shows no GPU usage and lacks a Settings folder. What should I do?**
- Using the nightly builds of Jan can address issues related to GPU usage and the absence of a Settings folder, as these builds contain the latest fixes and features.
- **Why does Jan display a toast message saying a model is loaded when it is not actually loaded?**
- This issue can be resolved by downloading the `.gguf` file from Hugging Face and replacing it in the model folder. This ensures the correct model is loaded.
- **How to enable CORS when running Nitro?**
- By default, CORS (Cross-Origin Resource Sharing) is disabled when running Nitro. Enabling CORS can be necessary for certain operations and integrations. Check the official documentation for instructions on how to enable CORS if your workflow requires it.
## Compatibility and Support
- **How to use GPU AMD for Jan?**
- Jan now supports AMD GPUs through Vulkan. This enhancement allows users with AMD graphics cards to leverage GPU acceleration, improving performance for AI model computations.
- **Is Jan available for Android or iOS?**
- Jan is primarily focused on the Desktop app and does not currently offer mobile apps for Android or iOS. The development team is concentrating on enhancing the desktop experience.
## Development and Features
- **Does Jan support Safetensors?**
- At the moment, Jan only supports GGUF. However, there are plans to support `.safetensor` files in the future.
- **I hope to customize the installation path of each model. Is that possible?**
- Yes you can customize the installation path. Please see [here](https://jan.ai/guides/advanced-settings/#access-the-jan-data-folder) for more information.
## Troubleshooting
- **What should I do if there's high CPU usage while Jan is idle?**
- If you notice high CPU usage while Jan is idle, consider using the nightly builds of Jan
- **What does the error "Failed to fetch" mean, and how can I fix it?**
- The "Failed to fetch" error typically occurs due to network issues or restrictions. Using the nightly builds of Jan may help overcome these issues by providing updated fixes and features.
- **What should I do if "Failed to fetch" occurs using MacBook Pro with Intel HD Graphics 4000 1536 MB?**
- Ensure that the model size is less than 90% of your available VRAM and that the VRAM is accessible to the app. Managing the resources effectively can help mitigate this issue.
:::info[Assistance and Support]
If you have questions, please join our [Discord community](https://discord.gg/Dt7MxDyNNZ) for support, updates, and discussions.
:::

BIN
docs/docs/guides/get-started/asset/download.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 MiB

BIN
docs/docs/guides/get-started/asset/gpt.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.3 MiB

BIN
docs/docs/guides/get-started/asset/model.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.4 MiB

25
docs/docs/guides/get-started/hardware-setup.mdx Normal file
View File

@ -0,0 +1,25 @@
---
title: Hardware Setup
slug: /guides/hardware
description: Jan Docs | Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
sidebar_position: 3
keywords:
[
Jan,
Rethink the Computer,
local AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language models,
hardware requirements,
Nvidia,
AMD,
CPU,
GPU,
]
---
Coming Soon

20
docs/docs/guides/get-started/overview.mdx Normal file
View File

@ -0,0 +1,20 @@
---
title: Overview
slug: /guides
description: Jan Docs | Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
sidebar_position: 1
keywords:
[
Jan,
Rethink the Computer,
local AI,
privacy focus,
free and open source,
private and offline,
conversational AI,
no-subscription fee,
large language models,
]
---
Coming Soon

Some files were not shown because too many files have changed in this diff Show More