diff --git a/docs/src/pages/docs/_assets/Anthropic-1.gif b/docs/src/pages/docs/_assets/Anthropic-1.gif deleted file mode 100644 index 594c53172..000000000 Binary files a/docs/src/pages/docs/_assets/Anthropic-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Anthropic-2.gif b/docs/src/pages/docs/_assets/Anthropic-2.gif deleted file mode 100644 index 63e76cc3b..000000000 Binary files a/docs/src/pages/docs/_assets/Anthropic-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Cohere-1.gif b/docs/src/pages/docs/_assets/Cohere-1.gif deleted file mode 100644 index 6441bd8a9..000000000 Binary files a/docs/src/pages/docs/_assets/Cohere-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Cohere-2.gif b/docs/src/pages/docs/_assets/Cohere-2.gif deleted file mode 100644 index 9488040c0..000000000 Binary files a/docs/src/pages/docs/_assets/Cohere-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Groq-1.gif b/docs/src/pages/docs/_assets/Groq-1.gif deleted file mode 100644 index d95e577b1..000000000 Binary files a/docs/src/pages/docs/_assets/Groq-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Groq-2.gif b/docs/src/pages/docs/_assets/Groq-2.gif deleted file mode 100644 index 46231696f..000000000 Binary files a/docs/src/pages/docs/_assets/Groq-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/LM-Studio-v1.gif b/docs/src/pages/docs/_assets/LM-Studio-v1.gif deleted file mode 100644 index 65dfb5db1..000000000 Binary files a/docs/src/pages/docs/_assets/LM-Studio-v1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/LM-Studio-v2.gif b/docs/src/pages/docs/_assets/LM-Studio-v2.gif deleted file mode 100644 index ad012df5d..000000000 Binary files a/docs/src/pages/docs/_assets/LM-Studio-v2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/LM-Studio-v3.gif b/docs/src/pages/docs/_assets/LM-Studio-v3.gif deleted file mode 100644 index d5208c2bc..000000000 Binary files a/docs/src/pages/docs/_assets/LM-Studio-v3.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Martian-1.gif b/docs/src/pages/docs/_assets/Martian-1.gif deleted file mode 100644 index 3da994cc4..000000000 Binary files a/docs/src/pages/docs/_assets/Martian-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Martian-2.gif b/docs/src/pages/docs/_assets/Martian-2.gif deleted file mode 100644 index 27fa761b7..000000000 Binary files a/docs/src/pages/docs/_assets/Martian-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Mistral-1.gif b/docs/src/pages/docs/_assets/Mistral-1.gif deleted file mode 100644 index 7d1e82108..000000000 Binary files a/docs/src/pages/docs/_assets/Mistral-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Mistral-2.gif b/docs/src/pages/docs/_assets/Mistral-2.gif deleted file mode 100644 index 25c8d2fca..000000000 Binary files a/docs/src/pages/docs/_assets/Mistral-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Ollama-1.gif b/docs/src/pages/docs/_assets/Ollama-1.gif deleted file mode 100644 index d9c7fa135..000000000 Binary files a/docs/src/pages/docs/_assets/Ollama-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Ollama-2.gif b/docs/src/pages/docs/_assets/Ollama-2.gif deleted file mode 100644 index eadc6afdd..000000000 Binary files a/docs/src/pages/docs/_assets/Ollama-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Ollama-3.gif b/docs/src/pages/docs/_assets/Ollama-3.gif deleted file mode 100644 index 6b09000b9..000000000 Binary files a/docs/src/pages/docs/_assets/Ollama-3.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/OpenAi-1.gif b/docs/src/pages/docs/_assets/OpenAi-1.gif deleted file mode 100644 index c98e2f2be..000000000 Binary files a/docs/src/pages/docs/_assets/OpenAi-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/OpenAi-2.gif b/docs/src/pages/docs/_assets/OpenAi-2.gif deleted file mode 100644 index 8cd449f7f..000000000 Binary files a/docs/src/pages/docs/_assets/OpenAi-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/OpenRouter-1.gif b/docs/src/pages/docs/_assets/OpenRouter-1.gif deleted file mode 100644 index cd6a7898f..000000000 Binary files a/docs/src/pages/docs/_assets/OpenRouter-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/OpenRouter-2.gif b/docs/src/pages/docs/_assets/OpenRouter-2.gif deleted file mode 100644 index 307b07deb..000000000 Binary files a/docs/src/pages/docs/_assets/OpenRouter-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/amd.gif b/docs/src/pages/docs/_assets/amd.gif deleted file mode 100644 index cc053b80c..000000000 Binary files a/docs/src/pages/docs/_assets/amd.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/assistant-slider.png b/docs/src/pages/docs/_assets/assistant-slider.png deleted file mode 100644 index 8289e0d51..000000000 Binary files a/docs/src/pages/docs/_assets/assistant-slider.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/assistant1.gif b/docs/src/pages/docs/_assets/assistant1.gif deleted file mode 100644 index 75a56ba27..000000000 Binary files a/docs/src/pages/docs/_assets/assistant1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/assistant2.gif b/docs/src/pages/docs/_assets/assistant2.gif deleted file mode 100644 index 44bfb2264..000000000 Binary files a/docs/src/pages/docs/_assets/assistant2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/chat.gif b/docs/src/pages/docs/_assets/chat.gif deleted file mode 100644 index 94b8c2719..000000000 Binary files a/docs/src/pages/docs/_assets/chat.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/clear.gif b/docs/src/pages/docs/_assets/clear.gif deleted file mode 100644 index 445708d22..000000000 Binary files a/docs/src/pages/docs/_assets/clear.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/configure.gif b/docs/src/pages/docs/_assets/configure.gif deleted file mode 100644 index 96daaaa61..000000000 Binary files a/docs/src/pages/docs/_assets/configure.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/data-folder.png b/docs/src/pages/docs/_assets/data-folder.png deleted file mode 100644 index def0f38c7..000000000 Binary files a/docs/src/pages/docs/_assets/data-folder.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/default.gif b/docs/src/pages/docs/_assets/default.gif deleted file mode 100644 index 3bc6a68f6..000000000 Binary files a/docs/src/pages/docs/_assets/default.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/delete-tensor.gif b/docs/src/pages/docs/_assets/delete-tensor.gif deleted file mode 100644 index 86f45fc65..000000000 Binary files a/docs/src/pages/docs/_assets/delete-tensor.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/delete-threads.png b/docs/src/pages/docs/_assets/delete-threads.png deleted file mode 100644 index b134e8e12..000000000 Binary files a/docs/src/pages/docs/_assets/delete-threads.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/delete.png b/docs/src/pages/docs/_assets/delete.png deleted file mode 100644 index 6b3f669c2..000000000 Binary files a/docs/src/pages/docs/_assets/delete.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/disable-tensor.gif b/docs/src/pages/docs/_assets/disable-tensor.gif deleted file mode 100644 index 91e9fe3d6..000000000 Binary files a/docs/src/pages/docs/_assets/disable-tensor.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-button.png b/docs/src/pages/docs/_assets/download-button.png deleted file mode 100644 index 2e8088d15..000000000 Binary files a/docs/src/pages/docs/_assets/download-button.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-button2.png b/docs/src/pages/docs/_assets/download-button2.png deleted file mode 100644 index 8f8c683c3..000000000 Binary files a/docs/src/pages/docs/_assets/download-button2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-button3.png b/docs/src/pages/docs/_assets/download-button3.png deleted file mode 100644 index cf05ec45a..000000000 Binary files a/docs/src/pages/docs/_assets/download-button3.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-icon.png b/docs/src/pages/docs/_assets/download-icon.png deleted file mode 100644 index f553e19f2..000000000 Binary files a/docs/src/pages/docs/_assets/download-icon.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-llama2.gif b/docs/src/pages/docs/_assets/download-llama2.gif deleted file mode 100644 index 18d671209..000000000 Binary files a/docs/src/pages/docs/_assets/download-llama2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-model.gif b/docs/src/pages/docs/_assets/download-model.gif deleted file mode 100644 index e10b293c2..000000000 Binary files a/docs/src/pages/docs/_assets/download-model.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-model2.gif b/docs/src/pages/docs/_assets/download-model2.gif deleted file mode 100644 index c69819a1e..000000000 Binary files a/docs/src/pages/docs/_assets/download-model2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-phi.gif b/docs/src/pages/docs/_assets/download-phi.gif deleted file mode 100644 index 5b2cd6034..000000000 Binary files a/docs/src/pages/docs/_assets/download-phi.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download.gif b/docs/src/pages/docs/_assets/download.gif deleted file mode 100644 index 10cb4b466..000000000 Binary files a/docs/src/pages/docs/_assets/download.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download.png b/docs/src/pages/docs/_assets/download.png deleted file mode 100644 index 571e44dd3..000000000 Binary files a/docs/src/pages/docs/_assets/download.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/exp-mode.png b/docs/src/pages/docs/_assets/exp-mode.png deleted file mode 100644 index 14ecf6bfe..000000000 Binary files a/docs/src/pages/docs/_assets/exp-mode.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/experimental.gif b/docs/src/pages/docs/_assets/experimental.gif deleted file mode 100644 index 11c2d8622..000000000 Binary files a/docs/src/pages/docs/_assets/experimental.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-04.png b/docs/src/pages/docs/_assets/extensions-04.png deleted file mode 100644 index 3dc270b23..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-04.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-05.png b/docs/src/pages/docs/_assets/extensions-05.png deleted file mode 100644 index 3967d90a8..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-05.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-06.png b/docs/src/pages/docs/_assets/extensions-06.png deleted file mode 100644 index b169e3926..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-06.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-07.png b/docs/src/pages/docs/_assets/extensions-07.png deleted file mode 100644 index 3d39f56d9..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-07.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-08.png b/docs/src/pages/docs/_assets/extensions-08.png deleted file mode 100644 index 3d124e367..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-08.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-09.png b/docs/src/pages/docs/_assets/extensions-09.png deleted file mode 100644 index 7d7cd6193..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-09.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-10.png b/docs/src/pages/docs/_assets/extensions-10.png deleted file mode 100644 index ecadd8475..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-10.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-page.png b/docs/src/pages/docs/_assets/extensions-page.png deleted file mode 100644 index 74a336769..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-page.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-page2.png b/docs/src/pages/docs/_assets/extensions-page2.png deleted file mode 100644 index 51863aa99..000000000 Binary files a/docs/src/pages/docs/_assets/extensions-page2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/gpt.gif b/docs/src/pages/docs/_assets/gpt.gif deleted file mode 100644 index 3972e812f..000000000 Binary files a/docs/src/pages/docs/_assets/gpt.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/gpu-accel.png b/docs/src/pages/docs/_assets/gpu-accel.png deleted file mode 100644 index 294013fcf..000000000 Binary files a/docs/src/pages/docs/_assets/gpu-accel.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/gpu.gif b/docs/src/pages/docs/_assets/gpu.gif deleted file mode 100644 index 58d1da970..000000000 Binary files a/docs/src/pages/docs/_assets/gpu.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/gpu2.gif b/docs/src/pages/docs/_assets/gpu2.gif deleted file mode 100644 index fcacc5e99..000000000 Binary files a/docs/src/pages/docs/_assets/gpu2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/history.png b/docs/src/pages/docs/_assets/history.png deleted file mode 100644 index 7112a0d24..000000000 Binary files a/docs/src/pages/docs/_assets/history.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/http.png b/docs/src/pages/docs/_assets/http.png deleted file mode 100644 index afa0aeae0..000000000 Binary files a/docs/src/pages/docs/_assets/http.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/https.gif b/docs/src/pages/docs/_assets/https.gif deleted file mode 100644 index e9ed05b85..000000000 Binary files a/docs/src/pages/docs/_assets/https.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/hub.gif b/docs/src/pages/docs/_assets/hub.gif deleted file mode 100644 index fad29d5db..000000000 Binary files a/docs/src/pages/docs/_assets/hub.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/hub.png b/docs/src/pages/docs/_assets/hub.png deleted file mode 100644 index 75b0f575d..000000000 Binary files a/docs/src/pages/docs/_assets/hub.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/import-folder.gif b/docs/src/pages/docs/_assets/import-folder.gif deleted file mode 100644 index fc7822207..000000000 Binary files a/docs/src/pages/docs/_assets/import-folder.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/import-hf.gif b/docs/src/pages/docs/_assets/import-hf.gif deleted file mode 100644 index 4ca1a8b2a..000000000 Binary files a/docs/src/pages/docs/_assets/import-hf.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/import.png b/docs/src/pages/docs/_assets/import.png deleted file mode 100644 index 5ca7fff42..000000000 Binary files a/docs/src/pages/docs/_assets/import.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/import2.png b/docs/src/pages/docs/_assets/import2.png deleted file mode 100644 index caf2c022d..000000000 Binary files a/docs/src/pages/docs/_assets/import2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/inf.gif b/docs/src/pages/docs/_assets/inf.gif deleted file mode 100644 index aaef44380..000000000 Binary files a/docs/src/pages/docs/_assets/inf.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/install-ext.gif b/docs/src/pages/docs/_assets/install-ext.gif deleted file mode 100644 index 4357ed236..000000000 Binary files a/docs/src/pages/docs/_assets/install-ext.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/install-ext.png b/docs/src/pages/docs/_assets/install-ext.png deleted file mode 100644 index ee2ff8453..000000000 Binary files a/docs/src/pages/docs/_assets/install-ext.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/install-tensor.gif b/docs/src/pages/docs/_assets/install-tensor.gif deleted file mode 100644 index 927ee0b9b..000000000 Binary files a/docs/src/pages/docs/_assets/install-tensor.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/install-tensor.png b/docs/src/pages/docs/_assets/install-tensor.png deleted file mode 100644 index 050075d84..000000000 Binary files a/docs/src/pages/docs/_assets/install-tensor.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/linux.png b/docs/src/pages/docs/_assets/linux.png deleted file mode 100644 index 72230191c..000000000 Binary files a/docs/src/pages/docs/_assets/linux.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/llama2.gif b/docs/src/pages/docs/_assets/llama2.gif deleted file mode 100644 index 0c9502aa6..000000000 Binary files a/docs/src/pages/docs/_assets/llama2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api1.png b/docs/src/pages/docs/_assets/local-api1.png deleted file mode 100644 index d636d37ad..000000000 Binary files a/docs/src/pages/docs/_assets/local-api1.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api2.png b/docs/src/pages/docs/_assets/local-api2.png deleted file mode 100644 index 58652775c..000000000 Binary files a/docs/src/pages/docs/_assets/local-api2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api3.png b/docs/src/pages/docs/_assets/local-api3.png deleted file mode 100644 index aa8372640..000000000 Binary files a/docs/src/pages/docs/_assets/local-api3.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api4.png b/docs/src/pages/docs/_assets/local-api4.png deleted file mode 100644 index 2a66c9c1f..000000000 Binary files a/docs/src/pages/docs/_assets/local-api4.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api5.png b/docs/src/pages/docs/_assets/local-api5.png deleted file mode 100644 index 2dc80c6c7..000000000 Binary files a/docs/src/pages/docs/_assets/local-api5.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/mac2.png b/docs/src/pages/docs/_assets/mac2.png deleted file mode 100644 index aeb4b6bbc..000000000 Binary files a/docs/src/pages/docs/_assets/mac2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-anthropic.gif b/docs/src/pages/docs/_assets/model-anthropic.gif deleted file mode 100644 index 8de07e625..000000000 Binary files a/docs/src/pages/docs/_assets/model-anthropic.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-cohere.gif b/docs/src/pages/docs/_assets/model-cohere.gif deleted file mode 100644 index 2bf5007ff..000000000 Binary files a/docs/src/pages/docs/_assets/model-cohere.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-groq.gif b/docs/src/pages/docs/_assets/model-groq.gif deleted file mode 100644 index 7c6d20280..000000000 Binary files a/docs/src/pages/docs/_assets/model-groq.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-management-01.png b/docs/src/pages/docs/_assets/model-management-01.png index 61524ecdb..9020c66b2 100644 Binary files a/docs/src/pages/docs/_assets/model-management-01.png and b/docs/src/pages/docs/_assets/model-management-01.png differ diff --git a/docs/src/pages/docs/_assets/model-management-04.png b/docs/src/pages/docs/_assets/model-management-04.png index 6bc4cea35..71d72dff2 100644 Binary files a/docs/src/pages/docs/_assets/model-management-04.png and b/docs/src/pages/docs/_assets/model-management-04.png differ diff --git a/docs/src/pages/docs/_assets/model-management-06.png b/docs/src/pages/docs/_assets/model-management-06.png index b8c5afd96..ebd2de3d1 100644 Binary files a/docs/src/pages/docs/_assets/model-management-06.png and b/docs/src/pages/docs/_assets/model-management-06.png differ diff --git a/docs/src/pages/docs/_assets/model-management-07.png b/docs/src/pages/docs/_assets/model-management-07.png index a36f0038b..612904ac7 100644 Binary files a/docs/src/pages/docs/_assets/model-management-07.png and b/docs/src/pages/docs/_assets/model-management-07.png differ diff --git a/docs/src/pages/docs/_assets/model-martian.gif b/docs/src/pages/docs/_assets/model-martian.gif deleted file mode 100644 index 8bf4dbbf5..000000000 Binary files a/docs/src/pages/docs/_assets/model-martian.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-mistral.gif b/docs/src/pages/docs/_assets/model-mistral.gif deleted file mode 100644 index a26661903..000000000 Binary files a/docs/src/pages/docs/_assets/model-mistral.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-openai.gif b/docs/src/pages/docs/_assets/model-openai.gif deleted file mode 100644 index b20aa9c0e..000000000 Binary files a/docs/src/pages/docs/_assets/model-openai.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-tab.png b/docs/src/pages/docs/_assets/model-tab.png deleted file mode 100644 index ea595bf19..000000000 Binary files a/docs/src/pages/docs/_assets/model-tab.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model.gif b/docs/src/pages/docs/_assets/model.gif deleted file mode 100644 index e5abdf757..000000000 Binary files a/docs/src/pages/docs/_assets/model.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/modelparam.gif b/docs/src/pages/docs/_assets/modelparam.gif deleted file mode 100644 index f33cf5ab5..000000000 Binary files a/docs/src/pages/docs/_assets/modelparam.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/models.gif b/docs/src/pages/docs/_assets/models.gif deleted file mode 100644 index 8dc4d7925..000000000 Binary files a/docs/src/pages/docs/_assets/models.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/mymodels.png b/docs/src/pages/docs/_assets/mymodels.png deleted file mode 100644 index d12664b86..000000000 Binary files a/docs/src/pages/docs/_assets/mymodels.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/nitro-network.png b/docs/src/pages/docs/_assets/nitro-network.png deleted file mode 100644 index 5f1f7104e..000000000 Binary files a/docs/src/pages/docs/_assets/nitro-network.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/nvidia-1.gif b/docs/src/pages/docs/_assets/nvidia-1.gif deleted file mode 100644 index df28abc1f..000000000 Binary files a/docs/src/pages/docs/_assets/nvidia-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/nvidia-2.gif b/docs/src/pages/docs/_assets/nvidia-2.gif deleted file mode 100644 index 4c628884a..000000000 Binary files a/docs/src/pages/docs/_assets/nvidia-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/phi.gif b/docs/src/pages/docs/_assets/phi.gif deleted file mode 100644 index e8cd587d2..000000000 Binary files a/docs/src/pages/docs/_assets/phi.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/preserve.png b/docs/src/pages/docs/_assets/preserve.png deleted file mode 100644 index 459c940e2..000000000 Binary files a/docs/src/pages/docs/_assets/preserve.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/quick-ask.gif b/docs/src/pages/docs/_assets/quick-ask.gif deleted file mode 100644 index f4081ca3c..000000000 Binary files a/docs/src/pages/docs/_assets/quick-ask.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/quick-ask.png b/docs/src/pages/docs/_assets/quick-ask.png deleted file mode 100644 index 34620569b..000000000 Binary files a/docs/src/pages/docs/_assets/quick-ask.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/reset-jan.png b/docs/src/pages/docs/_assets/reset-jan.png deleted file mode 100644 index 2c20ca667..000000000 Binary files a/docs/src/pages/docs/_assets/reset-jan.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/reset.gif b/docs/src/pages/docs/_assets/reset.gif deleted file mode 100644 index 6ff171f25..000000000 Binary files a/docs/src/pages/docs/_assets/reset.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/retrieval.gif b/docs/src/pages/docs/_assets/retrieval.gif deleted file mode 100644 index 38d3fbb58..000000000 Binary files a/docs/src/pages/docs/_assets/retrieval.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/retrieval1.png b/docs/src/pages/docs/_assets/retrieval1.png deleted file mode 100644 index 8b5d63bb8..000000000 Binary files a/docs/src/pages/docs/_assets/retrieval1.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/retrieval2.png b/docs/src/pages/docs/_assets/retrieval2.png deleted file mode 100644 index 2c9443c44..000000000 Binary files a/docs/src/pages/docs/_assets/retrieval2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/scheme.png b/docs/src/pages/docs/_assets/scheme.png deleted file mode 100644 index fe50d4b6a..000000000 Binary files a/docs/src/pages/docs/_assets/scheme.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/search-bar.png b/docs/src/pages/docs/_assets/search-bar.png deleted file mode 100644 index b6e559773..000000000 Binary files a/docs/src/pages/docs/_assets/search-bar.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/select-openai.gif b/docs/src/pages/docs/_assets/select-openai.gif deleted file mode 100644 index 3196c4320..000000000 Binary files a/docs/src/pages/docs/_assets/select-openai.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-anthropic.gif b/docs/src/pages/docs/_assets/server-anthropic.gif deleted file mode 100644 index da684dbd4..000000000 Binary files a/docs/src/pages/docs/_assets/server-anthropic.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-cohere.gif b/docs/src/pages/docs/_assets/server-cohere.gif deleted file mode 100644 index 72c9bd883..000000000 Binary files a/docs/src/pages/docs/_assets/server-cohere.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-groq.gif b/docs/src/pages/docs/_assets/server-groq.gif deleted file mode 100644 index dcb3e03f7..000000000 Binary files a/docs/src/pages/docs/_assets/server-groq.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-llama2.gif b/docs/src/pages/docs/_assets/server-llama2.gif deleted file mode 100644 index b1ebafc84..000000000 Binary files a/docs/src/pages/docs/_assets/server-llama2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-martian.gif b/docs/src/pages/docs/_assets/server-martian.gif deleted file mode 100644 index a33dd7f19..000000000 Binary files a/docs/src/pages/docs/_assets/server-martian.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-mistral.gif b/docs/src/pages/docs/_assets/server-mistral.gif deleted file mode 100644 index 8fdb6ef0f..000000000 Binary files a/docs/src/pages/docs/_assets/server-mistral.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-openai.gif b/docs/src/pages/docs/_assets/server-openai.gif deleted file mode 100644 index fd10aa75b..000000000 Binary files a/docs/src/pages/docs/_assets/server-openai.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-openai2.gif b/docs/src/pages/docs/_assets/server-openai2.gif deleted file mode 100644 index b4338d122..000000000 Binary files a/docs/src/pages/docs/_assets/server-openai2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-phi.gif b/docs/src/pages/docs/_assets/server-phi.gif deleted file mode 100644 index 4abc916c1..000000000 Binary files a/docs/src/pages/docs/_assets/server-phi.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/set-model.gif b/docs/src/pages/docs/_assets/set-model.gif deleted file mode 100644 index 87de85732..000000000 Binary files a/docs/src/pages/docs/_assets/set-model.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/set-tensor.gif b/docs/src/pages/docs/_assets/set-tensor.gif deleted file mode 100644 index 22e824882..000000000 Binary files a/docs/src/pages/docs/_assets/set-tensor.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/set-url.gif b/docs/src/pages/docs/_assets/set-url.gif deleted file mode 100644 index b675b3ea4..000000000 Binary files a/docs/src/pages/docs/_assets/set-url.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/settings-06.png b/docs/src/pages/docs/_assets/settings-06.png index b097464c2..3a582b42a 100644 Binary files a/docs/src/pages/docs/_assets/settings-06.png and b/docs/src/pages/docs/_assets/settings-06.png differ diff --git a/docs/src/pages/docs/_assets/settings.png b/docs/src/pages/docs/_assets/settings.png deleted file mode 100644 index e53f71d1e..000000000 Binary files a/docs/src/pages/docs/_assets/settings.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/shortcut.png b/docs/src/pages/docs/_assets/shortcut.png deleted file mode 100644 index b7912f838..000000000 Binary files a/docs/src/pages/docs/_assets/shortcut.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/specific-model.gif b/docs/src/pages/docs/_assets/specific-model.gif deleted file mode 100644 index 2664bcc47..000000000 Binary files a/docs/src/pages/docs/_assets/specific-model.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/spell.png b/docs/src/pages/docs/_assets/spell.png deleted file mode 100644 index 92e9a6d8c..000000000 Binary files a/docs/src/pages/docs/_assets/spell.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/ssl.gif b/docs/src/pages/docs/_assets/ssl.gif deleted file mode 100644 index b18f075c1..000000000 Binary files a/docs/src/pages/docs/_assets/ssl.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/ssl.png b/docs/src/pages/docs/_assets/ssl.png deleted file mode 100644 index 970051744..000000000 Binary files a/docs/src/pages/docs/_assets/ssl.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-anthropic.gif b/docs/src/pages/docs/_assets/start-anthropic.gif deleted file mode 100644 index 9ca938655..000000000 Binary files a/docs/src/pages/docs/_assets/start-anthropic.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-cohere.gif b/docs/src/pages/docs/_assets/start-cohere.gif deleted file mode 100644 index b28fde498..000000000 Binary files a/docs/src/pages/docs/_assets/start-cohere.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-groq.gif b/docs/src/pages/docs/_assets/start-groq.gif deleted file mode 100644 index 8e5ced12a..000000000 Binary files a/docs/src/pages/docs/_assets/start-groq.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-martian.gif b/docs/src/pages/docs/_assets/start-martian.gif deleted file mode 100644 index 31a79d16d..000000000 Binary files a/docs/src/pages/docs/_assets/start-martian.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-mistral.gif b/docs/src/pages/docs/_assets/start-mistral.gif deleted file mode 100644 index 1012a651a..000000000 Binary files a/docs/src/pages/docs/_assets/start-mistral.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-router.gif b/docs/src/pages/docs/_assets/start-router.gif deleted file mode 100644 index ca1737de0..000000000 Binary files a/docs/src/pages/docs/_assets/start-router.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-mili.png b/docs/src/pages/docs/_assets/system-mili.png deleted file mode 100644 index 0a0f28592..000000000 Binary files a/docs/src/pages/docs/_assets/system-mili.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-mili2.png b/docs/src/pages/docs/_assets/system-mili2.png deleted file mode 100644 index 6135c5a49..000000000 Binary files a/docs/src/pages/docs/_assets/system-mili2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-monitor.png b/docs/src/pages/docs/_assets/system-monitor.png deleted file mode 100644 index 04d6749cb..000000000 Binary files a/docs/src/pages/docs/_assets/system-monitor.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-monitor2.png b/docs/src/pages/docs/_assets/system-monitor2.png deleted file mode 100644 index e03dfbf63..000000000 Binary files a/docs/src/pages/docs/_assets/system-monitor2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-slider.png b/docs/src/pages/docs/_assets/system-slider.png deleted file mode 100644 index 409338f70..000000000 Binary files a/docs/src/pages/docs/_assets/system-slider.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-slider2.png b/docs/src/pages/docs/_assets/system-slider2.png deleted file mode 100644 index 9114d32c3..000000000 Binary files a/docs/src/pages/docs/_assets/system-slider2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/tensor.png b/docs/src/pages/docs/_assets/tensor.png deleted file mode 100644 index 812640de3..000000000 Binary files a/docs/src/pages/docs/_assets/tensor.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/tensorrt-llm-01.png b/docs/src/pages/docs/_assets/tensorrt-llm-01.png deleted file mode 100644 index 2f839f7a5..000000000 Binary files a/docs/src/pages/docs/_assets/tensorrt-llm-01.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/tensorrt-llm-02.png b/docs/src/pages/docs/_assets/tensorrt-llm-02.png deleted file mode 100644 index de9841874..000000000 Binary files a/docs/src/pages/docs/_assets/tensorrt-llm-02.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/theme.png b/docs/src/pages/docs/_assets/theme.png deleted file mode 100644 index ae14f2054..000000000 Binary files a/docs/src/pages/docs/_assets/theme.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/thread-settings.png b/docs/src/pages/docs/_assets/thread-settings.png deleted file mode 100644 index a7cc60304..000000000 Binary files a/docs/src/pages/docs/_assets/thread-settings.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/thread-settings2.png b/docs/src/pages/docs/_assets/thread-settings2.png deleted file mode 100644 index b0d728d43..000000000 Binary files a/docs/src/pages/docs/_assets/thread-settings2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/tools.png b/docs/src/pages/docs/_assets/tools.png deleted file mode 100644 index 8104f6336..000000000 Binary files a/docs/src/pages/docs/_assets/tools.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/trouble-shooting-01.png b/docs/src/pages/docs/_assets/trouble-shooting-01.png index 093529e15..a12ea97b0 100644 Binary files a/docs/src/pages/docs/_assets/trouble-shooting-01.png and b/docs/src/pages/docs/_assets/trouble-shooting-01.png differ diff --git a/docs/src/pages/docs/_assets/turn-off.png b/docs/src/pages/docs/_assets/turn-off.png deleted file mode 100644 index a16a26a5b..000000000 Binary files a/docs/src/pages/docs/_assets/turn-off.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/ui.gif b/docs/src/pages/docs/_assets/ui.gif deleted file mode 100644 index 946ad8fb9..000000000 Binary files a/docs/src/pages/docs/_assets/ui.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/vulkan.png b/docs/src/pages/docs/_assets/vulkan.png deleted file mode 100644 index e8104bcc7..000000000 Binary files a/docs/src/pages/docs/_assets/vulkan.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/windows.png b/docs/src/pages/docs/_assets/windows.png deleted file mode 100644 index 69ef6f9a8..000000000 Binary files a/docs/src/pages/docs/_assets/windows.png and /dev/null differ diff --git a/docs/src/pages/docs/extensions-settings/_meta.json b/docs/src/pages/docs/extensions-settings/_meta.json index 2ff4e0274..100d720dd 100644 --- a/docs/src/pages/docs/extensions-settings/_meta.json +++ b/docs/src/pages/docs/extensions-settings/_meta.json @@ -1,8 +1,4 @@ { - "cortex": { - "title": "Cortex", - "href": "/docs/extensions-settings/cortex" - }, "model-management": { "title": "Model Management", "href": "/docs/extensions-settings/model-management" diff --git a/docs/src/pages/docs/extensions-settings/cortex.mdx b/docs/src/pages/docs/extensions-settings/cortex.mdx deleted file mode 100644 index 94043dab7..000000000 --- a/docs/src/pages/docs/extensions-settings/cortex.mdx +++ /dev/null @@ -1,120 +0,0 @@ ---- -title: Cortex -description: Learn about Jan's default extensions and explore how to configure them. -keywords: - [ - Jan, - Customizable Intelligence, LLM, - local AI, - privacy focus, - free and open source, - private and offline, - conversational AI, - no-subscription fee, - large language models, - Jan Extensions, - Extensions, - ] ---- - -import { Callout } from 'nextra/components' -import { Settings } from 'lucide-react' - -# Cortex - -Configure how [Cortex.cpp](/docs/extensions#cortexcpp) runs AI models. Access these settings through **Settings** () > **Core Extensions** > **Cortex**: - -## Performance Settings - -### Continuous Batching -- Enables processing multiple requests simultaneously -- Improves throughput for concurrent operations, which usually improves performance - -**Continuous Batching** is turned on by default: -
-![Continuous Batching](../_assets/extensions-04.png) -
- - -### Parallel Operations -- Number of prompts to run in parallel -- Affects model inference speed - -By default, **Parallel Operations** is 4: -
-![Parallel Operations](../_assets/extensions-05.png) -
- - -### CPU Threads -- Number of CPU cores to use when running without GPU -- Adjust based on your system's capabilities - - -Higher thread counts may improve performance but increase CPU usage. - - -Leave blank at **CPU Threads** to use system defaults: -
-![Parallel Operations](../_assets/extensions-06.png) -
- - -## Memory Settings - -### Flash Attention -- Optimizes attention computation -- Reduces memory usage -- Recommended for most cases - -**Flash Attention** is turned on by default: -
-![Parallel Operations](../_assets/extensions-07.png) -
- -### Caching -- Enable to store recent prompts and responses -- Improves response time for repeated prompts - -**Caching** is turned on by default: -
-![Caching](../_assets/extensions-08.png) -
- -### KV Cache Type -- KV cache implementation type; controls memory usage and precision trade-off. -- Options: - - f16 (most stable) - - q8_0 (balanced) - - q4_0 (lowest memory) - - -Changing KV cache type from default (f16) may affect model stability. - - -By default, **KV Cache Type** is f16: -
-![KV Cache Type](../_assets/extensions-09.png) -
- - -### MMAP -- Enables memory-mapped model loading -- Reduces memory usage -- Recommended for large models - -**MMAP** is turned on by default: -
-![MMAP](../_assets/extensions-10.png) -
- -> **Best Practices:** -> - Start with default settings -> - Adjust based on your hardware capabilities -> - Monitor system performance -> - Test changes with your specific use case - - -Performance impact varies by hardware, model size, and usage patterns. - - diff --git a/docs/src/pages/docs/extensions.mdx b/docs/src/pages/docs/extensions.mdx index a3c1b02e5..aa612a940 100644 --- a/docs/src/pages/docs/extensions.mdx +++ b/docs/src/pages/docs/extensions.mdx @@ -33,7 +33,7 @@ Extensions can be managed through **Settings** ( +## Performance Settings + + +| Setting | Description | Default | +|---------|-------------|---------| +| **Continuous Batching** | - Enables processing multiple requests simultaneously

- Improves throughput for concurrent operations | Enabled | +| **Parallel Operations** | - Number of prompts to run in parallel

- Affects model inference speed | 4 | +| **CPU Threads** | - Number of CPU cores to use when running without GPU

- Higher thread counts may improve performance but increase CPU usage | Auto-detected based on your system's capabilities | + + + +## Memory Settings + +| Setting | Description | Default | +|---------|-------------|---------| +| **Flash Attention** | - Optimizes attention computation

- Reduces memory usage

- Recommended for most cases | Enabled | +| **Caching** | - Enable to store recent prompts and responses

- Improves response time for repeated prompts | Enabled | +| **KV Cache Type** | - KV cache implementation type; controls memory usage and precision trade-off

- Options:

• f16 (most stable)

• q8_0 (balanced)

• q4_0 (lowest memory) | f16 | +| **MMAP** | - Enables memory-mapped model loading

- Reduces memory usage

- Recommended for large models | Enabled | + + +## Best Practices +- Start with default settings +- Adjust based on your hardware capabilities +- Monitor system performance +- Test changes with your specific use case + + +Performance impact varies by hardware, model size, and usage patterns. + + + diff --git a/docs/src/pages/docs/local-engines/tensorrt-llm.mdx b/docs/src/pages/docs/local-engines/tensorrt-llm.mdx deleted file mode 100644 index 6c46b9ea9..000000000 --- a/docs/src/pages/docs/local-engines/tensorrt-llm.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: TensorRT-LLM -description: A step-by-step guide on customizing the TensorRT-LLM engine. -keywords: - [ - Jan, - Customizable Intelligence, LLM, - local AI, - privacy focus, - free and open source, - private and offline, - conversational AI, - no-subscription fee, - large language models, - TensorRT-LLM Engine, - TensorRT, - tensorRT, - engine, - ] ---- - -import { Tabs } from 'nextra/components' -import { Callout, Steps } from 'nextra/components' -import { Settings, EllipsisVertical, Plus, FolderOpen, Pencil } from 'lucide-react' - -# TensorRT-LLM -## Overview -Jan uses **TensorRT-LLM** as an optional engine for faster inference on NVIDIA GPUs. This engine uses [Cortex-TensorRT-LLM](https://github.com/janhq/cortex.tensorrt-llm), which includes an efficient C++ server that executes the [TRT-LLM C++ runtime](https://nvidia.github.io/TensorRT-LLM/gpt_runtime.html) natively. It also includes features and performance improvements like OpenAI compatibility, tokenizer improvements, and queues. - - -TensorRT-LLM engine is only available for **Windows** users, **Linux** support is coming soon! - - - -## Requirements -- NVIDIA GPU with Compute Capability 7.0 or higher (RTX 20xx series and above) -- Minimum 8GB VRAM (16GB+ recommended for larger models) -- Updated NVIDIA drivers -- CUDA Toolkit 11.8 or newer - - -For detailed setup guide, please visit [Windows](/docs/desktop/windows#compatibility). - - -## Enable TensorRT-LLM - - -### Step 1: Install Additional Dependencies -1. Navigate to **Settings** () > **Local Engine** > **TensorRT-LLM**: -2. At **Additional Dependencies**, click **Install** - -
-![Click Tensor](../_assets/tensorrt-llm-01.png) -
- -3. Verify that files are correctly downloaded: -```bash -ls ~/jan/data/extensions/@janhq/tensorrt-llm-extension/dist/bin - -# Your Extension Folder should now include `cortex.exe`, among other artifacts needed to run TRT-LLM -``` -4. Restart Jan - -### Step 2: Download Compatible Models - -TensorRT-LLM can only run models in `TensorRT` format. These models, also known as "TensorRT Engines", are prebuilt specifically for each operating system and GPU architecture. - -We currently offer a selection of precompiled models optimized for NVIDIA Ampere and Ada GPUs that you can use right away: - -1. Go to **Hub** -2. Look for models with the `TensorRT-LLM` label & make sure they're within your hardware compatibility -3. Click **Download** - - -This download might take some time as TensorRT models are typically large files. - - -
-![Download TensorRT-LLM Model](../_assets/tensorrt-llm-02.png) -
- -### Step 3: Start Threads -Once the model(s) is downloaded, start using it in [Threads](/docs/threads) - - - -