---
title: llama.cpp
slug: /guides/providers/llama-cpp
---

<head>
    <title>llama.cpp - Jan Guides</title>
    <meta name="description" content="Learn about llama.cpp, the inference server used by Nitro, the default AI engine downloaded with Jan. Understand how Nitro provides an OpenAI-compatible API, queue, & scaling."/>
    <meta name="keywords" content="Jan AI, Jan, ChatGPT alternative, llama.cpp, Nitro, inference server, OpenAI-compatible API, queue, scaling"/>
    <meta property="og:title" content="llama.cpp - Jan Guides"/>
    <meta property="og:description" content="Learn about llama.cpp, the inference server used by Nitro, the default AI engine downloaded with Jan. Understand how Nitro provides an OpenAI-compatible API, queue, & scaling."/>
    <meta property="og:url" content="https://jan.ai/guides/providers/llama-cpp"/>
    <meta name="twitter:card" content="summary"/>
    <meta name="twitter:title" content="llama.cpp - Jan Guides"/>
    <meta name="twitter:description" content="Learn about llama.cpp, the inference server used by Nitro, the default AI engine downloaded with Jan. Understand how Nitro provides an OpenAI-compatible API, queue, & scaling."/>
</head>

## Overview

[Nitro](https://github.com/janhq/nitro) is an inference server on top of [llama.cpp](https://github.com/ggerganov/llama.cpp). It provides an OpenAI-compatible API, queue, & scaling.

Nitro is the default AI engine downloaded with Jan. There is no additional setup needed.
