ggml-org /

llama.cpp

Live map
View repository
Snapshotmaster · 10bf611

From an OpenAI-compatible request to local tensor kernels and streamed text.

Click a building or route
Choose a path

Resolve the selected model, map its weights, and create the inference context.

System map8 components · 10 connections
130%
llama.cpp inference city architectureStart at the API Gate, follow the generation loop clockwise, and notice that model loading is a separate startup path.model path + load opti…model metadata + tenso…GGUF ModelLoaderCLI & API GateInferenceContextTokenizerRequest SlotsModel GraphBackendSchedulerCPU / GPUKernels
Connections
BuildingsEntryServiceComputeStorageRuntimeToolingExternal

Terms, in plain English

GGUFA file format that stores model metadata, tokenizer data, and weight tensors for efficient inference.

How to study this repo

  1. 1
    Trace the public API

    Start in include/llama.h, then find each API implementation in src to connect concepts to concrete code.

  2. 2
    Follow one decode

    Read the server slot loop into llama_decode, graph creation, backend scheduling, and logit extraction.

  3. 3
    Compare backends

    Inspect the CPU and one GPU backend to see how the same GGML operation receives different kernels.