unslothai /

Unsloth

Live map
View repository
Snapshotmain · aa3f3c2

A fast fine-tuning path from a model ID to optimized training and portable weights.

Click a building or route
Choose a path

Follow a model ID and text examples into a patched model, token batches, and fused training operations.

System map8 components · 9 connections
130%
Unsloth architectureStart at the public model-loading API on the upper left. The middle buildings prepare model and token data, the GPU district performs training work, and the right edge publishes the result.model ID, revision, se…AutoConfig, model type…tokenizer files, speci…patched forward functi…input_ids, attention m…patched model, PEFT ad…activation tensors, gr…Fast model APIModel loaderModel patcherFused GPUkernelsTokenizerpipelineModel hubTraining loopModel exporter
Connections
BuildingsEntryServiceComputeStorageRuntimeToolingExternal

Terms, in plain English

TokenizerA reversible-ish mapping that turns text into integer token IDs a model can process.

How to study this repo

  1. 1
    Trace loading first

    Read models/loader.py from FastLanguageModel.from_pretrained until it selects an architecture-specific patcher.

  2. 2
    Compare one patched layer

    Match the assignments in models/llama.py to the fused forward/backward implementation in kernels/fast_lora.py.

  3. 3
    Complete the artifact lifecycle

    Follow one save_pretrained_merged or push_to_hub path and list every file needed to reload the result.