pytorch /

PyTorch

Live map
View repository
Snapshotmain · 8f988c9

How tensors reach device operators, record gradients, compile graphs, and synchronize distributed training.

Click a building or route
Choose a path

Follow a Python tensor operation through dispatch, autograd recording, the device kernel, backward scheduling, and optional gradient reduction.

System map8 components · 9 connections
130%
PyTorch architectureThe tensor API enters dispatch in the center. The eager/autograd route bends toward the engine and distributed reducer; the compilation route goes through Dynamo and Inductor before both paths meet device operators.operator schema, tenso…dispatch-key set, diff…tensor storage, sizes,…root gradient edges, o…parameter index, gradi…flattened gradient ten…Tensor APIc10 dispatcherATendevice opsAutograd graphTorchDynamoAutogradengineTorchInductorDistributedreducer
Connections
BuildingsEntryServiceComputeStorageRuntimeToolingExternal

Terms, in plain English

Dispatch keyA tag derived from tensor state that selects layers such as Autograd and backends such as CPU or CUDA.

How to study this repo

  1. 1
    Inspect dispatcher choices

    Choose one operator and use PyTorch dispatch inspection tools, then connect its key order to Dispatcher.cpp and its ATen kernel.

  2. 2
    Draw a backward graph

    Print grad_fn and next_functions for a tiny expression, then map those nodes to function.h and engine.cpp.

  3. 3
    Explain one compiled graph

    Use torch.compile diagnostics on a small function and identify its Dynamo guards, FX operations, Inductor output, and graph breaks.