PyTorch Concepts — Interactive Demo

A hands-on simulator to visualise Tensors, Embeddings (RAG), and Neural Network training loops.

Simulated in Javascript • Offline Ready • No Python Backend Required

The Core Elements of PyTorch

PyTorch translates complex mathematical concepts into clean, object-oriented code. Understanding how data travels through tensors, how gradients are automatically tracked, and how modules package parameters is the secret to mastering deep learning.

How Data Flows

The diagram below shows the typical path of data in a PyTorch program: Raw numbers are wrapped in Tensors, operations are recorded by Autograd to compute derivatives, and parameters are grouped inside a Module to define trainable networks.

TENSOR Multi-dimensional array with GPU support
torch.tensor([[1.0, 2.0]])
↓
AUTOGRAD Tracks ops to build computation graph
requires_grad=True
↓
nn.MODULE Encapsulates model weights and layers
class Net(nn.Module)

1. Tensors: The Raw Material

Tensors are the fundamental data structure in PyTorch. They represent vectors, matrices, or higher-dimensional grids. Unlike standard arrays, they can be processed on GPUs to accelerate training.

2. Autograd: The Mathematical Engine

Autograd handles automatic differentiation. By recording all mathematical steps performed on tensors, it can automatically compute derivatives (gradients) using the chain rule during the backward pass.

3. nn.Module: The Structured blueprint

The nn.Module base class allows you to define neural network layers, track trainable parameters (weights/biases), and write the forward pass structure in a structured, clean manner.

Vectors & Matrices

Tensors store weights and inputs. Operations like matrix multiplication form the basis of neural calculations.

Chain Rule Tracking

Autograd acts like a tape recorder, measuring how changes in weights affect model prediction errors.

Trainable Layers

Modules combine active functions, state weights, and custom logic to produce the final model architecture.

1. Document Store

Edit these documents to see how text changes alter their vector representation and retrieval properties.

2. Semantic Retrieval

Type a query to embed it and find the most semantically similar documents in the database using Cosine Similarity.

Query Embedding Vector (8-Dimensions)

Ranked Search Results (Retrieval Stage)

What is RAG (Retrieval-Augmented Generation)?

  • Embedding Models convert chunks of text into multi-dimensional coordinates (vectors).
  • Vector Databases (like Pinecone or pgvector) store these coordinates for rapid retrieval.
  • Cosine Similarity calculates the angle between vector coordinates; smaller angles indicate similar semantic meaning.
  • Retrieving the top matches and feeding them to an LLM context is the core of RAG pipeline designs.

3. Spatial Map

2D Circular Radar Projection of the 8D Vector Space

This radar projection plots documents (numbered dots) and your query (gold star) based on their embedding coordinates. Lines connect the query to the top 3 matching documents.

4-Step Training Loop: Forward → Loss → Backward → Step

W1 Matrix (Hidden Size: 6, Input Size: 1)
W2 Matrix (Output Size: 1, Hidden Size: 6)
Current Epoch / Step
0
Loss Value (MSE)
0.00000

Weights and biases update live on each step. Positive parameters show as teal values, negative parameters show as dim values.

Model Predictions vs. Target

Solid red line = Neural Net predictions; Dashed line = Target function

Training Loss History

Tracks MSE drop as the optimizer adjusts parameters

Understanding the training stages:

  • FORWARD: The model computes inputs through layers: x → Linear + Tanh → Linear → Prediction.
  • LOSS: Evaluates accuracy using MSE loss (Prediction - Target)².
  • BACKWARD: Calculates gradients (partial derivatives) of loss w.r.t weights using manual matrix chain-rule backpropagation.
  • STEP: Moves parameter weights down the gradient slope (Stochastic Gradient Descent): weights -= lr * gradients.