VeritasRL is a clean-room, zero-mock reference implementation of the four systems pillars behind modern Frontier Reasoning Post-Training and Distributed Scaling (DeepSeek-R1 / DAPO, OpenAI o-Series Process Reward Models, ICML 2024 Oral DoRA, and NVIDIA Megatron-LM + Causal Ring-Attention):
GRPO + DAPO + Schulman $k_3$ KL): Eliminates the value/critic network entirely (50% value-model VRAM saved), scoring $G$ parallel policy rollouts per prompt with a non-hackable deterministic symbolic execution verifier, normalizing group advantages in native C++20, and optimizing a causal Transformer policy via DAPO asymmetric clipping ($[1-\varepsilon_{\text{low}}, 1+\varepsilon_{\text{high}}] = [0.80, 1.28]$) and Schulman’s exact non-negative $k_3$ KL estimator ($e^{\log(\pi_{\text{ref}}/\pi_\theta)} - \log(\pi_{\text{ref}}/\pi_\theta) - 1 \ge 0$).DoRA, Liu et al., ICML 2024 Oral): Decouples fine-tuning updates into a learnable row-magnitude vector $m \in \mathbb{R}^{d_{\text{out}}}$ and normalized directional low-rank matrices $W_0 + \frac{\alpha}{r} B A$, overcoming LoRA’s Eckart-Young rank floor while folding into a zero-overhead nn.Linear at serving time (max |Δ| = 0.00000000).Causal Online-Softmax Ring-Attention + Megatron-LM 2D MLP): Proves exact forward and PyTorch autograd.backward gradient equivalence (< 1.5e-6 max |Δ|) between $P$-rank Ring-Attention (circulating $(K_p, V_p)$ shards via associative online log-sum-exp state $(O, m, \ell)$ without materializing the $S \times S$ attention matrix) and $P$-rank Column-Parallel $\to \operatorname{GeLU} \to$ Row-Parallel All-Reduce(SUM) MLPs.PRM) & Test-Time Compute Scaling (qwen3.5:4b): Evaluates live local qwen3.5:4b via Ollama across 8 multi-step Olympiad Combinatorics, Chinese Remainder, Diophantine Optimization, and Number Theory problems—verifying intermediate lhs = rhs AST equations (+ - * / %) and public problem invariants to lift accuracy from 62.50% (5/8 Pass@1) to 100.00% (8/8 PRM Best-of-6, +37.50% lift).
flowchart LR
subgraph CppKernel["Native C++20 Kernel Layer (csrc/veritas_kernels.cpp)"]
K1["Group-Relative Z-Score\nAdvantage Normalization"]
K2["Fused Backward GAE-λ +\nSchulman Non-Negative k3 KL"]
K3["Associative Online-Softmax\n(O, m, l) Ring Tile Merger"]
end
subgraph PostTrain["Pillar 1 & 2: RLVR Post-Training & DoRA"]
PT1["Cold-Start Atomic Primitives\n-> Frozen Reference π_ref"] --> PT2["G=12 Rollouts -> Symbolic\nExecution Verifier R ∈ [0, 1]"]
PT2 --> PT3["DAPO Asymmetric Clip [0.80, 1.28]\n+ DoRA Magnitude/Direction"]
end
subgraph Dist2D["Pillar 3: 2D Sequence + Tensor Parallelism"]
DP1["Causal Ring-Attention\nO(S²/P) Per-Rank Memory"] --> DP2["Megatron Column -> GeLU -> Row\nSingle All-Reduce Autograd"]
end
subgraph TTC["Pillar 4: Test-Time Compute Scaling (Ollama)"]
T1["Live Qwen-3.5-4B\nN=6 Parallel CoT Rollouts"] --> T2["AST Step-Equation PRM +\nPublic Invariant Consensus"]
end
K1 & K2 --> PT3
K3 --> DP1
PT3 --> TTC
Every figure below is produced by running the real PyTorch, clang++ -O3 -std=c++20, and Ollama pipelines in this repository:
| Subsystem | Benchmark Command | Measured Result | Verified Invariant |
|---|---|---|---|
GRPO / DAPO Verifiable RL (seed=17, G=12, 32 epochs) |
uv run veritas train-grpo |
6.25% → 50.00% Pass@1 (8.0x lift, +43.75% gain) |
50% critic VRAM saved; $k_3 \text{ KL} = 0.86972 \ge 0$; C++20 vs NumPy < 1e-5 |
DoRA vs. LoRA Adaptation (128×128, r=8, 70 steps) |
uv run veritas benchmark-dora |
DoRA MSE 0.00323 vs. LoRA 1.81892 (563x lower) |
-86.72% trainable params (16,384 -> 2,176); folded nn.Linear max \|Δ\| = 0.00000000 |
Causal Ring-Attention (P=4, S=64) |
uv run veritas verify-distributed |
Forward \|Δ\| = 3.0e-7 · Autograd $\nabla Q,\nabla K$ \|Δ\| = 8.3e-7 |
4.0x peak attention-matrix reduction; Hypothesis verified vs PyTorch SDPA |
Megatron 2D TP MLP (P=4, d_ff=128) |
uv run veritas verify-distributed |
Forward \|Δ\| = 1.43e-6 · Autograd $\nabla X,\nabla W_1$ \|Δ\| = 1.19e-6 |
Zero activation-stage IPC; single forward All-Reduce(SUM) |
Live qwen3.5:4b Step-Level PRM Scaling (N=6) |
uv run veritas eval-ttc --live-only |
62.50% (5/8) Pass@1 → 100.00% (8/8) PRM Best-of-6 |
+37.50% Olympiad lift recorded in benchmarks/real_qwen_ttc.json |
98% Test Coverage)Requires Python 3.12+ (3.13 recommended), uv, and a C++20 compiler (clang++ or g++; automatically falls back to exact NumPy if unavailable):
uv sync --python 3.13 --all-extras
# Run Ruff lint, Mypy --strict, and Pytest + Hypothesis property suite (>=95% gate; 98% achieved)
uv run ruff check src tests
uv run mypy src tests
uv run pytest --cov=veritas_rl --cov-report=term-missing --cov-fail-under=95
# 1. Critic-free GRPO/DAPO post-training with C++20 group advantages
uv run veritas train-grpo
# 2. DoRA vs. LoRA convergence & zero-latency weight folding proof
uv run veritas benchmark-dora
# 3. Causal Online-Softmax Ring-Attention & Megatron 2D TP forward + backward autograd proof
uv run veritas verify-distributed
# 4. Step-level PRM Test-Time Compute scaling (replays benchmarks/real_qwen_ttc.json if Ollama is offline, or pass --live-only)
uv run veritas eval-ttc
cd ui && npm install && npm run build && cd ..
uv run veritas serve --host 127.0.0.1 --port 8040
# Open http://127.0.0.1:8040
veritas-rl/
├── csrc/
│ └── veritas_kernels.cpp # C++20 (-O3) group advantages, GAE-λ + Schulman k3 KL & online-softmax
├── src/veritas_rl/
│ ├── kernels/native_bridge.py # SHA-256 cached clang++/g++ compiler & ctypes bridge + NumPy fallback
│ ├── rlvr/grpo_trainer.py # Critic-free GRPO + DAPO trainer & non-hackable symbolic verifier
│ ├── adapters/dora.py # ICML 2024 Oral DoRA vs LoRA & zero-overhead merge_to_linear()
│ ├── distributed/parallel_ops.py # Causal Online-Softmax Ring-Attention & Megatron 2D TP autograd proofs
│ ├── ttc/prm_ollama.py # AST step-equation PRM & live Ollama Best-of-N benchmark
│ ├── api/app.py # Typed FastAPI control plane + SPA static serving
│ └── cli.py # Rich & JSON CLI (`veritas`)
├── ui/ # React 18 + TypeScript + Vite Frontier RLVR Workbench
├── benchmarks/real_qwen_ttc.json # Recorded live qwen3.5:4b Pass@1 vs PRM Best-of-6 telemetry
├── docs/
│ ├── mathematical-proofs.md # Formal derivations of GRPO/DAPO, k3 KL >= 0, DoRA & Ring-Attention
│ └── adr/ # Architecture Decision Records (0001..0004)
└── tests/ # Unit + Hypothesis mathematical property suite (98% coverage)
Apache License 2.0 — Copyright (c) 2026 Nagarjuna Kamireddy.
Built by Nagarjuna Reddy Kamireddy (Nagarjuna Kamireddy / Nagarjuna K) — Lead AI Architect, GenAI & Agentic AI Lead, and Corporate AI Trainer (20+ years across Enterprise Data, SAP HANA & Production AI; AWS Certified Solutions Architect – Professional & NVIDIA-Certified Professional: Agentic AI).
llms-full.txt · LinkedIn