This document states and proves the exact mathematical invariants enforced in veritas-rl across code, C++20 kernels (csrc/veritas_kernels.cpp), and randomized Hypothesis property tests (tests/).
Standard Proximal Policy Optimization (PPO) maintains a separate critic network $V_\psi(s_t)$ of equal parameter count to the policy $\pi_\theta$, doubling activation and optimizer-state memory. Group Relative Policy Optimization (GRPO; Shao et al., DeepSeekMath / DeepSeek-R1) eliminates the critic by sampling a group of $G$ trajectories ${o_1, \dots, o_G} \sim \pi_{\theta_{\text{old}}}(\cdot \mid q)$ for each prompt $q$ and scoring each trajectory with a deterministic symbolic verifier $r_i = R_{\text{sym}}(q, o_i) \in [0, 1]$:
\[\hat{A}_i = \frac{r_i - \mu_G}{\sqrt{\sigma_G^2 + \varepsilon}}, \qquad \mu_G = \frac{1}{G}\sum_{j=1}^{G} r_j, \qquad \sigma_G^2 = \frac{1}{G}\sum_{j=1}^{G} (r_j - \mu_G)^2\]Proposition 1 (Zero-Mean Group Baseline). For every prompt group of size $G \ge 2$ and $\varepsilon > 0$, $\sum_{i=1}^G \hat{A}_i = 0$. Consequently, constant prompt-difficulty offsets cancel analytically without a learned value baseline.
Following DAPO (Yu et al., 2025), veritas-rl decouples the lower and upper clipping thresholds ($\varepsilon_{\text{low}} = 0.20 < \varepsilon_{\text{high}} = 0.28$) on the token likelihood ratio $\rho_{i,t}(\theta) = \frac{\pi_\theta(o_{i,t} \mid q, o_{i,<t})}{\pi_{\theta_{\text{old}}}(o_{i,t} \mid q, o_{i,<t})}$ to prevent entropy collapse on low-probability exploratory reasoning tokens:
Naive Monte-Carlo KL estimation $k_1 = \log \pi_\theta(a \mid s) - \log \pi_{\text{ref}}(a \mid s)$ takes negative values on individual tokens and exhibits high variance. veritas-rl implements Schulman’s $k_3$ estimator in both C++20 and PyTorch:
Theorem 1 (Pointwise Non-Negativity & Unbiasedness).
Given frozen pretrained weight $W_0 \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$, standard rank-$r$ LoRA parameterizes $W_{\text{LoRA}} = W_0 + \frac{\alpha}{r} B A$. Suppose downstream post-training rescales output neuron magnitudes via $W^\star = \operatorname{diag}(c)\, W_0$ with distinct entries $c \neq \mathbf{1}$. Since $( \operatorname{diag}(c) - I ) W_0$ generically has full rank $\min(d_{\text{out}}, d_{\text{in}})$, the Eckart-Young-Mirsky theorem lower-bounds LoRA’s Frobenius error by $\sum_{k=r+1}^{d} \sigma_k^2((\operatorname{diag}(c)-I)W_0) > 0$.
DoRA (Liu et al., ICML 2024 Oral) decouples magnitude $m \in \mathbb{R}^{d_{\text{out}}}$ from directional low-rank updates:
\[W_{\text{DoRA}} = \operatorname{diag}(m)\, \frac{W_0 + \frac{\alpha}{r} B A}{\|W_0 + \frac{\alpha}{r} B A\|_{\text{row}, 2}}\]nn.Linear weight tensor (max |Δ| = 0.00000000 in float32).parallel_ops.py & csrc/veritas_kernels.cpp)Partition sequence length $S$ across $P$ ring ranks ($S_{\text{local}} = S / P$). As $(K_p, V_p)$ blocks circulate around the ring, each rank maintains running state $(O^{(p)}, m^{(p)}, \ell^{(p)})$ updated via:
\(m_{\text{new}} = \max(m_{\text{old}}, \max_j S_{:,j}), \qquad \ell_{\text{new}} = e^{m_{\text{old}} - m_{\text{new}}}\ell_{\text{old}} + \sum_j e^{S_{:,j} - m_{\text{new}}}\) \(O_{\text{new}} = e^{m_{\text{old}} - m_{\text{new}}} O_{\text{old}} + e^{S - m_{\text{new}}} V_p\)
Telescoping $e^{m^{(p-1)} - m^{(P)}}$ across $p = 1, \dots, P$ cancels all intermediate maxima $m^{(p)}$, leaving $O^{(P)} / \ell^{(P)} = \operatorname{Softmax}(Q_r K^\top / \sqrt{d} + M_{\text{causal}}) V$ with $O(S^2 / P)$ per-rank memory instead of $O(S^2)$.
MegatronMLP2D)Partition $W_1 \in \mathbb{R}^{d_{\text{ff}} \times d_{\text{model}}}$ row-wise ($W_1 = [W_1^{(1)}; \dots; W_1^{(P)}]$, Column-Parallel in $X W_1^\top$) and $W_2 \in \mathbb{R}^{d_{\text{model}} \times d_{\text{ff}}}$ column-wise ($W_2 = [W_2^{(1)}, \dots, W_2^{(P)}]$, Row-Parallel in $H W_2^\top$). Because elementwise $\operatorname{GeLU}$ commutes with block concatenation:
\[\sum_{p=1}^P \operatorname{GeLU}\!\left(X W_1^{(p)\top}\right) W_2^{(p)\top} = \operatorname{GeLU}(X W_1^\top) W_2^\top\]requiring zero communication before the nonlinearity and exactly one All-Reduce(SUM) in the forward pass (and its dual identity/all-reduce in autograd.backward).