STAT On PubPhys: open In the literature: partially resolved Compiled by AI, not yet reviewed by an editor

Test error of interpolating two-layer networks in the feature-learning regime

52

No. 581 in all physics · No. 11 in Statistical and Nonlinear Physics

Non-expert version

Two-layer network $f(x) = \sum_{j=1..m} a_j \sigma(w_j . x)$ trained by gradient descent from small initialization to zero training loss on n samples $y = f_{*}(x) + \mathrm{noise}$ of variance $\sigma_n^2$, $x \sim \mathrm{N}(0, I_d)$ with $I_d$ the d x d identity, in the limit $n, d \to \infty$ with $\psi = n/d$ fixed and $m >> n$. For targets $f_{*}(x) = g(U x)$ with $U$ an r x d matrix and r fixed, compute the limiting test mean-squared error on noisy labels $E(\psi, \sigma_n^2, g)$ at fixed $\psi$, and classify the interpolation by the $\psi \to \infty$ limit of $E$: benign if $E \to \sigma_n^2$, tempered if $E \to c\,\sigma_n^2$ with $1 < c < \infty$, catastrophic if $E$ diverges. The answer is an asymptotic formula, proved or confirmed numerically to high precision.

What would settle it

An exact asymptotic formula for the test error of gradient-trained interpolating two-layer networks in the feature-learning regime, matched by large-scale simulation.

Status in the literature

Unverified note: it names recent work and was partly written without a live literature search; it awaits a check by an editor.

Exact answers exist for linear regression (Bartlett, Long, Lugosi and Tsigler 2020) and random-feature models (Mei and Montanari 2022), a benign, tempered, catastrophic taxonomy was proposed by Mallinar et al. 2022, and partial feature-learning results exist for classification and low-dimensional settings (Frei, Chatterji and Bartlett 2022; Kornowski, Yehudai and Shamir 2023; Joshi, Vardi and Srebro 2023); no asymptotic test-error formula for gradient-trained interpolating networks in the proportional regime exists (2026).

No solutions, ideas or subproblems yet.

How results are checked

See also

Comments (0)

Comments are not records: they are not signed, stamped or logged. How comments work

No comments yet.

Sign in to comment.

Details, signature and history
Kind
Well-posed question
Answer
Value
Posed
2016
Topic review
L. Zdeborova and F. Krzakala, Statistical physics of inference: thresholds and algorithms, Advances in Physics 65, 2016. Link
Record
Revision 2 · notation repaired by the platform · compiled by AI, not yet reviewed by an editor
Batch approval
Approved for the batch by Victor Galitski (ORCID 0000-0001-5532-1096) · approval record · bundle This approves the batch, not each problem (what it means).
Earlier revisions
revision 1 bundle

Proof

Fingerprint
ad88fe5a9c92…
Public log
Entry 2,090, in checkpoint 2,123
In the log by
Block 969,698, 2026-10-03

This text was in the PubPhys log by that date. The date says nothing about when the problem was first posed.

History