Test error of interpolating two-layer networks in the feature-learning regime
52No. 581 in all physics · No. 11 in Statistical and Nonlinear Physics
Networks with far more adjustable weights than training examples can fit noisy data exactly and still predict new data well, against the classical expectation of overfitting (memorizing noise). The task is to compute the error on new data for networks whose internal features adapt during training.
Status in the literature
Unverified note: it names recent work and was partly written without a live literature search; it awaits a check by an editor.
Exact answers exist for linear regression (Bartlett, Long, Lugosi and Tsigler 2020) and random-feature models (Mei and Montanari 2022), a benign, tempered, catastrophic taxonomy was proposed by Mallinar et al. 2022, and partial feature-learning results exist for classification and low-dimensional settings (Frei, Chatterji and Bartlett 2022; Kornowski, Yehudai and Shamir 2023; Joshi, Vardi and Srebro 2023); no asymptotic test-error formula for gradient-trained interpolating networks in the proportional regime exists (2026).
No solutions, ideas or subproblems yet.
See also
- Related Predicting neural scaling-law exponents from properties of the data
- Related Sample complexity of gradient training for multi-index targets
Details, signature and history
- Kind
- Well-posed question
- Answer
- Value
- Posed
- 2016
- Topic review
- L. Zdeborova and F. Krzakala, Statistical physics of inference: thresholds and algorithms, Advances in Physics 65, 2016. Link
- Record
- Revision 2 · notation repaired by the platform · compiled by AI, not yet reviewed by an editor
- Batch approval
- Approved for the batch by Victor Galitski (ORCID 0000-0001-5532-1096) · approval record · bundle This approves the batch, not each problem (what it means).
- Earlier revisions
- revision 1 bundle
Proof
- Fingerprint
- ad88fe5a9c92…
- Public log
- Entry 2,090, in checkpoint 2,123
- In the log by
- Block 969,698, 2026-10-03
This text was in the PubPhys log by that date. The date says nothing about when the problem was first posed.
Comments (0)
Comments are not records: they are not signed, stamped or logged. How comments work
No comments yet.
Sign in to comment.