Sample complexity of gradient training for multi-index targets
42No. 1,073 in all physics · No. 21 in Statistical and Nonlinear Physics
This text mentions work from 2024 on. Those parts were partly written without a live literature search and have not been checked by an editor yet.
A network trained by stochastic gradient descent (repeated small weight updates using random batches of data) must find a few hidden directions in a high-dimensional input. The question is how many training examples it needs as the input dimension grows, compared with the minimum any fast method needs.
Status in the literature
Unverified note: it names recent work and was partly written without a live literature search; it awaits a check by an editor.
For single-index polynomial targets, SGD with minibatch reuse on two-layer networks reaches $n \sim d\,\operatorname{polylog} d$ (Lee, Oko, Suzuki and Wu, arXiv:2406.01581, 2024); correlational SGD on multi-index targets is governed by the leap complexity (Abbe, Boix-Adsera and Misiakiewicz 2023); the low-degree benchmark for multi-index targets is set by the generative leap (Damian, Lee and Bruna 2025); whether standard SGD reaches that benchmark for general multi-index $g$ is open (2026).
No solutions, ideas or subproblems yet.
See also
- Related Test error of interpolating two-layer networks in the feature-learning regime
- Related A correct low-degree criterion for polynomial-time hardness of inference
Details, signature and history
- Kind
- Well-posed question
- Answer
- Value
- Topic review
- L. Zdeborova and F. Krzakala, Statistical physics of inference: thresholds and algorithms, Advances in Physics 65, 2016. Link
- Record
- Revision 2 · notation repaired by the platform · compiled by AI, not yet reviewed by an editor
- Batch approval
- Approved for the batch by Victor Galitski (ORCID 0000-0001-5532-1096) · approval record · bundle This approves the batch, not each problem (what it means).
- Earlier revisions
- revision 1 bundle
Proof
- Fingerprint
- 91fdc52a2137…
- Public log
- Entry 2,095, in checkpoint 2,123
- In the log by
- Block 969,698, 2026-10-03
This text was in the PubPhys log by that date. The date says nothing about when the problem was first posed.
Comments (0)
Comments are not records: they are not signed, stamped or logged. How comments work
No comments yet.
Sign in to comment.