STAT On PubPhys: open In the literature: partially resolved Compiled by AI, not yet reviewed by an editor

Sample complexity of gradient training for multi-index targets

42

No. 1,073 in all physics · No. 21 in Statistical and Nonlinear Physics

This text mentions work from 2024 on. Those parts were partly written without a live literature search and have not been checked by an editor yet.

Non-expert version

Inputs $x \sim N(0, I_d)$ and labels $y = g(W x)$ with $W$ an $r$ x d matrix of rank r fixed as $d \to \infty$ and $g$ a fixed link function. For a standard two-layer network trained by online or multi-pass SGD on the square loss without data preprocessing, determine the exponent $\gamma(g)$ in $n \sim d^\gamma$ required to reach test error $o(1)$, and determine whether it equals the benchmark for statistical-query and low-degree algorithms, $n \sim d^{\max(1, k_*/2)}$, with $k_*$ the generative exponent for single-index targets and the generative leap exponent for multi-index targets (Damian, Lee and Bruna, arXiv:2506.05500). The answer is $\gamma$ as a function of the Hermite structure of $g$, with proof.

What would settle it

Matching upper and lower bounds on the sample complexity of standard gradient training for general multi-index targets.

Status in the literature

Unverified note: it names recent work and was partly written without a live literature search; it awaits a check by an editor.

For single-index polynomial targets, SGD with minibatch reuse on two-layer networks reaches $n \sim d\,\operatorname{polylog} d$ (Lee, Oko, Suzuki and Wu, arXiv:2406.01581, 2024); correlational SGD on multi-index targets is governed by the leap complexity (Abbe, Boix-Adsera and Misiakiewicz 2023); the low-degree benchmark for multi-index targets is set by the generative leap (Damian, Lee and Bruna 2025); whether standard SGD reaches that benchmark for general multi-index $g$ is open (2026).

No solutions, ideas or subproblems yet.

How results are checked

See also

Comments (0)

Comments are not records: they are not signed, stamped or logged. How comments work

No comments yet.

Sign in to comment.

Details, signature and history
Kind
Well-posed question
Answer
Value
Topic review
L. Zdeborova and F. Krzakala, Statistical physics of inference: thresholds and algorithms, Advances in Physics 65, 2016. Link
Record
Revision 2 · notation repaired by the platform · compiled by AI, not yet reviewed by an editor
Batch approval
Approved for the batch by Victor Galitski (ORCID 0000-0001-5532-1096) · approval record · bundle This approves the batch, not each problem (what it means).
Earlier revisions
revision 1 bundle

Proof

Fingerprint
91fdc52a2137…
Public log
Entry 2,095, in checkpoint 2,123
In the log by
Block 969,698, 2026-10-03

This text was in the PubPhys log by that date. The date says nothing about when the problem was first posed.

History