Learning Where to Steer Noise
Efficient offline multi-objective optimization with generative models
1
unmodified ODE solve per candidate, after a single noise displacement
5–24×
faster sampling with steering alone than the best prior generative method, PCD, on the timed tasks
2
new families of guidance operators, data-adaptive and Pareto-aware, that never differentiate through the sampler
8.13
average rank of our method over 47 tasks, the best of any generative method
Generative models are usually adapted for optimization by guiding or conditioning every step of the sampling process. We take a different approach, Active-Subspace Noise Steering (ASNS): we displace the initial noise once, along a small set of learned directions, and leave the sampler unchanged. Across the 47 tasks of Off-MOO-Bench4, this single displacement achieves a better average rank than the best prior generative method while sampling 5 to 24 times faster.
The animation below runs ASNS on a two-objective toy problem. Each point on the left is a noise vector. The flow maps it to a design on the right, plotted by its two objective values. Designs generated from steered noise are shown in green.
The flow turns noise into designs
Each noise vector z is carried by the flow Φθ to a design, plotted here by its two objective values. The designs land inside the offline data, short of the Pareto front.
Score the designs, learn one direction
The proxies score every design. Colouring each noise vector by the score of its design shows where in noise space the score rises. A Recursive Feature Machine learns that direction u from the pairs alone, without differentiating the flow.
Shift the noise, run the same flow
Every noise vector moves a distance γ along u. The unmodified flow decodes it in one pass, and the designs land past the offline data, toward the front.
Figure 1. Toy illustration, not benchmark data: two objectives and a 16-dimensional noise space, shown in a 2-D slice. The direction is learned in your browser by a small Recursive Feature Machine. The trade-off is fixed at equal weights here. The paper sweeps it, and a new trade-off costs no new flow evaluations.
01The setting: better than the data, without new data
In offline multi-objective optimization you get a fixed table of designs and their scores on several competing objectives: accuracy and latency for a neural architecture, speed and energy efficiency for a robot policy, cost and deflection for a welded beam. You may not evaluate anything new. Three terms come up throughout:
- Pareto front. The designs you cannot improve on one objective without losing on another. It is the set of best possible trade-offs.
- Hypervolume. The amount of objective space a set of designs covers. Bigger means closer to the front and better spread along it. It is the score every method is judged by.
- Proxies. Neural networks trained on the table to predict each objective, since the real objectives are off limits.
A flow model trained on the table generates designs that look valid. But a faithful model of the data is not biased toward better designs: its samples land back inside the data cloud. Something has to push them outward.
02Where the push goes in
Existing generative methods push during sampling. ParetoFlow2 and PGD-MOO3 add a guidance term at every integration step. PCD1 trains an objective-conditioned model and leans on classifier-free guidance. Either way, the push is paid for at every step of every sample, and guidance built on an estimate of the final design is least reliable early on, when the sample is still mostly noise.
The second option exists because the sampler is deterministic. Once the noise \(\mathbf{z}\) is drawn, the design \(\Phi _{\theta }(\mathbf{z})\) is fixed, so every objective is already a function of the noise. Pick a trade-off \(\mathbf{w}\), meaning how much you care about each objective, and you get one number per noise vector:
\[H_{\mathbf{w}}(\mathbf{z}) = \mathbf{w}^{\top } \hat{\mathbf f}(\Phi _{\theta }(\mathbf{z}))\]
Our approach. ASNS identifies, once per task, the few noise directions along which \(H_{\mathbf{w}}\) varies, and displaces every noise vector along them before sampling.
03The objective only cares about a few noise directions
Our key observation is that the noise-space objective is effectively low-rank. Although the noise space may have thousands of dimensions, \(H_{\mathbf{w}}\) is sensitive to only a few of them: averaged over the noise, a small number of directions account for nearly all of its sensitivity. This structure is what makes noise steering practical.
On the benchmark. The same concentration shows up on all 47 tasks of Off-MOO-Bench4. The engineering tasks use about 1 to 6 effective directions, the architecture-search tasks 9 to 27 out of up to 174, and the two policy tasks 65 and 100 out of roughly 10,000 (100 is the most the fit retains there). The share of noise space that matters shrinks as the dimension grows, down to about one percent on the policy tasks.
The likely reason is that real data sits near a low-dimensional surface, and the flow squeezes most noise directions away on the way there. The important directions therefore belong to the model and the proxies, not to any particular noise vector, so they can be found once and reused for every sample.
04Learning the directions from function values
We decode a batch of noise vectors, score the results with the proxies, and fit a Recursive Feature Machine5 to the pairs of noise and score, one per objective. An RFM is a kernel regressor that keeps re-estimating which input directions its own fit is sensitive to. We read the steering directions off that estimate. The flow is never differentiated, and the fit is forgiving: half the samples and about half the iterations stay within 1% of the default hypervolume on the tasks we swept.
One cache for every trade-off. The sensitivity pattern for any trade-off is a weighted sum of a few fixed matrices, one per pair of objectives. We store those once. A new trade-off costs one recombination and one eigendecomposition, with no refit and no new flow evaluations to find its directions.
The saving is large. On ZDT2, assembling a trade-off from the cache takes about a millisecond, against 0.56 seconds to refit from scratch, which is more than 500 times cheaper. The price is a few percent of hypervolume on average. At the balanced trade-off the cache matches a refit, and at the most lopsided one the refit leads by about 12%.
05The steering step, and why it goes uphill
\[\mathbf{z}_{\mathbf{w}} = \mathbf{z} + \gamma \sum _{j} \beta _{j} \sigma _{j} \mathbf{u}_{j}\]
Read it as: move the noise, with strength \(\gamma\), along the important directions \(\mathbf{u}_{j}\), leaning harder on the more important ones (\(\beta _{j}\)) and going the uphill way along each (\(\sigma _{j}\)). The uphill way comes for free from the cached scores: if noise vectors further along a direction scored higher, that way is up. Then run the unmodified flow, one ODE solve per candidate.
The step comes with a guarantee. Write \(\bar{d}_{j}\) for the average slope of \(H_{\mathbf{w}}\) along the \(j\)-th direction.
Theorem 1. Steering ascends the scalarized objective.
If the curvature of \(H_{\mathbf{w}}\) is bounded by \(L\), then on average over the noise
\[\mathbb{E}[H_{\mathbf{w}}(\mathbf{z}_{\mathbf{w}}) - H_{\mathbf{w}}(\mathbf{z})] \ge \gamma \langle \boldsymbol{\beta}, |\bar{\mathbf{d}}|\rangle - \tfrac12 L \gamma ^{2} \|\boldsymbol{\beta}\|^{2}\]
The bound is positive for every strength \(0 < \gamma < 2\gamma^{*}\) and largest at \(\gamma^{*} = \langle \boldsymbol{\beta}, |\bar{\mathbf{d}}|\rangle / (L \|\boldsymbol{\beta}\|^{2})\).
The gain grows in proportion to the step and the curvature penalty with its square, so a sufficiently small step always improves the objective.
Two further results say that these are the right directions, and that sweeping the trade-off recovers a front.
Proposition 1. The active subspace is optimal.
Among all subspaces of dimension \(r\), the span of the top \(r\) eigenvectors captures the largest expected squared gradient of \(H_{\mathbf{w}}\), equal to \(\lambda_1 + \dots + \lambda _{r}\). What it leaves out is exactly the remaining eigenvalues, \(\sum _{j > r} \lambda _{j}\).
Theorem 3. Front recovery.
If the attainable front curves away from its supporting hyperplanes and moves smoothly with the trade-off, sweeping the trade-off over a grid of weights with spacing \(\eta\) produces a front whose distance from the best front the flow can generate satisfies
\[d_{\mathrm{H}} \le \sqrt{2\varepsilon/\mu} + L_{\mathcal{P}}\,\eta, \qquad \varepsilon = \max_{\mathbf{w}} [\tau (\mathbf{w}) + \rho (\mathbf{w})] + 2\varepsilon _{\mathrm{proxy}}\]
Here \(\tau\) is what the dropped directions cost, \(\rho\) is what a single step fails to reach, \(\varepsilon _{\mathrm{proxy}}\) is the proxy error and \(\mu\) measures the curvature of the front and \(L_{\mathcal{P}}\) how fast it moves with the trade-off. With an exact proxy, full rank and a step that reaches its target, the distance goes to zero as the grid is refined.
The estimate itself is also controlled: the metric built from \(n\) decoded samples converges to that of the surrogates at rate \(n^{-1/2}\).
These guarantees are about the proxies, not the true objectives, and the front they refer to is the best the flow can produce, not the true Pareto front. Closing either gap would need new evaluations, which no offline method has.
06Guided steering: two new families of guidance operators
A single displacement stops improving when the mean gradient leaves the learned directions, or when the best designs lie beyond the range in which the flow decodes reliably. To recover part of that gap we add guidance late in the same trajectory, for \(t \ge t_{\mathrm{start}}\), where the model's estimate of the final design can be trusted. Standard guidance follows the summed proxy gradient as it is. We introduce two families of operators that use information it discards. Each guided step costs one proxy backpropagation, and none of the operators differentiates through the ODE solve.
Family 1Data-adaptive guidance
These operators restrict the gradient to directions that the objective responds to and that the data supports.
- RFM-APG, active-subspace projected guidance. The learned geometry is rebuilt once per task at \(t_{\mathrm{start}}\), on intermediate states instead of noise. The gradient is projected onto the resulting active subspace, so each step moves only along directions in which the remaining flow still changes the objective.
- RFM-DAMG, manifold-constrained guidance. The gradient is reweighted by the cached spectrum and then projected onto the local tangent space of the data manifold, estimated from neighbouring candidates in the same batch. Where no low-dimensional local structure can be certified, the projection is skipped, so the operator uses a manifold where one exists and otherwise reduces to spectrally weighted guidance.
We show that each of these projections and reweightings, applied on its own, yields a direction that does not decrease the scalarized proxy to first order.
What is new in RFM-DAMG
Manifold-constrained guidance exists, but the existing versions need something offline design problems do not provide. To our knowledge, RFM-DAMG is the first such operator that needs nothing beyond the batch being sampled, and the first to combine the data manifold with the geometry of the objective.
| Method | Where the manifold comes from | What it requires | Uses objective geometry |
|---|---|---|---|
| MPGD8 | A pretrained autoencoder, or the latent space of a latent diffusion model | A pretrained autoencoder or latent diffusion model | No |
| CFG++9 | No explicit tangent, only a modified classifier-free guidance update | A conditional model that supports classifier-free guidance | No |
| RFM-DAMG (ours) | Local PCA over neighbouring candidates in the same batch, with a rank test and a fallback | Nothing extra: no autoencoder, no conditional model, no retraining | Yes, the cached spectrum |
This matters in offline design, where the data are architectures, policies or engineering parameters and a pretrained autoencoder is rarely available. The step is constrained twice, by where the data lives and by where the objective responds.
Family 2Pareto-aware guidance: RFM-Cone
Standard guidance adds the objectives' gradients together before using them, and a step that raises the sum can still lower an individual objective. RFM-Cone keeps the per-objective gradients separate. It forms a preference direction that pulls hardest on the objectives lagging behind the requested trade-off, and projects it onto the cone of directions along which no objective decreases.
Theorem 2. Per-objective improvement.
The RFM-Cone step \(\mathbf{s}\) satisfies \(\langle \nabla \hat{f}_{a}, \mathbf{s}\rangle \ge 0\) for every objective \(a\). For a small enough step, every objective with a strictly positive inner product increases, and the others change only at second order. The cone contains a strictly ascending direction if and only if the current point is not Pareto-stationary. At a stationary point the step vanishes and sampling continues with the unmodified flow.
No guidance built on the summed gradient can give this guarantee.
What is new in RFM-Cone
Existing guidance for multi-objective generation either follows a requested trade-off or protects every objective. RFM-Cone is the only operator among them that does both, and the only sampling-time operator among them with a proof that covers the update it actually runs.
| Method | Follows a chosen trade-off | No objective sacrificed at a step | Extra model to train | Where it acts |
|---|---|---|---|---|
| MGDA6, the minimum-norm direction | No | Yes | None | An optimizer, not a sampler |
| ParetoFlow2 | Yes, a weighted sum | No | None | Every step |
| PGD-MOO3 | No | No | A pairwise dominance classifier | Every step |
| SPREAD7 | No | Proved only for a special case of its update | A conditional diffusion model | Every step |
| RFM-Cone (ours) | Yes | Yes, Theorem 2 | None | Late steps only |
The exact projection onto the cone, with the minimum-norm direction only as a fallback, is what lets it follow the trade-off as far as the objectives allow and no further. It needs no extra model, only one proxy backpropagation and an active-set projection per guided step.
Effect on results. All three operators improve on steering alone in overall rank: 8.93 without guidance, 8.49 with RFM-DAMG, 8.36 with RFM-APG and 8.13 with RFM-Cone. In the guidance ablation, they improved on steering alone in 65%, 72% and 78% of task and seed pairs respectively.
07Results
We ran 47 tasks in six families against 14 baselines: 11 surrogate methods, which search a learned model with an evolutionary algorithm, and three generative ones. Everything was rerun under one protocol. On each task every method is ranked by hypervolume, and ranks are averaged. Our variants carry the names used in the paper: RFM-AN is steering alone, and RFM-APG, RFM-DAMG and RFM-Cone add one guidance operator each.
| Average rank by task family | PCD | ParetoFlow | RFM-AN | RFM-Cone |
|---|---|---|---|---|
| Synthetic DTLZ (7 tasks) | 9.60 | 15.31 | 12.40 | 9.80 |
| Synthetic ZDT (5) | 11.08 | 14.12 | 11.56 | 9.48 |
| Architecture search, C-10 (9) | 7.64 | 10.60 | 5.96 | 5.41 |
| Architecture search, IN-1K (9) | 5.60 | 11.09 | 6.74 | 6.31 |
| Policies, MORL (2) | 8.50 | 12.70 | 8.10 | 8.20 |
| Engineering, RE (15) | 12.95 | 15.40 | 9.64 | 9.63 |
The best of the four shown in each family is highlighted. Surrogate methods, not shown here, lead on DTLZ, ZDT, MORL and RE.
Steering is what helps. ParetoFlow+ is ParetoFlow's per-step guidance running on our flow and our proxies. Swapping that guidance for one noise shift moves the overall rank from 13.56 to 8.93. Steering alone also outranks PCD1, the best previous generative method, at 9.64. With guidance, RFM-Cone reaches 8.13, the best of any generative method and second among all 18, just behind the best surrogate method at 8.07.
The wins are bigger than the losses, with one exception. Every other loss is under 9%, while five wins are above 11%. The exception is RE31, where our variants lose about 40% and vary widely across seeds, which points to a collapse on some seeds.
And it is cheap. Steering alone is about four times faster than ParetoFlow and 5 to 24 times faster than PCD. RFM-Cone gets its better rank at about a third of PCD's average sampling time. The largest gap is on MO-Hopper, a 10,184-dimensional policy task, where steering takes 6.3 s against 149 s for PCD.
08How far to steer
Because steering only moves the noise, its length says directly how far the steered noise leaves the region the flow was trained on. The right length depends on where the front sits. On ZDT4 the front is already within reach, and a long step costs about 15% of hypervolume. On the architecture-search tasks the gain rises and then levels off. On the engineering task RE36 it is still rising at the longest step we tried, at 1.22 times the offline data. A short step never hurt on any of the five tasks in that sweep.
09What to take away
If you already have a flow or diffusion model of your designs, you may not need to guide it at every step or retrain it with conditioning. Its noise space has a few directions that control the objectives, they are cheap to find, and one shift along them does most of the work.
10Where this could go
ASNS needs a pretrained generative model with a deterministic sampler, a continuous design representation and a proxy for each objective. Nothing else ties it to the benchmark.
- AI for science. Diffusion models of molecules, proteins, crystals and polymers already exist, and the goals are always in tension: binding affinity against toxicity, catalytic activity against stability, battery capacity against cost. ASNS could turn such a model into a multi-objective designer with one precomputation and no retraining. The same holds for inverse design in optics and metamaterials, and for scientific simulators where a learned emulator provides the proxy.
- Engineering and hardware design. Airfoils, circuits and accelerator configurations, wherever a simulator is too slow to search and a proxy exists. For the discrete choices in these problems, our Divide and Learn10 is the natural companion.
- Policies and controllers. The method already runs on 10,000-dimensional policy weights. Generative models of controllers in robotics are a direct next target.
- Any pretrained generator, any score. A reward model or a constraint violation can stand in for the proxy, which would give a training-free way to bias image, audio or structure generators toward several targets at once.
- Open ends. Stochastic samplers, discrete design spaces and non-convex fronts, where a Tchebycheff scalarization would extend the recovery guarantee.
11References
- 1Jatan Shrestha, Santeri Heiskanen, Kari Hepola, Severi Rissanen, Pekka Jääskeläinen and Joni Pajarinen. Pareto-Conditioned Diffusion Models for Offline Multi-Objective Optimization. ICLR 2026. arxiv.org/abs/2602.00737
- 2Ye Yuan, Can Chen, Christopher Pal and Xue Liu. ParetoFlow: Guided Flows in Multi-Objective Optimization. ICLR 2025. arxiv.org/abs/2412.03718
- 3Yashas Annadani, Syrine Belakaria, Stefano Ermon, Stefan Bauer and Barbara E. Engelhardt. Preference-Guided Diffusion for Multi-Objective Offline Optimization. NeurIPS 2025. arxiv.org/abs/2503.17299
- 4Ke Xue, Rong-Xi Tan, Xiaobin Huang and Chao Qian. Offline Multi-Objective Optimization. ICML 2024. arxiv.org/abs/2406.03722
- 5Adityanarayanan Radhakrishnan, Daniel Beaglehole, Parthe Pandit and Mikhail Belkin. Mechanism for feature learning in neural networks and backpropagation-free machine learning models. Science, 2024. www.science.org/doi/10.1126/science.adi5639
- 6Jean-Antoine Désidéri. Multiple-gradient descent algorithm (MGDA) for multiobjective optimization. Comptes Rendus Mathématique, 2012. doi.org/10.1016/j.crma.2012.03.014
- 7Sedjro Salomon Hotegni and Sebastian Peitz. SPREAD: Sampling-based Pareto front Refinement via Efficient Adaptive Diffusion. ICLR 2026. arxiv.org/abs/2509.21058
- 8Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J. Zico Kolter, Ruslan Salakhutdinov and Stefano Ermon. Manifold Preserving Guided Diffusion. ICLR 2024. arxiv.org/abs/2311.16424
- 9Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam and Jong Chul Ye. CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models. ICLR 2025. arxiv.org/abs/2406.08070
- 10Esha Singh, Dongxia Wu, Chien-Yi Yang, Tajana Rosing, Rose Yu and Yi-An Ma. Divide and Learn: Multi-Objective Combinatorial Optimization at Scale. In Proceedings of the 43rd International Conference on Machine Learning (ICML), PMLR, 2026.
12Paper, code & BibTeX
Paper (arXiv) · PDF · Code (GitHub)
The proofs, the full per-task tables and the ablations are in the paper, and the code reproduces the paper's results. If you use this work, please cite:
@misc{lu2026learningwheresteer, title={Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models}, author={Yuan Lu and Esha Singh and Yi-An Ma and Yusu Wang}, year={2026}, eprint={2609.38920}, archivePrefix={arXiv}, primaryClass={cs.LG}, url={https://arxiv.org/abs/2609.38920}, }