NoiseShift: Resolution-Aware Noise Recalibration for Better Low-Resolution Image Generation
News Release Summary
Diffusion image models are usually trained for a particular range of image sizes, and simply sampling them at a much smaller resolution can make images deteriorate even though the computation is cheaper. NoiseShift traces this issue to a mismatch between the scheduled noise level and how corrupted an image actually appears at a lower resolution. Rather than retraining a model or changing its sampling schedule, the method recalibrates the noise-conditioning signal with a small coarse-to-fine procedure. On Stable Diffusion 3, Stable Diffusion 3.5, and Flux-Dev, the authors report consistent gains at low resolution without added inference overhead—for example, improving SD3's 128-by-128 FID from 203 to 171 on LAION-COCO.
abstract
Text-to-image diffusion models often degrade when sampled at resolutions outside the final training resolution set. Prior work has largely emphasized higher resolution generation, enabling pretrained diffusion models to extrapolate beyond the resolutions seen during training. In this work, we instead target lower-resolution generation, performing inference at reduced resolution to significantly cut computational cost. We show that network conditioning of the noise level induces a train-test mismatch that directly degrades low-resolution generation: the same scheduled noise level can correspond to a different perceptual corruption level at lower resolutions, mis-calibrating the denoiser timestep and noise embedding. To this end, we propose NoiseShift, a training-free recalibration method that keeps the original noise sampling schedule unchanged and instead re-indexes the noise conditioning of the denoiser to restore local forward-reverse consistency. Using a lightweight coarse-to-fine calibration on a small set of image-text pairs, NoiseShift learns a resolution-specific mapping from scheduler noise to conditioning noise, reducing train-test mismatch and improving lower-resolution generation quality. When NoiseShift is applied to Stable Diffusion 3 (SD3), Stable Diffusion 3.5 (SD3.5), and Flux-Dev, generation quality at low resolutions improves consistently. Particularly, SD3 generation at 128x128 resolution gets an improved FID score from 203 to 171, and SD3.5 gets an improved FID score from 310 to 277 on LAION-COCO. Even Flux-Dev which already implements a complementary time-shifting strategy gets a modest boost from NoiseShift with an improved FID score from 120 to 113 at 64x64 resolution. More importantly, NoiseShift achieves such improvements with minimal implementation changes and no additional inference overhead.
citation
@article{henoiseshift,
title = {NoiseShift: Resolution-Aware Noise Recalibration for Better Low-Resolution Image Generation},
author = {He, Ruozhen and Haji-Ali, Moayed and Yang, Ziyan and Ordonez, Vicente},
journal = {arXiv preprint arXiv:2510.02307},
url = {https://arxiv.org/abs/2510.02307},
}
automatically generated questions, main contributions and limitations of this paper
Questions this paper helps answer
- What problem does NoiseShift solve? It improves text-to-image diffusion sampling at resolutions lower than those used in final training, where quality can otherwise degrade sharply.
- Why does low-resolution sampling hurt quality? The paper argues that a scheduled noise level corresponds to a different perceptual corruption level at lower resolution, which mis-calibrates the denoiser's timestep and noise embedding.
- Does NoiseShift retrain the diffusion model? No. It is a training-free method that keeps the original sampling schedule and re-indexes the denoiser's noise conditioning.
- What models were tested? The paper evaluates Stable Diffusion 3, Stable Diffusion 3.5, and Flux-Dev and reports consistent low-resolution improvements.
Main contributions
- NoiseShift identifies a resolution-dependent conditioning mismatch as a source of low-resolution diffusion-image degradation.
- The method learns a resolution-specific mapping from scheduler noise to conditioning noise with a lightweight calibration set.
- It preserves the original noise schedule and adds no reported inference overhead.
- Reported results improve FID for SD3, SD3.5, and Flux-Dev at low resolutions, including SD3 from 203 to 171 at 128 by 128 on LAION-COCO.
Limitations and cautions
- The evaluation centers on low-resolution generation for three diffusion families, leaving room to test the recalibration on additional architectures and image domains.
- NoiseShift uses a small image-text calibration set, so future work could characterize how calibration-set composition affects robustness across prompts and resolutions.
- The paper reports image-quality metrics and qualitative behavior; broader product evaluations could measure perceived quality, latency, and downstream task value together.
- The method is deliberately narrow in scope, but its lack of retraining and reported absence of inference overhead make it a useful practical intervention for low-resolution deployment.
How to read this result
NoiseShift is best read as a targeted, efficient correction for a concrete low-resolution diffusion mismatch, supported by consistent results across several widely used model families.