LUA: Latent Upscaling Adapter for Diffusion-Based Image Synthesis

Abstract

Generating high-resolution images with latent diffusion models is limited by the cost of high-resolution denoising, while post-hoc super-resolution introduces artifacts and additional latency by operating after decoding. We present the Latent Upscaling Adapter (LUA), a lightweight module that performs super-resolution directly on the generator's latent code before the final VAE decoding step. LUA integrates as a drop-in component requiring no modifications to the base model or additional diffusion stages, enabling high-resolution synthesis through a single feed-forward pass in latent space.

A shared Swin-style backbone with scale-specific pixel-shuffle heads supports ×2 and ×4 factors with nearly 3× lower decoding and upscaling time. Because LUA operates directly on latents, it transfers across diffusion systems — including SDXL, SD3, and FLUX — through architecture-level reuse, adapting only the input convolution with brief fine-tuning rather than retraining from scratch. Extensive experiments demonstrate that LUA closely matches the fidelity of native high-resolution generation at 2K and 4K while offering a practical and efficient path to scalable image synthesis.

Method

One feed-forward pass in latent space

LUA sits between the generator and the VAE decoder: it takes the low-resolution latent, upsamples it ×2 or ×4, and a single decode produces the final image. No extra diffusion stages, no base-model changes.

Animated LUA pipeline: FLUX generates a 16-channel 128×128 latent, LUA upscales it ×4 to 512×512, and one VAE decode produces a 4096×4096 image in 40.5 seconds. A measured 2K comparison shows 37.0 seconds with LUA versus 103.6 seconds for direct FLUX generation on the same RTX 6000 Ada GPU. — **From text to 4K.** One latent upscale, one VAE decode — no second diffusion stage.

Static LUA overview: a diffusion model produces a 64×64 latent; LUA upscales it ×2 or ×4 before a single VAE decode produces the higher-resolution image. — **From text to 4K.** One latent upscale, one VAE decode — no second diffusion stage.

Three-stage training curriculum on FLUX: latent feature maps (top) and 8× decoded crops (bottom) after stages I, II, and III. — **Three-stage curriculum.** Stage I aligns latent structure and frequency spectra; Stage II couples latent fidelity to decoded appearance with high-frequency emphasis; Stage III refines edges in pixel space, without re-diffusion. Top: latent feature maps; bottom: 8× decoded crops.

Results

Native-level fidelity at a fraction of the cost

The best single-pass fidelity and the lowest latency among compared methods at 2K and 4K.

SDXL · 2048²

7.23 s → 3.52 s

direct → with LUA
GPU: H100

FLUX · 2048²

103.6 s → 37.0 s

direct → with LUA
GPU: RTX 6000 Ada

SDXL / FLUX · 4096²

6.87 / 31.9 s

with LUA
GPU: H100

User study

81.9%

preferred over SDXL direct
blind 2AFC, 20 annotators

Qualitative comparison at 2048² and 4096² from identical 1024² SDXL bases, with 12× crops; SDXL+LUA produces clean textures at the lowest latency. — 2048² and 4096² results from identical 1024² SDXL bases; red boxes: 12× crops, headers: per-image runtime. **SDXL + LUA**: clean textures at the lowest latency.

Resolution	Method	FID ↓	pFID ↓	KID ↓	pKID ↓	CLIP ↑	Time (s) ↓
2048×2048	HiDiffusion	200.72	114.30	0.0030	0.0090	0.738	4.97
	DemoFusion	184.79	177.67	0.0030	0.0100	0.750	28.99
	LSRNA–DemoFusion	181.24	98.09	0.0019	0.0066	0.762	20.77
	SDXL (Direct)	202.87	116.57	0.0030	0.0086	0.741	7.23
	SDXL + SwinIR	183.16	100.09	0.0020	0.0077	0.757	6.29
	SDXL + LUA (ours)	180.80	97.90	0.0018	0.0065	0.764	3.52
4096×4096	HiDiffusion	233.65	95.95	0.0158	0.0214	0.698	122.62
	DemoFusion	185.36	177.89	0.0043	0.0113	0.749	225.77
	LSRNA–DemoFusion	177.95	62.07	0.0023	0.0071	0.757	91.64
	SDXL (Direct)	280.42	101.89	0.0396	0.0175	0.663	148.71
	SDXL + SwinIR	183.15	65.71	0.0018	0.0103	0.756	7.29
	SDXL + LUA (ours)	176.90	61.80	0.0015	0.0152	0.759	6.87

OpenImages evaluation with an SDXL base at 1024². Median per-image runtime on H100, batch 1. Best per column in bold; lower is better except CLIP. The 1024² block, cross-model, and ablation tables are in the paper.

Pareto frontier of speed versus quality at 2048²–4096²; LUA sits on the favorable speed–quality frontier. — **Speed–quality Pareto frontier** at 2048²–4096². LUA reaches near-native fidelity at a fraction of the runtime of multi-stage methods.

Gallery

From the release benchmark

FLUX + LUA ×2 at 2048² on an RTX 6000 Ada — ~37 s per image vs ~104 s direct.

Citation

Citing LUA

arXiv: 2511.10629 · to appear at ECCV 2026.

@inproceedings{razin2026lua,
  title     = {{LUA}: Latent Upscaling Adapter for Diffusion-Based Image Synthesis},
  author    = {Razin, Aleksandr and Kazantsev, Danil and Makarov, Ilya},
  booktitle = {Computer Vision -- ECCV 2026},
  series    = {Lecture Notes in Computer Science},
  volume    = {17008},
  publisher = {Springer},
  year      = {2026}
}

@article{razin2025lua_arxiv,
  title   = {{LUA}: Latent Upscaling Adapter for Diffusion-Based Image Synthesis},
  author  = {Razin, Aleksandr and Kazantsev, Danil and Makarov, Ilya},
  journal = {arXiv preprint arXiv:2511.10629},
  year    = {2025}
}

LUA: Latent Upscaling Adapter for Diffusion-Based Image Synthesis

Why latent, not pixels?

One lightweight adapter, three properties

Single-stage latent upscaling

Multi-scale, shared backbone

Cross-model generalization

One feed-forward pass in latent space

One adapter, many models

Native-level fidelity at a fraction of the cost

Drag to compare

From the release benchmark

Beyond photographs

Citing LUA