DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models

TL;DR

DiverseVAR makes text-conditioned visual autoregressive models produce different outputs for the same prompt, with no retraining and little extra compute. Noise on the text embedding creates the variation, and scale-travel repairs the image quality that the noise costs.

Abstract

We introduce DiverseVAR, a test-time framework that enhances the output diversity of text-conditioned visual autoregressive models (VAR) without additional training or substantial computational overhead. While VAR models have recently emerged as strong competitors of diffusion and flow models for image and video generation, they suffer from a critical diversity limitation: even simple prompts often produce nearly identical outputs.

We address this limitation in two stages. First, we systematically explore diversity enhancement techniques by injecting noise into different components of VAR at test time, finding that noise injection into the text embedding yields the best diversity gains. However, this comes at the cost of sharp degradation in image quality. To recover quality, we propose scale-travel: a latent refinement technique that leverages a multi-scale autoencoder to extract coarse-scale tokens and resume generation from intermediate stages.

Extensive experiments demonstrate that combining text-embedding noise injection with scale-travel refinement substantially improves diversity while minimizing quality degradation, advancing the diversity-quality Pareto front.

2-Stage Pipeline

DiverseVAR runs in two stages. The first weakens the text condition early in generation so that different seeds stop converging on the same image. The second sends the resulting sample back through the multi-scale representation that VAR already uses, and regenerates its finer scales.

1

Condition Annealing

Start with weak conditioning and gradually strengthen it across scales.

Where the noise goes: (a) CFG scheduling mixes the conditional and unconditional logits, (b) condition annealing perturbs the <SOS> token or the text embedding.

2

Scale-Travel

Travel back to an earlier stage and regenerate to refine visual artifacts.

(+) Refines visual artifacts with a slight diversity drop. (−) Adds only 20% inference overhead.

Inference time

Seconds per image on an RTX 3090, batch size 1, averaged over 100 samples.

Method	Time	Slower
Infinity	2.115s	–
+ Cond. Anneal.	2.266s	×1.07
+ Scale-Travel	2.638s	×1.24
Switti	1.642s	–
+ Cond. Anneal.	1.692s	×1.03
+ Scale-Travel	1.892s	×1.15

Results

We evaluate DiverseVAR on Infinity and Switti for text-to-image generation and on InfinityStar for text-to-video generation, using MJHQ-30K and MS-COCO prompts. More samples are on the qualitatives page.

Quantitative comparisons

Text-to-image on MJHQ-30K and MS-COCO. FID and ImageReward (IR) measure quality, MPD_L and Vendi measure per-prompt diversity.

Text-to-video on InfinityStar-8B, generated from VBench-2.0 prompts and scored with VBench. Condition annealing gains more diversity (+132.2%) but loses quality. Scale-travel raises diversity from 0.270 to 0.487 (+80.4%) and gives up 0.002 points of quality.

Citation

@inproceedings{park2026diversevar,
  title     = {DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models},
  author    = {Park, Mingue and Phunyaphibarn, Prin and Lee, Phillip Y. and Sung, Minhyuk},
  booktitle = {European Conference on Computer Vision},
  year      = {2026}
}