ECCV 2026

Test-Time Registers as Global Priors for Tokenized Image Generation

Cheng-Yao Hong1* Yifan Wang1* Yuewei Lin2 Chenyu You1

1 Stony Brook University

2 Brookhaven National Laboratory

* Equal contribution

Paper Coming Soon
Code

News

  • RegToken was accepted at ECCV 2026.
  • The paper is available on arXiv.

Overview

Turning register structure into a compact global prior

Vision transformers can develop attention sinks: a small set of tokens repeatedly receives attention and accumulates unusually large activations. In vision models, these outliers are associated with registers and carry smooth, scene-level information that is distinct from both [CLS] readouts and local patch summaries.

RegToken is a training-free procedure that converts this frozen register structure into a small set of global prior tokens. The priors are inserted into a compact 1D token generation pipeline to supply global context, without modifying pretrained backbone or decoder weights.

Method

A four-stage, frozen-model procedure

  1. 01

    NFN-based localization

    Normalized Feature Norms identify the layers and modules where register-related amplification is most prominent.

  2. 02

    TokenRank-guided extraction

    TokenRank-guided importance identifies register neurons and the corresponding register subspace.

  3. 03

    Projection and conservation

    A projection-and-conservation update constructs RegToken from the register subspace while approximately preserving its global statistics.

  4. 04

    Frozen-token insertion

    The resulting global prior tokens are inserted into a frozen compact 1D token generation pipeline.

RegToken overview: NFN and TokenRank identify register-related layers and neurons, then a register-derived global prior is passed to a tokenizer generator.
Method overview. RegToken localizes register structure in a frozen vision transformer, constructs a global prior, and inserts it into tokenized generation.

Feature-spectrum analysis

Register features emphasize low-frequency structure

Across DINOv2 and OpenCLIP, register-associated features show stronger low-frequency concentration than both [CLS] and patch-mean features. The analysis compares feature spectra, rather than treating these signals as direct pixel measurements.

The relationship to pixel-space DCT low-frequency energy is consistent but moderate, supporting an association with global image structure without claiming a perfect correspondence.

Feature spectra for register, patch-mean, and CLS features in OpenCLIP and DINOv2, separated into low and mid/high frequency bins.
Frequency analysis. Register-associated embeddings show stronger low-frequency concentration than the compared token summaries.

Analysis

Register dynamics

What happens after register insertion?

With a single register token inserted at NFN-selected layers using the top-50 register neurons, the register accumulates large norm and CLS-to-register attention mass. The norm and attention statistics of image tokens remain largely unchanged.

Six DINOv2 histograms comparing image-token and inserted-register norm values and CLS attention before insertion, after vanilla insertion, and after NFN-guided insertion.
Effect of register insertion on DINOv2 features. The inserted register concentrates the reported norm and attention mass while image-token statistics remain comparatively stable.

Outlier suppression and attention mixing

Each point pairs the fraction of patch tokens above the pre-95th norm threshold with λ₂, the second eigenvalue of the attention Markov chain (median over heads). Vanilla test-time registers show a modest shift toward fewer outliers and lower λ₂; the NFN-guided, head-gated variant shows a larger reduction in outliers and a clearer drop in λ₂, which the paper associates with stronger mixing.

Four OpenCLIP and DINOv2 scatter plots of lambda two, median over heads, against the fraction of patch tokens above the pre-95th norm threshold, comparing pre and post results for vanilla and NFN-guided methods.
Correlation between λ₂ and outlier tokens. The NFN-guided, head-gated variant shows the clearer reported shift in both quantities.

ImageNet results

Global priors improve quality, alignment, and efficiency

FID-5k

20.520.1

Lower is better

SigLIP

3.63.9

Higher is better

Steps@τ

7452

Lower is better

Qualitative results

Diverse decoding examples

RegToken qualitative example 1 RegToken qualitative example 2 RegToken qualitative example 3 RegToken qualitative example 4 RegToken qualitative example 5 RegToken qualitative example 6 RegToken qualitative example 7 RegToken qualitative example 8 RegToken qualitative example 9 RegToken qualitative example 10 RegToken qualitative example 11 RegToken qualitative example 12 RegToken qualitative example 13 RegToken qualitative example 14 RegToken qualitative example 15 RegToken qualitative example 16 RegToken qualitative example 17 RegToken qualitative example 18 RegToken qualitative example 19 RegToken qualitative example 20 RegToken qualitative example 21 RegToken qualitative example 22 RegToken qualitative example 23 RegToken qualitative example 24 RegToken qualitative example 25 RegToken qualitative example 26 RegToken qualitative example 27 RegToken qualitative example 28 RegToken qualitative example 29 RegToken qualitative example 30
Examples from the final paper’s qualitative results under the HCT protocol.

Citation

Provisional citation

@inproceedings{hong2026regtoken,
  title={Test-Time Registers as Global Priors for Tokenized Image Generation},
  author={Hong, Cheng-Yao and Wang, Yifan and Lin, Yuewei and You, Chenyu},
  booktitle={European Conference on Computer Vision},
  year={2026}
}