Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?
arXiv Preprint Archive January 12, 2026 Lingchen Sun, Rongyuan Wu, Zhengqiang Zhang et al.
A method called SelfTranscendence accelerates diffusion transformer (DiT) training using only internal feature supervision, avoiding dependence on external pretrained models like DINO. The authors argue that DiTs can guide their own training if internal features are made structurally clean and semantically discriminative. They first align DiT features with clean VAE latent features for a short phase (e.g., 40 epochs), then apply classifier-free guidance to intermediate features. These enriched internal features then supervise a new DiT trained from scratch. The method improves generation quality and convergence speed over existing self-contained approaches and can surpass REPA, which uses external DINO features, on class-to-image and text-to-image tasks.