Training Text-to-Image Models: Key Takeaways from Ablations

Training Design for Text-to-Image Models: Lessons from Ablations

The Photoroom team at Hugging Face recently published a blog post detailing their experiments with training efficient text-to-image models from scratch. They introduced their goal of training a competitive text-to-image foundation model entirely from scratch, in the open, and at scale. The team focused on architectural choices and motivated the core design decisions behind their model, PRX. In this article, we will summarize the key takeaways from their ablation studies.

Why This Matters

Training text-to-image models is a complex task that requires careful consideration of various factors, including architecture, training objectives, and data quality. The ideal model would be able to generate high-quality images that are consistent with the input text, while also being efficient and scalable. However, in practice, there are many challenges to overcome, such as mode collapse, unstable training, and poor image quality. The Photoroom team’s work provides valuable insights into the importance of representation alignment, token routing, and data quality in achieving good performance.

Key Insights

• REPA (Yu et al., 2024): Representation alignment with a pre-trained visual encoder can significantly improve convergence and quality metrics.
• JiT (Li and He, 2025): Predicting clean images instead of noise or velocity can make the learning problem easier and improve training efficiency.
• TREAD (Krause et al., 2025): Token routing can reduce compute costs and improve training efficiency, especially at high resolutions.

Working Example

# Example code for REPA alignment
import torch
import torch.nn as nn

class REPA(nn.Module):
    def __init__(self, teacher_encoder, student_encoder):
        super(REPA, self).__init__()
        self.teacher_encoder = teacher_encoder
        self.student_encoder = student_encoder

    def forward(self, x):
        # Get teacher embeddings
        teacher_embeddings = self.teacher_encoder(x)

        # Get student hidden tokens
        student_hidden_tokens = self.student_encoder(x)

        # Compute alignment loss
        alignment_loss = torch.mean((teacher_embeddings - student_hidden_tokens) ** 2)

        return alignment_loss

Practical Applications

• Use Case: Use REPA alignment to improve the quality of generated images in a text-to-image model.
• Pitfall: Be careful when using token routing, as it can lead to a loss in quality if not implemented correctly.

References:

• https://huggingface.co/blog/Photoroom/prx-part2
• https://arxiv.org/abs/2410.06940
• https://arxiv.org/abs/2506.05350
• https://arxiv.org/abs/2510.21986

---