Self-supervised segmentation, after the paper hype

For my final-year project I pretrained a ResNet50 with SimCLR and MoCo on a corpus of medical images, then fine-tuned it for segmentation against a fully supervised baseline. The self-supervised encoder won on the small labelled set, lost on the large one, and the crossover point was somewhere around 2,000 labelled images.

That result is consistent with the literature. What is less consistent is how I spent the four months before noticing the crossover. I tried multiple projection heads, multiple temperature schedules, and multiple augmentation pipelines, when the data was the binding constraint from the start.

If I were starting over today, the order would be different. I would measure the labelled set size first, then pick the pretraining strategy that fits it, not the other way around. I would also use a stronger backbone; ResNet50 in 2026 is a baseline, not a result. And I would have written the evaluation harness before the training loop, not after, so the crossover point would have shown up in week two instead of week twelve.