Summary Points
- Synthetic data, real-world potential: Procedurally generated SET-card images provided accurate labels for boxes, corners, and masks—without manual annotation.
- Frozen DINOv3 backbone: A lightweight ConvNeXt Tiny model learned task-specific heads using synthetic images only, then tested on real phone photos.
- Strong zero-shot results: Card detection achieved 0.974 F1 on real images; corners and segmentation also transferred well, while rotated boxes lagged at 0.803 F1.
- Promising, but not finished: Background confusion, overlapping cards, and task-specific quirks remain. Next up: handle occlusion and recognize card features to find SETs automatically.
Synthetic Data, Known Labels
Collecting and labeling images takes time. Synthetic data offers another route: Build the images yourself, and you know exactly where each object sits.
For this experiment, the objects were SET playing cards. The goal was to locate cards on a table, not yet identify winning combinations. A procedural generator drew cards, then added textured surfaces, shadows, lighting changes and perspective. It also produced labels automatically, including boxes, corner points and segmentation masks.
Frozen Features, Trainable Heads
The model used a small ConvNeXt Tiny backbone distilled from DINOv3. Its features stayed frozen while a feature pyramid and task-specific heads learned from synthetic images. In other words, the model reused broad visual knowledge but practiced the card task only on generated scenes.
After training on 2,000 synthetic images, the models faced 40 phone photos from 11 real-world scenes. The card detector scored an F1 of 0.974. Corner prediction reached 0.951, semantic segmentation scored 0.911, and oriented boxes reached 0.803.
Promising Transfer, Real Limits
Those results suggest that rich pretrained features can help synthetic data transfer to real images. That could reduce the need for costly, task-specific labeling, especially when teams can control how they generate training examples.
Still, the test set was small and qualitative, so the scores do not prove broad reliability. The models sometimes mistook background objects for cards. Rotated boxes struggled with skew, corner predictions sometimes swapped points, and segmentation could merge touching cards. The experiment also left out overlapping cards and card recognition.
For adoption, the lesson is practical: Synthetic training can offer a strong starting point, not a universal substitute for real-world testing. Larger, more varied benchmarks and careful checks for missed objects and false detections should come next.
Stay Ahead with the Latest Tech Trends
Stay informed on the revolutionary breakthroughs in Quantum Computing research.
Stay inspired by the vast knowledge available on Wikipedia.
AITechV1
