Integrating deep neural networks into high-stakes medical applications remains limited by concerns about robustness and reliability. In particular, models may rely on spurious correlations, such as the frequent co-occurrence of medical devices with pathology, rather than clinically meaningful features. Addressing this problem requires both detecting dataset-specific spurious signals and reducing model reliance on them. We propose a two-stage approach: a fast few-shot method for estimating spurious labels using only a small number of expert-annotated samples, followed by a lightweight framework that combines these estimated labels with available metadata using a novel supervised contrastive loss. Although designed for medical imaging, the proposed approach also generalizes beyond this domain. Our method achieves the strongest spurious-mitigation performance, demonstrating that combining reliable spurious-label estimates with available metadata can substantially improve robustness.
M.Sc. student under the supervision of Prof. Guy Gilboa and Dr. Eyal Gofer.
