Joint breast neoplasm detection and subtyping using multi-resolution network trained on large-scale H&E whole slide images with weak labels

Breast cancer is the most commonly diagnosed cancer in the world. The use of artificial intelligence (AI) to help diagnose the disease from digital pathology images has the potential to greatly improve patient outcomes. However, methods for training these models for detecting, segmenting, and subtyping breast neoplasms and other proliferative lesions often rely on costly and time-consuming manual annotation, which can be infeasible for large-scale datasets. In this work, we propose a weakly supervised learning framework to jointly detect, segment, and subtype breast neoplasms. Our approach leverages top-k multiple instance learning to train an initial neoplasm detection backbone network from weakly-labeled whole slide images, which is then used to automatically generate pixel-level pseudo-labels for whole slides. A second network is trained using these pseudo-labels, and slide-level classification is performed by training an aggregator network that fuses the embeddings from both backbone networks. We trained and validated our framework on large-scale datasets with more than 125k whole slide images and demonstrate its effectiveness on tasks including breast neoplasms detection, segmentation, and subtyping.