Exploring the possibility of extracting cancer morphology from deep feature clusters

Our research is aimed at an improved understanding of mammographic abnormality classification deep learning models and how these might be related to abnormality morphology. We generated clusters of deep learned features generated by a multi-view deep learning model classifying breast cancer subtypes. This model was constructed from two ResNet50 blocks, supplemented with concatenation layers to merge the outputs of the blocks. The modelling was based on the Optimam dataset, using 2193 cases (543 DCIS and 1649 IDC samples) with supporting meta-data. We reduced the features to two dimensions using dimensional reduction techniques to facilitate visualization and evaluation in a 2-dimensional plot. Our chosen methods for dimensional reduction were Principal Component Analysis (PCA) for linear reduction and Uniform Manifold Approximation and Projection (UMAP), a non-linear manifold learning method. To identify potential trends, we adopted two analytical approaches. Firstly, we examined existing metadata to identify global or local trends within our data, and we observed that overlaying metadata describing lesions did only reveal limited discernible trends in the data (when using lesion type or abnormality classification). Secondly, we employed handcrafted features such as density and lesion area and GLCM texture features including Dissimilarity and Homogeneity to be represented as heat maps, which indicated clear patterns in the data. Clusters using heat maps, display trends within the data showing that lesions of similar characteristics are positioned locally. Additional meta data and expert evaluation is required to draw full conclusions, and future work includes investigating if the low dimensional deep learned representation is locally linked to morphological aspects of the abnormalities.