Multi-Phase Dual-Encoder Model for Anomaly Detection in Medical Imaging

Abstract

An error in medical diagnosis is enough to change a life, and many are suffering from incorrect diagnoses. Deep learning models in healthcare face several challenges due to limited and imbalanced datasets, where anomaly samples often dominated in publicly available data and it is problematic to collect independent data due to strict privacy regulations. Furthermore, current models often perform poorly on small sample datasets and lack robustness across various types of medical imaging. To address these issues, we developed a novel model that capable of detecting anomalies in medical imaging across both small and large datasets. This model features a multiple phase of training and validation, with dual encoders and a shared decoder architecture. Our results demonstrate that this model outperforms established classification methods in medical imaging datasets, including BraTS2021, RESC, and BreastMNIST, by achieving superior accuracy in distinguishing normal and anomalous images based on reconstruction error metrics. Furthermore, the research explores the interpretability of latent space features using explanation methods along with visualization techniques. By automating the diagnostic process, our model aligns with Malaysia’s Healthcare Government Plan 2023 to reform the health system over the next 15 years and reduces workload for healthcare professionals. Our model can serve as a foundation for developing reliable diagnostic tools with interpretable latent space to understand the model’s decision.