Smarter Health for Everyone with an AI System That Detects Four Diseases and Gives Easy-to-Understand Clinical Advice

Abstract

Medical-image artificial intelligence models are commonly developed for single diseases or isolated imaging modalities, which limits shared representation learning, calibration-aware prediction, and clinically usable decision-support reporting. This study proposes PolyDx-Mamba, a unified multi-disease medical image classification framework designed strictly for clinician-assisted decision support rather than autonomous diagnosis. The framework combines modality-specific stem and patch embedding, a shared Vision-Mamba hierarchical encoder, per-stage channel-spatial attention, cross-disease selective state-space global context aggregation, gated multi-scale feature fusion, uncertainty-aware latent representation learning, and task-specific classification heads. PolyDx-Mamba was evaluated across four clinically distinct imaging tasks: brain tumor MRI classification, skin lesion classification using dermoscopy, COVID-19 chest X-ray classification, and diabetic retinopathy grading from fundus images. A confidence-conditioned Gemini-1.5-Pro retrieval-augmented generation advisor was further incorporated to generate clinician-facing recommendation reports grounded in retrieved clinical context. PolyDx-Mamba achieved accuracies of 0.9926, 0.9209, 0.9880, and 0.8718 across the four datasets, with an average accuracy of 0.9433. Macro-F1 scores were 0.9917, 0.9151, 0.9863, and 0.8597, yielding an average macro-F1 of 0.9382. The model also demonstrated favorable calibration, with expected calibration error values of 0.0162, 0.0307, 0.0208, and 0.0421 across the respective tasks. Statistical testing confirmed significant performance improvements over all evaluated baselines, including the strongest Mamba-based comparator. The Gemini-1.5-Pro confidence-RAG advisor produced clinically relevant and guideline-aligned recommendation reports, with high factual accuracy, low hallucination rate, and strong safety compliance. These findings suggest that PolyDx-Mamba provides a calibrated and statistically supported multi-disease image classification framework with retrieval-grounded clinical advisory support. However, all predictions and generated recommendations require clinician confirmation, and external prospective validation remains necessary before any clinical deployment.