Paper Title
A MULTI-SCALE ATTENTION-BASED MASK R-CNN FOR CHROMOSOME INSTANCE SEGMENTATION

Abstract
Automated karyotyping plays a crucial role in cytogenetic research by streamlining chromosome analysis through deep learning-based segmentation and classification. A key challenge in this process is chromosome segmentation, as errors in segmentation directly affect classification accuracy. However, overlapping chromosomes, small chromosome structures, and low-contrast regions make automated segmentation difficult. To address these challenges,we propose an enhanced Mask R-CNN model with a Multi-Scale Fusion Module and an Attention-Based Feature Pyramid Network to improve segmentation accuracy, especially for overlapping and small chromosomes. Additionally, we incorporate gradient anomaly detection and mask refinement techniques to enhance boundary precision. Our model is trained and evaluated on a custom dataset with 24 chromosome classes, achieving a mean Average Precision (mAP) of 0.609 at IoU=0.50:0.95. Experimental results demonstrate that our approach effectively improves chromosome segmentation performance compared to the baseline Mask R-CNN, particularly in challenging cases. The proposed framework offers a robust solution for automated karyotyping, with potential applications in medical and cytogenetic research Keywords - Karyotyping, Chromosome Segmentation, Mask R-CNN, Multi-Scale Fusion, Attention-Based Feature Pyramid Network, Gradient Anomaly Detection, Mask Refinement, Instance Segmentation, Cytogenetic Analysis