Abstract:Multiple-object tracking of mice is a fundamental task in behavioral analysis and an important method for studying social behavior. In response to the limitations of traditional mouse tracking methods, such as the ability to track only a single mouse and the need for mouse labeling to track multiple-object mice, which affects mouse behavior, an unlabeled multiple-object mice tracking method was proposed based on the combination of instance segmentation network YOLO v8n-seg and improved Strongsort. RGB cameras were used to capture daily behavior videos of multiple-object mice, and a dataset for segmenting mouse body parts was annotated. After augmenting the dataset, the YOLO v8n-seg instance segmentation network was trained. The model achieved a precision of 97.7%, recall of 98.2%, mAP50 of 99.2%, and single-image detection time of 3.5ms. It accurately and quickly segmented mouse body parts, meeting the detection requirements of the Strongsort multi-object tracking algorithm. To address tracking errors in the Strongsort algorithm for multiple-object mice tracking, two improvements were made. Firstly, the matching process was improved by re-matching trajectories that did not match objects and unmatched objects based on Euclidean distance. Secondly, the Kalman filter was improved by replacing the rectangular bounding box representing the mouse position and motion state in the Kalman filter with a square box centered on the centroid of the mouse body contour and with a diagonal equal to the mouse body width. After testing, the improved Strongsort algorithm showed an ID switches of 14, MOTA of 97.698%, IDF1 of 85.435%, and MOTP of 75.858%. Compared with the original Strongsort, the ID switches count was reduced by 88%, MOTA was improved by 3.266 percentage points, and IDF1 was improved by 27.778 percentage points. Compared with Deepsort, ByteTrack, and Ocsort, there was a significant improvement in MOTA and IDF1, and the ID switches was greatly reduced. These results indicated that the improved Strongsort algorithm can enhance the stability and accuracy of unlabeled multiple-object mouse tracking, providing a technical approach for analyzing social behavior in mice.