Classification of Sow Postures Using Convolutional Neural Network and Depth images

Abstract

The US swine industry reports an average preweaning mortality of approximately 16% where approximately 6% of them are attributed to piglets overlayed by sows. Detecting postural transitions and estimating sows’ time budgets for different postures are valuable information for breeders and engineering design of farrowing facilities to eventually reduce piglet death. Computer vision tools can help monitor changes in animal posture accurately and efficiently. To create a more robust system and eliminate varying lighting issues within a day including daytime/ nighttime differences, there is an advantage to using depth cameras over digital cameras. In this study, a computer vision system was used for continuous depth image acquisition in several farrowing crates. The images were captured by top down view Kinect v2 depth sensors in the crates at 10 frames per minute for 24 h. The captured depth images were converted into Jet colormap images. A total of 14277 images from six different sows from 18 different days were randomly selected and labeled into six posture categories (standing, kneeling, sitting, sternal lying, lying on the right and lying on the left). The Convolutional Neural Network (CNN) architectures, that is, Resnet-50, Inception v3 with ‘imagenet’ pre-trained weight, were used for model training and posture images were tested. The dataset was randomly split training (75%) and validation (roughly 25%) sets. For testing, another dataset with 2885 images obtained from six different sows (from 12 different days) was labelled. Among the models tested in the test dataset, the Inception v3 model outperformed all the models, resulting in 95% accuracy in predicting sow postures. We found an F1 score between 0.90 and 1.00 for all postures except the kneeling posture (F1=0.81) since this is a transition posture. This preliminary result indicates the potential use of transfer learning models for this specific task. This result also indicates that depth images are suitable for identifying the postures of sows. The outcome of this study will lead to the identification and generation of posture data in a commercial farm scale to study the behavioral differences of sows within different characteristics of farm facilities, health status, mortality rates, and overall production parameters.