Pineapple fruit detection and size determination in a juicing factory in the Eastern Cape, South Africa

Abstract

This research presents a deep learning approach to determine pineapple size from images, to identify the instances of pineapples, and subsequently to extract fruit dimensions. This was achieved by first detecting pineapples in each image using a Mask region-based convolutional neural network (Mask R-CNN), and then extracting the pixel diameter and length measurements and the projected areas from the detected mask outputs. Various Mask R-CNNs were considered for the task of pineapple detection. The best-performing detector (Model 4: COCO Fliplr Res50) made use of MS COCO starting weights, a ResNet50 CNN backbone, and horizontal flipping data augmentation during the training process. This model achieved a validation AP@[0.5:0.05:0.95] of 0.914 and a test AP@[0.5:0.05:0.95] of 0.901, and was used to predict masks for an unseen data set containing images of pre-measured pineapples. The distributions of measurements extracted from the detected masks were compared to those of the manual measurements using two-sample Z-tests and Kolmogorov–Smirnov tests. There was sufficient similarity between the distributions, and it was therefore established that the reported method is appropriate for pineapple size determination in this context.

Significance:

• The methods we applied are traditional CNN models. Our main contribution is testing the application of different starting weights with multiple augmentation techniques used under different CNN backbones, ultimately comparing seven different model specifications of Mask R-CNNs.
• We have validated the results against manually measured fruits using statistical techniques and show that fruit size can be confidently determined from images instead of manual measurement, which is very labour and time intensive.
• The method developed is currently being used within a factory.

Open data set: https://github.com/Jess-cah/measure-pineapple