Click on “Download PDF” for the PDF version or on the title for the HTML version.

If you are not an ASABE member or if your employer has not arranged for access to the full-text, Click here for options.

Kiwifruit yield estimation using deep learning and binocular stereo vision on Android smartphones

Published by the American Society of Agricultural and Biological Engineers, St. Joseph, Michigan

Citation:  2021 ASABE Annual International Virtual Meeting  2100289.(doi:10.13031/aim.202100289)
Authors:   Zhongxian Zhou, Longsheng Fu
Keywords:   Android; binocular stereo vision; deep learning; yield estimation

Abstract. Knowing beforehand yield of kiwifruit to be harvested leads to better logistics and decisions making in agricultural industry. Fruit yield estimation has been relied on time-consuming manual assessment, which is thus highly desirable to be automated. Deep learning has achieved outstanding results in the field of fruit detection. Binocular stereo vision has widely been used to measure distance and object‘s size. Besides, easy-carry smartphones are getting powerful and often have two or more rear lenses. In this paper, deep learning and binocular stereo vision technology were employed to develop an Android APP for detecting and counting kiwifruits in real-time videos and calculating their sizes. Existing studies calculating size all used two same lenses to construct a binocular stereo vision system. However, lenses on smartphone were different from each other, which usually included a standard lens and a wide-angle lens. This paper first got intrinsic and extrinsic parameters matrix of the two lenses by calibrating them using Zhang‘s calibration method, then generated disparity map by rectifying image pairs and SGBM stereo matching algorithm, and final calculated kiwifruit size. A lightweight model trained by SSD with quantized MobileNetV2 and a detection line method were used to detect and count kiwifruits in the images. Results showed that the APP obtained average counting error rate of 15.2% and counting speed of 30 ms per image. Average depth and diameter error between the APP and manual measurement was 6.92% and 6.58%, respectively. These results indicated that the proposed Android APP is promising for kiwifruit yield estimation.

(Download PDF)    (Export to EndNotes)