Click on “Download PDF” for the PDF version or on the title for the HTML version.

If you are not an ASABE member or if your employer has not arranged for access to the full-text, Click here for options.

Seed Phenotyping on Neural Networks using Domain Randomization and Transfer Learning

Published by the American Society of Agricultural and Biological Engineers, St. Joseph, Michigan

Citation:  2021 ASABE Annual International Virtual Meeting  2100174.(doi:10.13031/aim.202100174)
Authors:   Venkat Margapuri, Mitchell Neilsen
Keywords:   Seed Phenotyping, Domain Randomization, Transfer Learning, Mask RCNN, YOLO, COCO, ImageNet

Abstract. Seed phenotyping is the idea of analyzing the morphometric characteristics of a seed to predict the behavior of the seed in terms of development, tolerance and yield in various environmental conditions. The performance of seed phenotyping requires that the morphometry of seeds be estimated which can be a complex task to perform considering that seeds, even of a certain variety, are not uniform. As a result, the manual estimation of seed morphometry requires a ginormous amount of man-power. In recent times, applications, both mobile and desktop, that estimate seed morphometry from images have become available. While the applications alleviate the problem to a degree, one key problem is the segmentation of clustered seeds on images. It is often the case that the seeds on the image are in contact with each other which makes it hard to distinguish one seed from another. This phenomenon inevitably leads to erroneous estimates of seed morphometry. Recent developments in the field of machine learning have led to the development of neural networks that perform object detection and instance segmentation. The focus of the work is the application and feasibility analysis of the state-of-the-art object detection and localization neural networks, Mask R-CNN and YOLO (You Only Look Once), for seed phenotyping using Tensorflow. One of the major bottlenecks of such an endeavor is the need for a large amount of training data. While the capture of a multitude of seed images is taunting, the images are also required to be annotated to indicate the boundaries of the seeds on the image and converted to data formats that the neural networks are able to consume. Although tools that manually perform the task of annotation are available for free, the amount of time required is enormous. In order to tackle such a scenario, the idea of domain randomization i.e. the technique of applying models trained on images containing simulated objects to real-world objects, is considered. Besides, transfer learning i.e. the idea of applying the knowledge obtained while solving a problem to a different problem, is used. The networks are trained on pre-trained weights from the popular ImageNet and COCO data sets. Five types of seeds i.e. canola, rough rice, sorghum, soy and wheat, are experimented with, as part of the work. The performance of the technique is evaluated using average precision and recall. In order to apply domain randomization, a sample of 40 seeds of each type is considered. Images of each of the seeds are captured and then laid on a uniform background in different orientations and sizes. In essence, this procedure creates a multitude of training images that are used to train the neural networks. This technique ensures that the user does need to possess numerous seeds to train the neural networks. Also, a plethora of training images can be generated on-demand with a desired number of seeds on each image. The ability to scale and orient the seed instances on the images means that the neural networks can be trained to be scale-invariant, a common problem in image processing. Upon the segmentation of seeds, a technique that closely follows the guidelines laid out by the International Seed Morphology Association is proposed to estimate the morphometry of the seeds using the neural networks. Briefly, the standard US government issued coin, the Penny, is used. Since the dimensions of the penny are known in advance, a relationship between the coin morphometry in pixels and metric units is established. This relationship is later leveraged to perform a simple cross-multiplication to yield the morphometry of the seeds in question.

(Download PDF)    (Export to EndNotes)