Development of a vision system for a technological line for sorting apple fruits based on an artificial neural network

   This article describes the process of creating a training sample of an artificial neural network (hereinafter – ANN) of a vision system. Training the ANN was carried out on the basis of annotated images of real apples containing a description of various defects in the form of separate polygons using the LabelMe program. On the image of the fruit, the apple itself and its pomological features, such as receptacle, stalk and leaf, were marked, as well as 10 different fruit defects, each of which was given an appropriate name: mesh, pressure, cut, rot, scab, hailstone, etc. The obtained labeled images of fetuses with defects formed a reference training set for the ANN. The performance of the ANN was tested by evaluating the correctness of recognition of fetal images when comparing them with reference images. Training the ANN for each of the defects in apples was stopped when 95 % of the probability of correct assessment of the defect was reached. The ANN trained on the created training sample was used in the vision system of the LSP-4 production line, which sorted apples into three commercial varieties by size and defects from mechanical damage, diseases, and pests. The accuracy of sorting by size was 75.4 %, and by the presence of defects – 73.1 %.

1. Smirnov I. G., Khort D. O., Kutyrev A. I. Intelligent Technologies and Robotic Machines for Garden Crops Cultivation. Sel'skokhozyaystvennye mashiny i tekhnologii = Agricultural Machinery and Technologies. 2021;15(4):35-41. (In Russ.). doi: 10.22314/2073-7599-2021-15-4-35-41

2. Balabanov P. V., Divin A. G., Mishchenko S. V., Makarova V. S., Markov A. V., Sadomov Ya. O. Robotic complex for sorting apples. Digitalization of the agro-industrial complex: collection of scientific articles of the II International Scientific and Practical Conference. Tambov: Tambovskiy gosudarstvennyy tekhnicheskiy universitet, 2020. Vol. II. pp. 44-47. URL: https://elibrary.ru/item.asp?id=45032750&pff=1

3. Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the Inception Architecture for Computer Vision. Cornel Univercity Library. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016. pp. 2818-2826. doi: 10.1109/CVPR.2016.308

4. Yuzhen Lu, Renfu Lu. Development of a multispectral Structured Illumination Reflectance Imaging (SIRI) system and its application to bruise detection of apples. Transactions of the ASABE. 2017;60(4):1379-1389. doi: 10.13031/trans.12158

5. Kazakevich P. P., Yurin A. N., Prokopovich G. A. Technical vision system for apple defects recognition: justification, development, testing. Vestsі Natsyyanal'nay akademіі navuk Belarusі. Seryya agrarnykh navuk = Proceedings of the National Academy of Sciences of Belarus. Agrarian series. 2021;59(4):488-500. (In Belarus). doi: 10.29235/1817-7204-2021-59-4-488-500

6. Yurin A. N. Innovative technological processes and technical complexes for intensive horticulture in Belarus. Minsk: Belaruskaya navuka, 2022. 208 p.

7. Khort D. O., Kutyrev A. I., Smirnov I. G., Filippov R. A., Vershinin R. V. Developing algorithms for a berry recognition system used in robotized harvesting of garden strawberry. Elektrotekhnologii i elektrooborudovanie v APK. 2020;67(1(38)):133-141. (In Russ.). doi: 10.22314/2658-4859-2020-67-1-133-141

8. Zhirkova A. A., Balabanov P. V., Divin A. G. Automated System for Hyperspectral Inspection of Apple Defects. Modern science: theory, methodology, practice: Proceedings of the 3rd All-Russian (national) scientific and practical. conf. Tambov: Izdatel'stvo IP Chesnokova A. V., 2021. pp. 291-296. URL: https://elibrary.ru/item.asp?id=45831350&selid=46177515

9. Khort D. O., Kutyrev A. I., Filippov R. A., Vershinin R. V., Smirnov I. G. Neural network for recognition of fruits and berries of horticultural crops: certificate of registration of a computer program no. 2020660182 RF. 2020.

10. Azarenko V. V., Komlach D. I., Goldyban V. V., Baranovskiy I. A., Prokopovich G. A. Development of mounted system for controlling row crop cultivator in automatic mode. Vestsі Natsyyanal'nay akademіі navuk Belarusі. Seryya agrarnykh navuk = Proceedings of the National Academy of Sciences of Belarus. Agrarian series. 2021;59(2):232-242. (In Belarus). doi: 10.29235/1817-7204-2021-59-2-232-242

11. Dorokhov A. S., Aksenov A. G., Khort D. O., Kutyrev A. I., Teterev A. V., Sibirev A. V., Moskovskiy M. N., Filippov R. A., Semichev S. V., Mosyakov M. A. Database of spectral images of diseases and injuries of cereal crops, fruits and tubers of potatoes: certificate of registration of the database no. 2021620285 RF. 2021.

12. Kortylewski A., Schneider A., Gerig T., Egger B., Morel-Forster A., Vetter T. Training deep face recognition systems with synthetic data. Cornell University Library. 2018. URL: https://arxiv.org/pdf/1802.05891.pdf

13. Huang J., Rathod V., Sun Ch., Zhu M., Korattikara A., Fathi A., Fischer I., Wojna Z., Song Ya., Guadarrama S., Murphy K. Speed/accuracy trade-offs for modern convolutional object detectors. Cornel University Library. 2016. URL: https://arxiv.org/pdf/1611.10012.pdf

14. Ganganagowdar N. V., Gundad A. V. An intelligent computer vision system for vegetables and fruits quality inspection using soft computing techniques. Agricultural Engineering International. 2019;21(3):171-178. URL: https://cigrjournal.org/index.php/Ejounral/article/view/5188

15. Yurin А. N., Viktorovich V. V., Ignatchik A. A. Reducing labor costs by using the vision system when sorting apples. Mekhanizatsiya i elektrifikatsiya sel'skogo khozyaystva = Mechanization and Electrification of Agriculture. 2022;(55):88-95. (In Russ.). URL: https://mechel.belal.by/jour/article/view/707/712