Application of computer vision and deep learning for automated monitoring of garden strawberry plant growth

The article presents the developed algorithm and software for automated monitoring of strawberry plant growth using neural network technologies. The YOLO11x and YOLOx-seg models, pre-trained by transfer learning, are adapted to recognize and classify plants (plant class), leaves (leaf class), and a reference marker (ref_obj class) of a known size. Segmentation of strawberry leaves using the YOLO11x-seg model makes it possible to analyze the morphometric parameters of individual leaf plates (area, perimeter, roundness, aspect ratio). A set of RGB images (2000 pieces) obtained using a GoPro HERO11 camera under controlled laboratory conditions was formed and annotated, followed by augmentation to increase the model's resistance to variations in shooting conditions. The developed algorithm converts the coordinates of the bounding boxes and segmentation masks of recognized objects into metric units using calibration coefficients calculated from a marker of known size (100×100 mm). The software implemented using PyQt5, TensorFlow, Keras, and OpenCV libraries provides not only visualization of results but also data storage in a local SQLite database with the ability to export to JSON and Excel formats. Validation of the model showed high accuracy in detecting plant bounding boxes (mAP50 = 0.906) and leaf segmentation (mAP50 -mask = 0.625). The average processing speed was 20.3 ms/frame for detection and 34.5 ms/frame for segmentation. The measurement error was less than 3.5 % for the overall parameters of the plant and 5.2 % for the morphometric parameters of the leaves, confirming the effectiveness of the method for assessing the height, width and area of plants, as well as the analysis of the leaf apparatus. The research results show the promise of an approach for automating plant phenotyping in real time.

1. Ozherelev V. N., Ozhereleva M. V., Grin A. M., Somin V. V. Dynamics of garden strawberries production all over the world. Vestnik Bryanskoy gosudarstvennoy sel'skokhozyaystvennoy akademii = Vestnik of the Bryansk State Agricultural Academy. 2019;(4(74)):60–66. (In Russ.). URL: https://elibrary.ru/item.asp?id=39174971

2. Marchenko L. A. Productivity of garden strawberry and breeding possibilities to improve it. Sibirskiy vestnik sel'skokhozyaystvennoy nauki = Siberian Herald of Agricultural Science. 2021;51(3):65–74. (In Russ.). DOI: https://doi.org/10.26898/0370-8799-2021-3-7

3. Guan Z., Abd-Elrahman A., Fan Z., Whitaker V. M., Wilkinson B. Modeling strawberry biomass and leaf area using object-based analysis of high-resolution images. ISPRS Journal of Photogrammetry and Remote Sensing. 2020;163:171–186. DOI: https://doi.org/10.1016/j.isprsjprs.2020.02.021

4. Marchenko L. А., Akimova S. V., Solovyov A. V., Makarov S. S., Samoshenkov E. G., Ter-Petrosyants G. E., Zubkov A. V. Role of mineral elements in the nutrition of garden strawberry plants. Ovoshchi Rossii = Vegetable crops of Russia. 2024;(5):79–83. (In Russ.). DOI: https://doi.org/10.18619/2072-9146-2024-5-79-83

5. Haida M., El Khalloufi F., Mugani R., Redouane E. M., Campos А., Vasconcelos V., Oudra B. Effects of Irrigation with Microcystin-Containing Water on Growth, Physiology, and Antioxidant Defense in Strawberry Fragaria vulgaris under Hydroponic Culture. Toxins. 2022;14(3):198 DOI: https://doi.org/10.3390/toxins14030198

6. Mi Z., Yan W. Q. Strawberry Ripeness Detection Using Deep Learning Models. Big Data and Cognitive Computing. 2024;8(8):92. DOI: https://doi.org/10.3390/bdcc8080092

7. Khort D. O., Kutyrev A. I., Smirnov I. G., Voronkov I. V. Development of an Automated Management System for Agricultural Technologies in Horticulture. Sel'skokhozyaystvennye mashiny i tekhnologii = Agricultural Machinery and Technologies. 2021;15(2):61–68. (In Russ.). DOI: https://doi.org/10.22314/2073-7599-2021-15-2-61-68

8. Zhang Y., Zhang L., Yu H., Guo Z., Zhang R., Zhou X. Research on the Strawberry Recognition Algorithm Based on Deep Learning. Applied Sciences. 2023;13(20):11298. DOI: https://doi.org/10.3390/app132011298

9. Shurygin B., Smirnov I. G., Chilikin A., Khort D. O., Kutyrev A. I., Zhukovskaya S., Solovchenko A. Mutual augmentation of spectral sensing and machine learning for non-invasive detection of apple fruit damages. Horticulturae. 2022;8(12):1111. DOI: https://doi.org/10.3390/horticulturae8121111

10. Khort D. O., Kutyrev A., Smirnov I., Andriyanov N., Filippov R., Chilikin A. et al. Enhancing sustainable automated fruit sorting: hyperspectral analysis and machine learning algorithms. Sustainability . 2024:16(22):10084. DOI: https://doi.org/10.3390/su162210084

11. Khort D. O., Kutyrev A. I., Smirnov I. G., Filippov R. A., Vershinin R. V. Razrabotka algoritmov sistemy raspoznavaniya yagod zemlya-niki sadovoy pri robotizirovannom sbore. Elektrotekhnologii i elektrooborudovanie v APK. 2020:67(1(38)):133–141. (In Russ.). DOI: https://doi.org/10.22314/2658-4859-2020-67-1-133-141

12. Xie D., Yao W., Sun W., Song Z. Real-Time Identification of Strawberry Pests and Diseases Using an Improved YOLOv8 Algorithm. Symmetry. 2024:16(10):1280. DOI: https://doi.org/10.3390/sym16101280

13. Yang J.-W., Kim H.-I. An Overview of Recent Advances in Greenhouse Strawberry Cultivation Using Deep Learning Techniques: A Review for Strawberry Practitioners. Agronomy. 2024;14(1):34. DOI: https://doi.org/10.3390/agronomy14010034

14. Jovanović R., Djordjevic A., Stefanovic M., Eric M., Pajić N. Enhanced Defect Management in Strawberry Processing Using Machine Vision: A Cost-Effective Edge Device Solution for Real-Time Detection and Quality Improvement. Applied Sciences. 2024;14(17):7771. DOI: https://doi.org/10.3390/app14177771

15. Fan L., Yu J., Zhang P., Xie M. Prediction of Strawberry Quality during Maturity Based on Hyperspectral Technology. Agronomy. 2024;14(7):1450. DOI: https://doi.org/10.3390/agronomy14071450

16. Basak J. K., Paudel B., Kim N. E., Deb N. C. Kaushalya Madhavi B. G., Kim H. T. Non-Destructive Estimation of Fruit Weight of Strawberry Using Machine Learning Models. Agronomy. 2022;12(10):2487. DOI: https://doi.org/10.3390/agronomy12102487

17. Christakakis P., Giakoumoglou N., Kapetas D., Tzovaras D., Pechlivani E.-M. Vision Transformers in Optimization of AI-Based Early Detection of Botrytis cinerea. AI. 2024;5(3):1301–1323. DOI: https://doi.org/10.3390/ai5030063

18. Xiao J.-R., Chung P.-C., Wu H.-Y., Phan Q.-H., Yeh J.-L.A., Hou M.T.-K. Detection of Strawberry Diseases Using a Convolutional Neural Network. Plants. 2021;10(1):31. DOI: https://doi.org/10.3390/plants10010031

19. Kesler S., Karakan A., Oğuz Y. Real-Time Strawberry Plant Classification and Efficiency Increase with Hybrid System Deep Learning: Microcontroller and Mobile Application. Applied Sciences. 2022;12(17):8860. DOI: https://doi.org/10.3390/app12178860

20. Kim H., Kim D. Deep-Learning-Based Strawberry Leaf Pest Classification for Sustainable Smart Farms. Sustainability. 2023;15(10):7931. DOI: https://doi.org/10.3390/su15107931

21. Wang C., Wang H., Han Q., Zhang Z., Kong D., Zou X. Strawberry Detection and Ripeness Classification Using YOLOv8+ Model and Image Processing Method. Agriculture. 2024;14(5):751. DOI: https://doi.org/10.3390/agriculture14050751

22. Chen Y., Xu H., Chang P., Huang Y., Zhong F., Jia Q. et al. CES-YOLOv8: Strawberry Maturity Detection Based on the Improved YOLOv8. Agronomy. 2024;14(7):1353. DOI: https://doi.org/10.3390/agronomy14071353

23. Maxwell A. E., Warner T. A., Guillén L. A. Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies - Part 1: Literature Review. Remote Sensing. 2021;13(13):2450. DOI: https://doi.org/10.3390/rs13132450

24. Saha K. K., Tsoulias N., Weltzien C., Zude-Sasse M. Estimation of Vegetative Growth in Strawberry Plants Using Mobile LiDAR Laser Scanner. Horticulturae. 2022;8(2):90. DOI: https://doi.org/10.3390/horticulturae8020090