Thiago T. Santos presented a webinar, "Deep Learning and SfM: a promising framework for agricultural automation – an example with grapes (and some apples)" on December 10 (Tuesday), 2019, at 3:00pm/15:00h BRT (UTC -3:00). The edited video is available through YouTube, viewing at our page here, and download as a .mp4 [173MB].
Thiago Santos is from the Brazilian Agricultural Research Corporation, Embrapa, in Campinas, Brazil.
Abstract.
Detection of grape clusters in a wine vineyard, subject of Thiago Santos' Decemeber 12, 2019 webinar.
Differently of industrial applications, agricultural environments lack structured spatial data and well-defined settings. Such an environment is complex and presents large variations between fields and even intra-field, so intelligent systems are needed for its dynamic interpretation. For example, in fruit growing, not even the number and the placement of fruits is known in advance, which imposes a challenge for any automation attempt. Fortunately, recent computer vision advances are able to automate the recovery of the 3-D structure of crop fields, by using techniques as structure-from-motion (SfM) and SLAM, and detect and classify objects of interest, such as plants, leaves and fruits, by using state-of-the art techniques in machine vision such as Convolutional Neural Networks (CNNs). In this presentation, we will focus on wine grapes, a crop presenting large variability in shape, color, and compactness. We can successfully detect grape clusters, and segment them using state-of-the-art CNNs. In a dataset containing 408 grape clusters from images taken on field, we have reached a F1-score up to 0.91 for instance segmentation, a fine separation of each cluster from other structures in the image that allows a more accurate assessment of fruit size and shape. We will also show that clusters can be identified and tracked along video sequences recording orchards rows, using 3-D information from a SfM system.
Three-dimensional reconstruction of grape canopy and clusters' structure.
We also will present a public dataset containing grape clusters properly annotated in 300 images and a novel annotation methodology for segmentation of complex objects in natural images. This computer vision pipeline can be replicated for different crops and production systems, and we will present a preliminary work with apples to illustrate that. Such pipeline can be employed on the development of sensing components for several agricultural and environmental applications.
Supplementary Materials.
Grape detection, segmentation and tracking using deep neural networks and three-dimensional association: arXiv.
Thiago Teixeira Santos, webinar speaker on December 10, 2019.
Video demo: YouTube.
3-D model demo: SketchFab.
WGISD Dataset: Github.
Other works: http://ttsantos.net.
Presenter Biography.
Presenter #1 Bio: Thiago Teixeira Santos was born in São Paulo, Brazil, in 1979. He received the Bachelor, the M.Sc. and the Ph.D. degrees in computer science from the University of São Paulo, Brazil, in 2000, 2004 and 2009, respectively. In 2010, he joined the Brazilian Agricultural Research Corporation, Embrapa, as a researcher. He works on computer vision applications for agriculture, including image-based plant phenotyping, fruit detection, and analysis of grassland for livestock. Dr. Santos is a member of the Brazilian Computer Society (SBC), and the Association for Computing Machinery (ACM).