AI model maps dormant grapevine stems and branches with 88.25% mean IoU
A computer-vision model tested on 2,150 winter vineyard images from two sites in Xinjiang, China, distinguished woody vine structures more accurately than several established segmentation systems. Its developers present it as one perception component for possible future pruning robots, not as a complete or commercially validated robotic-pruning solution.
A perception system for winter pruning
Agricultural engineers have developed the Dual-branch Transformer-Mamba Fusion Network, or DTMF, to separate grapevine stems and pruning branches from complex outdoor images. The research, published in Smart Agricultural Technology and described on November 13, addresses computer vision for dormant-season vineyards rather than delivering a complete autonomous pruning machine.1
Winter pruning influences vine growth, crop production, vigor and grape quality, but it remains dependent on skilled crews in many regions. The work can be costly, slow and difficult to standardize across large properties, particularly amid labor shortages. Automation could therefore have commercial value if it eventually proves dependable under production conditions.1
DTMF tackles a prerequisite for robotic cutting: identifying the vine and its slender canes against trellis wires, soil, weeds and other visual clutter. Conventional segmentation systems may miss fine branch edges even when they capture the vine's overall form, making thin woody structures particularly difficult to trace continuously.1
Dual processing and staged training
The network uses two complementary processing paths. A vision-transformer branch examines the complete scene and relationships across distant parts of the image, while a branch combining a discrete wavelet transform with Mamba state-space modules concentrates on texture, edges, frequency information and continuity along narrow structures. The two streams remain separate through most of the computation and their predictions are combined only near the end.1
Training also proceeds in two stages: the initial phase emphasizes basic pixel classification, while the later phase gives more weight to boundary refinement. A superpixel-based loss is intended to keep the predicted outlines aligned with natural object boundaries.1
Dataset and benchmark results
The researchers used 2,150 high-resolution RGB images collected during winter dormancy in Xinjiang. The material comprised 1,250 images recorded in November 2024 at an experimental vineyard at Shihezi University and 900 recorded in December 2024 at a commercial production orchard in Turpan. Every image was manually annotated at pixel level for stems and branches.1
The dataset was divided into 1,505 training images, 215 validation images and 430 test images. Data augmentation increased the effective training collection to 4,515 images.1
On the test set, the complete DTMF system recorded 94.24% mean pixel accuracy and 88.25% mean intersection over union, or IoU. It outperformed DeepLabV3, PSPNet, FCN, Segformer, U-Net and a transformer-based U-Net in the reported comparisons.1
The largest advantage appeared in the more difficult branch class. DTMF reached 85.12% branch IoU, versus 70.13% for the best baseline in that category, a difference of 14.99 percentage points. Its stem IoU was 87.68%, although competing modern models were closer on stem segmentation.1
Component tests and broader demonstrations
Ablation experiments indicated that the dual encoder-decoder structure, the wavelet-Mamba fusion module and the two-stage loss strategy each contributed to performance. A transformer-only version produced 71.98% mean IoU, compared with 88.25% for the full architecture. In separate training-method tests, later-stage supervision and superpixel constraints increased branch IoU from 74.33% to 85.12%.1
Examples presented in the paper showed fewer missing or interrupted cane segments across difficult backgrounds, including wires and uneven soil, than the comparison systems. Tests on external images of other woody plants suggested possible wider structural applications, but the authors treated them only as illustrations, not as a formal assessment of generalization beyond vineyard data.1
Limits before robotic deployment
Image segmentation represents only one element of robotic pruning. A deployable machine would also have to reconstruct vines in three dimensions, determine cutting points, plan motion and operate a cutting tool reliably in vineyard conditions. DTMF's parallel architecture also raises computational requirements, potentially complicating its use on embedded hardware carried by agricultural robots.1
The evidence is geographically and seasonally limited: all images came from two Xinjiang locations during winter dormancy. The model has not been validated in other wine-growing regions, under different trellis systems or in summer vineyards with dense foliage. The dataset was split by image rather than by individual vine, and the researchers did not report repeated training runs that would show variation between random initializations.1
The reported figures consequently demonstrate progress in visual perception, not readiness for widespread commercial deployment. Faster future versions, broader validation and integration with the remaining robotic functions would be needed before the approach could contribute to steadier labor planning or more uniform vine management in vineyards supplying wine and other grape-based drinks.1
Sources
Written by the Dionysus editors from the sources above.
