Precision agriculture aims to optimize crop yields while minimizing resource waste through targeted interventions. A key challenge in paddy farming is weed management, where conventional broadcast herbicide application wastes chemicals on weed-free areas and risks crop damage. An autonomous tractor capable of navigating crop rows, detecting weeds in real time, and spraying only confirmed weed locations would significantly reduce herbicide usage and labor costs.
System Overview
This paper presents AgriNav, an autonomous paddy field tractor designed and implemented at Florida Gulf Coast University to address all three challenges in a single integrated system. The architecture is composed of three cooperating modules communicating over the Robot Operating System (ROS). The full data flow extends from raw sensor inputs through perception and localization.
The three modules are:
- A vision-based object detection subsystem with two parallel implementations: a faithful WeedDet reimplementation and a lightweight CPU-trainable variant, plus an inverted-logic discrimination module that protects the rice class.
- A LiDAR-based crop row detection module providing both a navigation reference and a region-of-interest mask for the camera pipeline.
- A fused localization stack combining a six-state EKF over GNSS, IMU, and wheel odometry with a separate LiDAR-IMU EKF for under-canopy operation.
The key system-level integration point is the LiDAR-camera bridge. The same 2D LiDAR used for navigation also publishes a projected row-boundary mask to the camera pipeline, which uses it to constrain the inference region. This single channel reduces detection compute by an estimated 30 to 50 percent, removes false positives outside the inter-row zone, and supplies the world-coordinate ground-plane projection needed by the spray actuator. The bridge exploits a sensor that is already on the platform for navigation; the integration cost is software only.
Technical Contributions
The technical contributions of this work are:
Inverted-Logic Discrimination with Hardcoded Rice Veto
A discrimination module addresses the asymmetric-cost problem by protecting the rice class through a confidence gate that cannot be overridden by any downstream module.
LiDAR Crop Row Detection with EKF Tracking
A K-means clustering and RANSAC pipeline extracts left and right boundaries, with each row tracked by an independent two-state EKF that maintains tracking through short LiDAR occlusions in prediction-only mode.
Six-State CVTR EKF with Three-Level GNSS Outage Bridging
A constant-velocity-turn-rate localization filter fuses GNSS, IMU, and wheel odometry with explicit AVAILABLE, DEGRADED, and OUTAGE status modeling. Process noise is linearly inflated during dead reckoning, and the approach was validated through a 20-second simulated GNSS denial period with bounded positional drift of 2.46 m relative to simulator ground truth.
Four-Mechanism LiDAR-Camera Fusion Bridge
The primary system-level contribution consists of region-of-interest masking, world-coordinate projection of bounding boxes, ground-plane filtering, and bidirectional confidence fusion. All four mechanisms use a sensor already present for navigation, requiring no additional hardware.
End-to-End ROS Integration
A defined ROS topic interface contract specifies asymmetric per-class thresholds and timestamped messages for downstream time-aligned fusion.
Related Work
The AgriNav system draws on foundational publications that together define the technical landscape it builds on. This section summarizes the approaches adopted, modified, or extended by the AgriNav design.
LiDAR-Based Crop Row Detection
Liu and colleagues addressed over-canopy autonomous navigation in agricultural fields, where conventional vision-based row following becomes unreliable under variable lighting, shadow, and vegetation density. They proposed a pipeline that first segments LiDAR point clouds into left and right crop-row clusters using K-means clustering, then fits a line to each cluster using RANSAC to extract row geometry, and finally tracks each row over time using an Extended Kalman Filter.
The system was validated on a full-scale Amiga robot in Gazebo-simulated fields with corn and soybean crops at multiple growth stages, including challenging scenarios with weeds and row discontinuities.
Multi-Sensor Fusion for Under-Canopy Row Following
Higuti and colleagues addressed the reliability of autonomous row following for compact agricultural robots operating under canopy. They demonstrated that fusing IMU data with LiDAR measurements through an Extended Kalman Filter substantially improves navigation continuity under sensor noise, partial occlusion, and short row gaps.
Adding EKF-based sensor fusion improved the mean distance between manual interventions from 51.6 m without EKF to 400 m with EKF in fields without major gaps, an approximately 7.7-fold improvement.
LIO-EKF: High-Frequency LiDAR-Inertial Odometry
Wu and colleagues addressed whether classical EKF-based methods remain competitive with modern factor-graph and optimization-based approaches for LiDAR-inertial odometry. They proposed LIO-EKF, a lightweight system based on point-to-point registration with a classical EKF scheme that achieves pose estimation close to the IMU frame rate of 100 Hz while maintaining accuracy comparable to more complex methods.
The result confirms that classical EKF formulations remain a strong choice for real-time robotic applications when computational simplicity and predictable latency are required.
Application in AgriNav
AgriNav does not implement LIO-EKF directly, but the result motivates the overall architectural choice to use classical EKFs throughout the localization stack rather than a sliding-window optimizer or factor graph. The localization stack applies this principle to GNSS-IMU-odometry fusion, aiming for robustness in the under-canopy paddy environment with simpler tuning and predictable real-time performance.
Foundational Detection Components
Several other works are used directly as building blocks rather than being modified.
Smooth L1 loss provides outlier-robust bounding-box regression. Generalized IoU loss provides a non-zero gradient even when predicted and ground-truth boxes do not overlap, complementing Smooth L1 during early training.
The DeepWeeds dataset and benchmarking work established the typical performance envelope for comparable architectures and motivated the expected performance bounds at higher training resolution.
WeedDet: Improved RetinaNet for Real-Time Paddy Weed Detection
The detection logic developed for AgriNav is intentionally inverted from typical weed-detection pipelines. Rather than enumerating weed species, the system identifies the single rice class with high confidence and treats anything outside high-confidence rice regions as a weed.
The reasoning is based on asymmetric risk: a false negative on a rice plant results in herbicide damage to the crop, which is categorically worse than a missed weed. Neither of the foundational papers in this area addresses this asymmetry in its loss formulation, evaluation metrics, or actuation policy.
WeedDet Architecture
The detection model is a custom PyTorch implementation of WeedDet, an improved single-stage object detector based on RetinaNet and adapted here to a single-class rice-detection task. The architecture introduces three structural modifications to baseline RetinaNet.
Efficient Feature Pyramid Network
The neck is a three-level feature pyramid network. Lateral 1×1 convolutions reduce C3, C4, and C5 to 256 channels. Top-down upsampling and additive fusion produce P3, P4, and P5.
The standard RetinaNet P6 and P7 levels are intentionally omitted. Their omission saves approximately 7.71 million parameters without measurable accuracy loss on plant-scale objects, where multi-octave extreme-scale handling is unnecessary.
Data Pipeline
A single rice-detection dataset was sourced from Roboflow, containing approximately 4,041 images with tightly annotated per-plant bounding boxes. A secondary dataset of 1,347 augmented images was generated through horizontal flips and noise injection. Both datasets were exported in Pascal VOC XML format.
Training Configuration and Results
A baseline torchvision retinanet_resnet50_fpn was trained first as a sanity check on the data pipeline. The hyperparameters were:
- Optimizer: SGD
- Learning rate: 0.001
- Momentum: 0.9
- Weight decay: regularization coefficient; its numeric value was not provided
- Batch size: 2
- Scheduler: StepLR with a multiplicative factor of 0.1 after epoch 9
- Gradient clipping: maximum norm of 1.0
- Total training duration: 12 epochs
The baseline converged to an average loss of approximately 0.63.
The custom WeedDet model was trained for 94 epochs across multiple Google Colab sessions due to runtime limits. The configuration was:
- Optimizer: SGD
- Initial learning rate: 0.01
- Momentum: 0.9
- Weight decay: regularization coefficient; its numeric value was not provided
- Batch size: 2
- Gradient clipping: maximum norm of 1.0
- Learning-rate schedule: cosine schedule with warm restarts
Training data was augmented with horizontal flips, random shifts, random brightness changes, Gaussian blur, and CLAHE. Training stopped when validation loss diverged from training loss, indicating the onset of overfitting. The best observed loss was approximately 1.2346.
Quantitative mAP evaluation was not completed within the scope of the work, which is acknowledged as a limitation. Validation was qualitative.
On a dense ground-level paddy field image with overlapping rice stalks, the trained model produced approximately 60 detected rice instances at confidence scores from 0.32 to 0.88. On a sparse aerial-perspective image with well-separated young rice on light gravel, it produced approximately 22 detections at confidence scores from 0.33 to 0.77. On a post-flood image with high water-mud contrast, detections had confidence scores from 0.40 to 0.95.
Discrimination Module: Rice Protection by Inversion
The discrimination module sits between the detection model and the spray actuator. It converts a list of detections into a herbicide-actuation decision for each nozzle. Any image region not covered by a high-confidence rice bounding box is classified as a candidate weed zone.
Before any spray valve opens, every actuation candidate must pass through a confidence gate that imposes two conditions in conjunction:
- The rice detection’s IoU-aware classification score must exceed 0.5.
- The predicted class must not equal rice.
The second condition is the hardcoded rice veto. It is implemented as an unconditional return of spray = False whenever the rice class is predicted at any confidence level. The veto cannot be overridden by any downstream module.
An optional Excess Green minus Excess Red (ExGR) preprocessing transform is available. The transform subtracts normalized excess-red intensity from normalized excess-green intensity to increase contrast between green plant material and soil or water backgrounds.
ExGR is treated as a tunable preprocessing option rather than a fixed component because its benefit depends on ambient lighting.
Inference Post-Processing
During inference, anchors above a per-class confidence threshold are retained, and duplicates are suppressed using Non-Maximum Suppression at an IoU threshold of 0.45. The source text references recommended asymmetric thresholds, but does not provide their numeric values.
These asymmetric thresholds reflect the cost asymmetry of the task. A missed weed is recoverable; a missed rice plant means herbicide may be applied to the crop. The downstream discrimination module provides a second-stage filter, so the detection module is intentionally tuned for high recall.
Detection Performance
The lightweight model was trained for 12 epochs at the source text does not provide the training resolution on a CPU. Loss convergence was smooth under cosine annealing. Detection results on the test split are referenced in a table, but the table data is not included in the provided text.
The AP@IoU0.5 metric requires predicted boxes to overlap ground truth by at least 50 percent. At the source text does not provide the image resolution, small weed instances span only a few pixels, making precise localization difficult and causing IoU to fall below 0.5 even when the correct region is identified. This is a training-resolution artifact, not an architectural limitation.
The classification ability of the model is confirmed by the confidence distribution. Weed regions receive scores up to 0.995, and qualitative inspection confirms correct spatial localization of weed clusters.
For the system as a whole, this is the intended behavior: the detection module produces high-recall candidates, and the discrimination module downstream filters false positives before any actuation. Full GPU training at the source text does not provide the image resolution is expected to push weed AP above 30 percent and mAP above 60 percent, consistent with results reported in the literature for comparable architectures.
LiDAR-Based Crop Row Detection
The LiDAR-based row detection module is intended to support row navigation in environments where vision-only approaches become unstable. Lighting variation, shadows, and inconsistent vegetation can all degrade camera-based methods. LiDAR provides geometric structure that is largely independent of these conditions and can be processed with interpretable filtering and line-fitting steps.
The processing sequence is LiDAR scan, range and geometric filtering, row-boundary extraction, and centerline estimation.
Each incoming LaserScan message is converted to Cartesian coordinates in the robot frame. Points are separated into left and right clusters and filtered by expected row spacing. K-means clustering is applied within each side to group points belonging to the same row. RANSAC line fitting then extracts the row parameters with 100 iterations and a 0.05 m inlier threshold.
The intermediate outputs of this pipeline—filtered point clusters, fitted boundary lines, and centerline—are easy to interpret and visualize. This is a practical advantage compared with end-to-end black-box approaches because debugging and validation can be performed at each stage.
Outputs
The main output of the LiDAR module is the row centerline, computed as the bisector of the left and right boundaries.
These outputs serve a dual purpose. The centerline is used directly by the lane-following controller as the desired driving path. The boundary lines are also used to define a region of interest for the camera-based weed-detection module.
Observed Performance
The LiDAR row-detection results show several practical strengths:
- Reliable geometric row-structure estimation
- Interpretable intermediate outputs
- Direct support for centerline-based navigation
- The ability to define a region of interest for downstream perception
The detection confidence remained above 0.9 throughout simulated runs, including during GNSS outages where the system r
Frequently Asked Questions
What is the AgriNav system designed to do? AgriNav is an autonomous paddy-field tractor designed to navigate crop rows, detect weeds, and support targeted herbicide spraying.
Why does the detection system protect the rice class? A false negative on rice can cause herbicide damage to the crop, so the system uses a hardcoded rice veto that prevents spraying whenever rice is predicted.
How does LiDAR support camera-based weed detection? LiDAR detects crop-row boundaries and publishes a projected region-of-interest mask that constrains camera inference and supports world-coordinate projection.
How was the localization system evaluated during GNSS loss? A 20-second simulated GNSS denial period produced bounded positional drift of 2.46 m relative to simulator ground truth.