Under the Hood

PathVynt's fusion engine maintains two separate state estimates: a rigid-body extrinsic transform between the LiDAR and camera frames, and a per-frame uncertainty covariance over that transform. Most fusion implementations treat extrinsic calibration as a static pre-deployment parameter. PathVynt treats it as a continuously-estimated quantity.

The temporal alignment problem — LiDAR scans at 10Hz, cameras at 30Hz, with non-negligible communication latency on each — is handled by maintaining a short history of sensor frames and selecting the nearest camera frame within a configurable latency window (default: 15ms). If no frame exists within that window, the fusion result is flagged with TEMPORAL_GAP status but not dropped, preserving pipeline continuity.

Extrinsic drift is modeled as a random walk over the SE(3) manifold. The covariance grows with time and resets (partially) when a stable scene feature cluster is detected. This works well for the vibration profile of warehouse AGVs and delivery robot chassis. Long-haul truck vibration profiles require a tighter process noise parameter, which is exposed in FusionConfig.drift_model.

PathVynt's localization module uses LiDAR scan feature extraction (edge features and planar features from the LOAM decomposition) combined with keyframe matching against a pre-built map. The map is built from a single survey drive and stored as a compact keyframe database — typically 2-8MB per 100m of mapped corridor depending on feature density.

The particle filter update runs at 10Hz. Between updates, the pose estimate is propagated forward using wheel odometry or IMU integration if available, or using the last-known velocity estimate if not. Feature occlusion (robot partially blocked by a dynamic obstacle during scan) is handled by masking out the occluded angular sectors from the feature extraction step.

Dynamic reference updates allow the map to be patched without a full re-survey — useful for warehouse environments where rack configurations change. The patcher identifies which keyframes are affected by a change annotation and updates only those frames, preserving continuity of the pose estimate throughout the update.