TruckV2X Dataset

Overview

Autonomous trucking faces unique perception challenges due to large vehicle size and dynamic trailer movements, leading to extensive blind spots and occlusions. Tractor-trailer systems create persistent blind zones where structural components obstruct both self-perception and neighboring agents' sensing. Articulated motion exacerbates occlusions during maneuvers, while low-speed operation expands risk zones in mixed traffic.

Existing datasets primarily focus on light vehicles, lacking multi-agent configurations for heavy-duty scenarios. To address this gap, we present TruckV2X—the first large-scale truck-centered cooperative perception dataset with multi-modal sensing (LiDAR and cameras) and multi-agent cooperation (tractors, trailers, CAVs, RSUs). It supports the development of occlusion-resistant cooperative perception systems for autonomous trucking.

Dataset Highlights

• 88,396 LiDAR frames and 1.18M 3D bounding box annotations
• 64 scenarios covering urban/highway scenes
• Multi-agent data: tractors, trailers, CAVs, RSUs

Key Contributions

• First truck-centered multi-agent cooperative dataset
• Benchmarks for truck collaborative perception
• Quantifies trucks as occlusion sources and perception enhancers

Fig. 1: Illustration of truck-related occlusions.

Truck-specific Perception Challenges

Trucks present unique perception challenges due to their large size and articulated structure. The tractor-trailer configuration creates extensive blind zones that significantly impact both self-perception and the perception capabilities of surrounding vehicles.

Our analysis shows that tractor-trailer combinations create 1.5× more occluded area than passenger cars within 30m, worsening with trailer pivot (over 70% occlusion at 90° turns), making cooperative perception essential.

Fig. 2: Datasets available for the perception of autonomous driving.

Dataset Positioning

TruckV2X is the first multi-agent collaborative perception dataset specifically designed for heavy vehicles. TruckV2X fills a critical gap in existing autonomous driving datasets by focusing specifically on heavy-duty vehicles and their unique perception challenges.

Unlike previous datasets that primarily focus on passenger vehicles, TruckV2X incorporates multi-agent cooperation scenarios involving tractors, trailers, other connected vehicles, and road infrastructure, creating a comprehensive platform for developing cooperative perception algorithms.

Dataset Construction

Vehicle & Sensor Configurations

TruckV2X integrates a high-fidelity semi-trailer truck model into CARLA via Unreal Engine, with physics-based kinematic constraints for realistic articulation. Each agent is equipped with multi-modal sensors:

Fig. 3: Illustration of sensor configurations for the agents.

Multi-modal Sensing Architecture

Tractor/Trailer: 2×64-channel LiDARs (360° FOV, 120m range, 10Hz) and 5 cameras (800×600 resolution, 90°-120° FOV).

CAV: 1×64-channel LiDAR + 4 cameras (360° coverage).

RSU: 1×64-channel LiDAR + 1 camera (mounted at 4m height to minimize pedestrian blind spots).

Scenarios & Data Statistics

The dataset was built using CARLA, with 64 scenarios generated by adding controlled random vehicles and pedestrians. Scenarios cover diverse environments: country roads, urban streets, highways, and intersections (T-shaped, four-way, roundabouts).

Fig. 4: Representative scenarios with occlusion annotations (dashed ellipses denote blind zones).

Fig. 5: (a) Frame counts per scenario. (b) Distance statistics between agents. (c) Speed statistics of agents. (d) Articulation angle counts between the tractor and trailer. (e) Object category counts. (f) Statistics of co-visible objects among agents. (g) Distance statistics between objects and agents. (h) Speed statistics of objects.

Occlusions Analysis

Occlusions are a critical challenge in truck perception due to large size and articulation. TruckV2X captures extensive occlusion scenarios, with quantitative analysis revealing unique patterns:

Fig. 6: Distribution of objects occluded for different agents in a 120m range. Distribution of objects occluded (a) from truck; (b) from CAV by truck; (c) from RSU by truck; (d) from the tractor by trailer.

Cooperation Benefits: Occlusion Recovery Rate (ORR)

Cooperative perception mitigates occlusions. ORR is defined as the proportion of an ego agent's occluded objects detected by cooperators (range 0-1). Key findings:

Truck-CAV cooperation recovers >80% occluded objects in 60% of scenarios (short distances improve performance).
RSUs provide stable mid-range support, while trucks act as mobile platforms to reduce blind spots during maneuvers.

Fig. 7: The temporal evolution of occlusion recovery rates (ORR) for CAV provided by truck and CAV, combined with the distances between truck, RSU, and CAV over time, in the T-intersection scenario (left).

Fig. 8: Frequency distribution of occlusion recovery rates (ORR) across various ego and cooperator configurations, with superimposed probability density distributions of inter-agent distances stratified by ORR intervals.

Occlusion Recovery Patterns

Our analysis of Occlusion Recovery Rate (ORR) demonstrates how cooperative perception can significantly improve object detection in occluded scenarios.

The frequency distribution shows that different agent configurations excel at different distances, with trucks providing valuable perception support in medium-range scenarios, complementing the strengths of CAVs and RSUs.

This highlights the importance of heterogeneous multi-agent cooperation in addressing the unique occlusion challenges of trucking environments.

Benchmark Experiments

We evaluated 8 state-of-the-art cooperative perception methods on TruckV2X, focusing on 3D object detection. Metrics include Average Precision (AP) and mean AP (mAP) at IoU thresholds 0.3, 0.5, 0.7 for light vehicles, heavy vehicles, and VRUs.

Key Findings

Early Fusion outperforms other methods (mAP=47.88% at IoU=0.5 for truck ego) due to direct point cloud integration.
Intermediate Fusion (e.g., AttFuse, V2VNet) performs well for CAV ego, matching early fusion in light vehicle detection.
VRU detection remains challenging (low AP) due to small size and sparse LiDAR points.

Detection results for Truck, CAV, RSU as Ego

Fig. 9: Benchmark detection results of SOTA cooperative perception methods on TruckV2X, with truck, CAV, and RSU acting as ego in mutual collaboration, where tractor and trailer are integrated via early fusion to function as a unified agent truck.

Detection results for Tractor as Ego with Trailer Cooperation

Fig. 10: Benchmark detection results of SOTA cooperative perception methods on TruckV2X, with tractor(ego) and trailer collaborating.

Citation

If you use TruckV2X or find our work inspiring in your research, please cite our paper:

@ARTICLE{11096563,
  author={Xie, Tenghui and Song, Zhiying and Wen, Fuxi and Li, Jun and Liu, Guangzhao and Zhao, Zijian},
  journal={IEEE Robotics and Automation Letters}, 
  title={TruckV2X: A Truck-Centered Perception Dataset}, 
  year={2025},
  volume={},
  number={},
  pages={1-8},
  keywords={Agricultural machinery;Laser radar;Cameras;Robot sensing systems;Roads;Vehicle dynamics;Sensors;Safety;Vehicle-to-infrastructure;Training;Vehicle-to-everything;Cooperative Perception;Autonomous Trucking;Dataset},
  doi={10.1109/LRA.2025.3592884}}