UniCal: A Single-Branch Transformer-Based Model for Camera-to-LiDAR Calibration and Validation

Abstract

We introduce a novel architecture, UniCal, for Camera-to-LiDAR (C2L) extrinsic calibration which leverages self-attention mechanisms through a Transformer-based backbone network to infer the 6-degree of freedom (DoF) relative transformation between the sensors. Unlike previous methods, UniCal performs an early fusion of the input camera and LiDAR data by aggregating camera image channels and LiDAR mappings into a multi-channel unified representation before extracting their features jointly with a single-branch architecture. This single-branch architecture makes UniCal lightweight, which is desirable in applications with restrained resources such as autonomous driving. Through experiments, we show that UniCal achieves state-of-the-art results compared to existing methods. We also show that through transfer learning, weights learned on the calibration task can be applied to a calibration validation task without re-training the backbone.

Key Contributions

• Single-branch architecture: transformer model performing early fusion of camera and LiDAR data
• Lightweight design: single-branch backbone suited to resource-constrained deployment
• Benchmark results: competitive calibration accuracy vs. prior deep learning methods on standard datasets
• Transfer Learning Capability: Weights learned for calibration can be transferred to validation tasks
• 6-DoF Transformation: Infers complete relative transformation between camera and LiDAR sensors

Citation

@article{cocheteux2023unical,
    title={UniCal: a Single-Branch Transformer-Based Model for Camera-to-LiDAR Calibration and Validation},
    author={Cocheteux, Mathieu and Low, Aaron and Bruehlmeier, Marius},
    journal={arXiv preprint arXiv:2304.09715},
    year={2023}
}