Data Collection for Lidar-Camera Calibration
Calibration Target:
Checkerboard is the calibration target. Any checkerboard of different sizes and different internal corners can be used.
Eg: You can print the attached pdf file on a foam board at 1.0m x 0.6m. Most print shops can print this. It has 5 internal corners horizontally and 9 internal corners vertically. Each square size is 10 cms. And distance from left-most corner is 10cms, the distance from right-most corner is 10 cms, distance from top-most corner is 10 cms, distance from right-most corner is 10cms. https://drive.google.com/file/d/1mTR8HTpvROE1Pv0rmXEBVLSxs_yMDnvf/view?usp=sharing
Place the checkerboards at roughly 3m - 10m from the camera. For the closest checkerboard, the closer, the better but it should be far enough so that all the edges of the board are visible from the camera and lidar. All the checkerboards should stand on the ground which is as flat as possible. So, it is highly recommended to do this capture inside a building rather than outside. No checkerboard should be occluded by other boards in the camera or lidar view.
The size of the checkerboard squares should be about 10cm. The side of the squares must be parallel to the edge of the checkerboard.
All the checkerboards should be identical. Note that the distance from the edge of the checkerboard to the closest checkerboard corner should be identical too.
For example, please take images like the following.
The boards and all sensors should be static while collecting the data. In order to avoid time-synchronization problems, please keep the boards and the sensors stationary for at least 10 seconds while collecting each set of calibration data.
For example, these are the steps to collect one set of calibration data:
- 1.Orient the camera toward the checkerboards. Start recording. Wait for 10 seconds (Don't move/rotate your car/robot/sensors). Stop recording. You must have a recording of images and lidar data for 10 seconds. Extract one image from the camera and one frame of lidar data captured 5 seconds after recording has started (e.g. if you start recording at 3:00:00, you stop recording at 3:00:10. We need a frame captured at 3:00:05) and save them. (Dataset front_1)
- 2.Change the check boards' location and orientation. Start recording. Wait for 10 seconds (Don't move/rotate your car/robot). Stop recording. Again, you must have a recording of images and lidar data for 10 seconds. Extract one image from the camera and one frame of lidar data captured 5 seconds after recording has started and save them. (Dataset front_2)
- 3.In the same manner, save Dataset front_3, front_4, and front_5.
If you have other cameras, repeat the above procedure 1-3 for each camera.
Suppose you have n cameras, then you will have 5n datasets, each of which is a pair of image and lidar data. More specifically, you should have 5n lidar files + 5n png images (5 images for each of n cameras).
For the targetless calibration, users just have to record a scene in both the camera and the LiDAR sensor. No target is required for this calibration, but the scene should have vehicles (cars and vans) present both in the camera and the LiDAR data. For better calibration vehicles should be close to the LiDAR (3m - 10m) with a good number of points in LiDAR and present on both left and right sides on the image; having too many vehicles may result in calibration errors. Make sure that the vehicles (including ego) are either stopped or moving slowly. This reduces the effect of the time difference between LiDAR and the camera. Select 3-4 frames from the collected data which have vehicles on both sides of the images and vehicles in close proximity to the lidar.
Try to have the board position in either one of the following two ways.
- The checkerboard is placed on the ground (touching the ground) . In such case, enable the 'is checkerboard on the ground' flag in the checkerboard configuration. Also make sure that the lidar data captures the ground points. By doing so, we are able to optimise the extrinsic parameters using the ground points.
Sample image for board on the ground.
- The checkerboard is held up in the air by a holder and tilted right by around 45 degrees. In such case, enable 'is checkerboard tilted' flag in the checkerboard configuration. Using this approach, the deep optimise can be enabled and the extrinsic parameters are futher optimised using the edge points of the board.
Sample image for tilted board