Radar-Camera Calibration

Overview: Deepen Calibrate is a software tool that makes the critical task of sensor data calibration simple and quick.

1. Calibrations:

This page allows the users to create, list, launch and delete calibration datasets. Admins can manage user's access to these datasets on this page.
Click on ‘New Calibration’ to create a new calibration dataset.

2. Calibration Selection:

Upon clicking the ‘New Calibration’ button, the user can make a selection from the different calibrations. Select ‘Radar-Camera Calibration’ to create a new radar-camera calibration dataset.

3. Calibration Instructions Page:

Upon selecting ‘Radar-Camera Calibration’, the user is welcomed to the instructions page

Click on ‘Get started’ after finishing with the instructions. A pop-up is displayed that asks for:

Dataset name
Edge length of the reflector
Configuration of the checkerboard

The user must fill in all the mandatory fields and click on ‘Set Details’ to proceed ahead.

The user can also choose to add the configuration by uploading a JSON file if required. Please check the Steps to upload the configuration by JSON file section for more details.

4. Calibration Pipeline:

Radar-Camera Calibration consists of two stages:

Radar Camera setup: Add camera details, files, and reflector coordinates.
Detect corners: Detect the checkerboard corners in each image.
Calibrate: Run the calibration algorithm and visualise the results.

A few other details 👍

The user can choose to hit the ‘Save changes’ option at any stage to prevent loss of work.
The user can navigate between the stages and update the details as per convenience.
Download the results by using the ‘Export’ option.
Click on the ‘Help Center’ button ( the top right corner) to get your doubts answered related to the calibration.

4.1 Radar Camera Setup:

This stage is broadly categorised into three steps:

Setup Camera Intrinsics: Add/upload your camera’s intrinsic parameters. The user can click on ‘Get intrinsic parameters’ to calibrate the camera.
Add files: Upload the files captured using the camera.
Add reflector coordinates: Enter the coordinates of the reflector captured using radar.

To move on to the calibration stage, complete the radar-camera setup and then click on continue.

4.2 Detect corners:

Users can click on the 'Detect corners' button. We try to auto-detect most of the corners, if the checkerboard corners detection fails user can add four border markers to detect all the inner checkerboard corners.

Users can select four boundary marker points in the order (top-left, top-right, bottom-left, bottom-right). And then click on the 'Detect corners' button, to get the remaining inner corners of the checkerboard.

4.3 Calibrate:

The user can review the files and camera intrinsics here before hitting the ‘calibrate’ button. This may take up to 1-2 minutes for us to get accurate results, depending on the quality and number of image files.

Validate the results in Visualize mode and mark the calibration process as complete. These extrinsic, camera intrinsics and error parameters can be exported by using the ‘Export’ option.

Please note that :

The visualization renders calibration results both in 2D and 3D.
The reprojection error rate gives the mean distance error between points in 2D.
The translational error rate gives the mean distance error between points in 3D.

5. Extrinsic Calibration Output:

roll, pitch, yaw, px, py, pz are the extrinsic parameters downloaded from the calibration tool.
reflectorPointsInRadar is the 3d coordinates of the reflector in the radar coordinate system.
reflectorPointsInCamera is the 3d coordinates of the reflector in the camera coordinate system.

Steps to upload the configuration by JSON file :

Click on get started.
Click on the import config button in the top right corner of the modal.
Upload the JSON config file.

We process the config file and give users the option to select all or specific configs. Once the user clicks on done, the config is saved with us and the user doesn’t have to type anything else.

The only user input remaining now will be uploading images.

Important note: While uploading images, please keep in mind that the file name must be exactly the same as that mentioned in the JSON config file, else the user won’t see radar positions in the UI, and the user has to manually enter the radar positions.

If user want to skip some parent keys ["name", "type", "approach", "version", "sensors", "targets", "data"] they can skip it. Eg: if, user want to just enter radar position, then they can just add data section and skip all other keys. Keep in mind that you can only skip parent level keys and not nested keys.

JSON config with Description

{
   // Name of the calibration datasets.
   "name": "radar camera",
   // Type of the calibration.
   "type": "radar_camera_calibration",
   // Calibration approach type.
   "approach": "target",
   // Config version, defaults to 1.
   "version": 1,
   // List of sensors.
   "sensors": [
       {
	   // Type of sensor: [camera, radar]
           "type": "camera",
	   // Name of sensor
           "name": "camera1"
           // Camera lens type: pinhole/fisheye,
           "lens_model": "pinhole",
           "intrinsics": {
               "fx": 3092.1424896390727,
               "fy": 3086.110800366092,
               "cx": 1962.4427444227433,
               "cy": 1484.0856987530365,
               "distortion": {
                   "is_enabled": true,
                   "k1": 0.066861015025329,
                   "k2": -0.2939800579680618,
                   "k3": 0.27962457717517547,
                   "p1": -0.000008687426909951295,
                   "p2": 0.00009712594960396621
               }
           }
       },
       {
           "type": "radar",
           "name": "5mm radar"
       }
   ],
   // List of targets used in the calibration
   "targets": [
       {
           "type": "trihedral_corner_reflector",
           "edge_length": 0.1
       },
       {
           "type": "checkerboard",
           "square_dimension": 0.1,
           "thickness": 0.03,
           "horizontal_corners": 9,
           "vertical_corners": 5,
           "padding": {
               "top": 0.1,
               "right": 0.1,
               "bottom": 0.1,
               "left": 0.1
           }
       }
   ],
   // Any misc. data
   "data": {
       // File specific data
       "file_data": [
           {
	       // Uploaded file name
               "file_name": "1.jpg",
	       // Radar position
               "position": {
                   "x": -0.0354,
                   "y": 1.1188,
                   "z": 0.1833
               }
           },
           {
               "file_name": "2.jpg",
               "position": {
                   "x": 0,
                   "y": 1.0353,
                   "z": 0.1565
               }
           }
       ]
   }
}