Lidar Camera Calibration API

Introduction
The API requires the client to upload the images, PCD (pcap, csv, and bin are also supported), and configuration for camera setup in a zip file (.zip extension) in the format defined below. The contents of the zip file are called a dataset. 
1.The client makes an Upload and calibrate API call, which uploads their files and runs the calibration algorithm on the images and lidar files for the given configuration. 
2.The calibration process is completed without errors if the response to the Upload and calibrate API call will contain datasetId and Status as Done. 
3.The client can call the Get Extrinsic Parameters API using the datasetId obtained from the Upload response and calibrate API. This API responds with the various extrinsic parameters, error stats, and the query's status.
Folder Structure
We require image and lidar frame pairs from the camera and lidar for a given calibration. 
1.Place the images captured from the camera in a folder.
2.Place the Lidar data captured from the LiDAR in a folder.
3.config.json contains configuration details of the calibration (intrinsic parameters, calibration name, etc.)
contents of the zip file
Contents of the Camera folder
Contents of the LiDAR folder
1.The names of the folders and the images shown here are for demonstration purposes. Users should avoid using space in the folder and the image names.
2.The name of the JSON file should be config.json (case sensitive)
config.json for checkerboard
1
{
2
    "calibration_name": "Lidar camera calibration",
3
    "calibration_type": "lidar_camera_calibration",
4
    "multi_target": false,
5
    "max_correspondence": 0.05,
6
    "deep_optimization": false,
7
    "get_initial_estimates_from_lidar_autodetection": true,
8
    "mapping_pair_to_use_for_initial_estimates": 0,
9
    "target_matching_the_chosen_board":"0",
10
    "board_to_chose_from_left": 0,
11
​
12
    "lidar": {
13
        "name": "lidar"
14
    },
15
    "intrinsic_params": {
16
                "camera_name": "camera name",
17
                "fx": 4809.13303863791,
18
                "fy": 4804.6573641098275,
19
                "cx": 1994.0408528062305,
20
                "cy": 1441.0395643417517,
21
                "distortion_enabled": false,
22
                "lens_model": "pinhole",
23
                "k1": -0.03563526645635081,
24
                "k2": 0.2338404159941449,
25
                "k3": -1.3671429904044254,
26
                "k4": 0,
27
                "k5": 0,
28
                "k6": 0,
29
                "p1": -0.002478228973939787,
30
                "p2": -0.0026861612981927407
31
    },
32
    "targets": {
33
        "0": {
34
            "x": 7,
35
            "y": 8, 
36
            "type": "checkerboard",
37
            "square_size":0.12,
38
            "padding_right": 0.343,
39
            "padding_left": 0.22,
40
            "padding_top": 0.22,
41
            "padding_bottom": 0.22,
42
            "on_ground": false,
43
            "tilted": true
44
        }
45
    },
46
​
47
    "data": {
48
        "mappings": [
49
            [
50
                "camera/1.png",
51
                "lidar/1.pcd"
52
            ],
53
            [
54
                "camera/2.png",
55
                "lidar/2.pcd"
56
            ],
57
            [
58
                "camera/3.png",
59
                "lidar/3.pcd"
60
            ],
61
            [
62
                "camera/4.png",
63
                "lidar/4.pcd"
64
            ],
65
            [
66
                "camera/5.png",
67
                "lidar/5.pcd"
68
            ],
69
            [
70
                "camera/6.png",
71
                "lidar/6.pcd"
72
            ]  
73
        ]
74
    },
75
    "extrinsic_params_initial_estimates": {
76
        "roll": -91.22985012342338,
77
        "pitch": -1.8101400401363152,
78
        "yaw": -87.84825901836496,
79
        "px": 0.06356787067597357, 
80
        "py": -0.28854421270970754,
81
        "pz": -0.015338954542810408
82
    }
83
}
checkerboard_config.json
2KB
Code
config for checkboard
config.json for charucoboard
{
    "calibration_name": "Lidar camera calibration",
    "calibration_type": "lidar_camera_calibration",
    "multi_target": false,
    "max_correspondence": 0.05,
    "deep_optimization": false,
    "get_initial_estimates_from_lidar_autodetection": true,
    "mapping_pair_to_use_for_initial_estimates": 0,
    "target_matching_the_chosen_board":"0",
    "board_to_chose_from_left": 0,
    "lidar": {
        "name": "lidar"
    },
    "intrinsic_params": {
                "camera_name": "camera name",
                "fx": 4809.13303863791,
                "fy": 4804.6573641098275,
                "cx": 1994.0408528062305,
                "cy": 1441.0395643417517,
                "distortion_enabled": false,
                "lens_model": "pinhole",
                "k1": -0.03563526645635081,
                "k2": 0.2338404159941449,
                "k3": -1.3671429904044254,
                "k4": 0,
                "k5": 0,
                "k6": 0,
                "p1": -0.002478228973939787,
                "p2": -0.0026861612981927407
    },  
    "targets":{
        "0": {
            "rows": 14,
            "columns": 14,
            "type": "charucoboard",
            "square_size":0.08708571428,
            "marker_size": 0.06966857142,
            "dictionary": "5X5",
            "padding_right": 0,
            "padding_left": 0,
            "padding_top": 0,
            "padding_bottom": 0,
            "on_ground": true,
            "tilted": false
        },
        "1": {
            "rows": 13,
            "columns": 13,
            "type": "charucoboard",
            "square_size":0.09378461538,
            "marker_size": 0.0750276923,
            "dictionary": "6X6",
            "padding_right": 0,
            "padding_left": 0,
            "padding_top": 0,
            "padding_bottom": 0,
            "on_ground": true,
            "tilted": false
        },
        "2": {
            "rows": 12,
            "columns": 12,
            "type": "charucoboard",
            "square_size":0.1016,
            "marker_size": 0.08128,
            "dictionary": "7X7",
            "padding_right": 0,
            "padding_left": 0,
            "padding_top": 0,
            "padding_bottom": 0,
            "on_ground": true,
            "tilted": false
        },
        "3": {
            "rows": 13,
            "columns": 13,
            "type": "charucoboard",
            "square_size":0.09378461538,
            "marker_size": 0.0750276923,
            "dictionary": "original",
            "padding_right": 0,
            "padding_left": 0,
            "padding_top": 0,
            "padding_bottom": 0,
            "on_ground": true,
            "tilted": false
        }
    },
    "data": {
        "mappings": [
            [
                "Images/charuco_2_1.jpg",
                "PCDs/charuco_2_1.pcd"
            ],
            [
                "Images/charuco_2_2.jpg",
                "PCDs/charuco_2_2.pcd"
            ],
            [
                "Images/charuco_2_3.jpg",
                "PCDs/charuco_2_3.pcd"
            ]
        ]
    },
    "extrinsic_params_initial_estimates": {
        "roll": -91.22985012342338,
        "pitch": -1.8101400401363152,
        "yaw": -87.84825901836496,
        "px": 0.06356787067597357, 
        "py": -0.28854421270970754,
        "pz": -0.015338954542810408
    }
}
charucoboard_config.json
3KB
Code
Key
Value type
Description
calibration_name
string
Name of the calibration
calibration_type
string
Non-editable field.*Value should be lidar_camera_calibration
get_initial_estimates_from_lidar_autodetection 
Boolean
parameter to specify if we want to do autodetection of boards in lidar 
mapping_pair_to_use_for_initial_estimates
Integer
Index (0 based) of the mapping data i.e the files pair to use  for calculating initial estimates when initial estimates are not provided
target_matching_the_chosen_board
String
String corresponding to the target configuration. The target configuration should be the we want to use for initial estimates calculation
board_to_chose_from_left
number
optional parameter. When we are using 
"get_initial_estimates_from_lidar_autodetection" as true and the boards multiple boards we are using are of same size then we can specify the board to chose from left that should be considered for initial estimates calculation
​
​
​
multi_target
boolean
true: if multiple targets are used
false: if single target is used
max_correspondence
double
Accepted range is from 0 to 1
deep_optimisation
Boolean
Performs optimisation for the board edges.
true: If tilted = true and deep optimisation is needed 
false: If deep optimisation is not required or the tilted = false
lidar_name
string
It is the name given by the client to the lidar. The client can modify it as willed.
camera_name
string
It is the name given by the client to the camera. The client can modify it as willed.
lens_model
string
Describes the type of lens used by the camera. Accepted values
1.pinhole
2.fisheye
fx
double
Focal length of the cameras in the X-axis. Value in pixels.
fy
double
Focal length of the camera in the Y-axis. Value in pixels.
cx
double
Optical centre of the camera in the X-axis. Value in pixels.
cy
double
Optical centre of the camera in the Y-axis. Value in pixels.
distortion_enabled
boolean
Makes use of distortion coefficients (k1, k2, k3, k4, p1, p2) for the calibration algorithm when set true. Distortion coefficients (k1, k2, k3, k4, p1, p2) are not required if it is false.
k1, k2, k3, k4, p1, p2
double
These are the values for distortion coefficients of the camera lens.Note:
1.If the lens_model is pinhole we require k1, k2, k3, p1, and p2 values (no need of k4)
2.If the lens_model is fisheye then we require the k1, k2, k3, and k4 values. (p1 and p2 are not needed)
3.These parameters are not required if distortion_enabled is false.
targets
Object
It is a dictionary of dictionary with each dictionary having target properties
type
string
Describes the type of target used. Accepted values
1.checkerboard
2.charucoboard
x
integer
number of horizontaol corners in the checkerboard (this property is needed if the type = checkerboard)
y
integer
number of vertical corners in the checkerboar (this property is needed if the type = checkerboard)
rows
integer
number of horizontaol squares in the charucoboard (this property is needed if the type is charucoboard)
columns
integer
number of vertical squares in the charucoboard (this property is needed if the type is charcuboard)
square_size
double
Size of each square in meters
marker_size
double
The size of marker in a charucoboard in meters ( Normally it is 0.8 times of square size ) (this property is needed if the type is charucoboard)
dictionary
string
It is the string that defines the charuco dictionary of the target. We support
1.5X5
2.6X6
3.7X7
4.original
This property is needed if the type is charucoboard
padding_right
double
padding to the right of the board
padding_left
double
padding to the left of the board
padding_top
double
padding to the top of the board
padding_bottom
double
padding to the bottom of the board
on_ground
Boolean
true: if the board is kept on ground
false: if the board is not on the ground
tilted
Boolean
true: if the board is tilted
false: if the board is not tilted
data
Object
It stores the data related to mapping of the camera and the lidar files
mappings
List of lists
It is a list of lists, where each sub-list is a tuple containing names of the image and pcd paired together.
Note:
1.The first element in the tuple should be the image path
2.The second element in the tuple should be the lidar frame path from the lidar
3.The client can name their image and lidar frame as they want, but they must have the same name in the mapping list and be present in the provided path
extrinsic_params_initial_estimates
Object with all values as double
The estimated extrinsic parameters which will be optimised during calibration process.
1.roll
2.pitch
3.yaw
4.px
5.py
6.pz
Quickstart
Before invoking the APIs, the client must obtain the clientId and auth token from Deepen AI. If you are a calibration admin, you can create different Access Tokens using the UI and use those instead. clientId is part of the path parameters in most API calls, and the auth token should be prefixed with “Bearer “ and passed to the ‘Authorization’ header in all API requests.

How to get Access Tokens can be found on the following link: Access token for APIs​
API Reference
Upload file and calibrate
This API sends a zip file to the server and runs the calibration algorithm. Returns datasetId, extrinsic parameters, and status to the user as the response.
URL
POST https://tools.calibrate.deepen.ai/api/v2/external/clients/{clientId}/calibration_dataset
Request
Path parameters
Parameter name
Parameter type
Description
clientId
string
ClientId obtained from Deepen AI
Body
Key
Value
Description
file
.zip file
Zip file containing config and images in a suitable format
Response
JSON file containing dataset_id and status of the calibration.

Response object:
{
    "Dataset ID": "XXXXXXXXXXXXXXXXX",
    "Extrinsic Parameters": {
        "roll": -90.47755237974575,
        "pitch": -0.38434110269976385,
        "yaw": -87.95967045393508,
        "px": 0.06958801619530329,
        "py": -0.28251980028661544,
        "pz": -0.010306058948604074
    },
    "Error Stats": {
        "translation_error": 0.04085960836364045,
        "rotation_error": 0.7512576778920595,
        "reprojection_error": 27.18615944133744
    },
    "Status": "done"
}
Key
Status
dataset_id
A unique value to identify the dataset. dataset_id can be used to retrieve the extrinsic parameters.
status
Current status of the dataset.
1.ready: Files are uploaded, and the dataset is ready for Calibration.
2.in_progress: The calibration process has started
3.done: Calibration is done. Users can query for extrinsics.
Get Extrinsic Parameters
Returns the extrinsic parameters, error statistics, and the query's status.
URL
GET https://tools.calibrate.deepen.ai/api/v2/external/datasets/{datasetId}/extrinsic_parameters
Request
Path parameters
Parameter name
Parameter type
Description
datasetId
string
datasetId obtained from the response of Upload file and calibrate API.
Response
Returns a JSON dictionary containing datasetId, extrinsic parameters, error statistics, and query status.
Response Object:
{
    "Dataset ID": "XXXXXXXXXXXXXXXXX",
    "Extrinsic Parameters": {
        "roll": -90.47755237974575,
        "pitch": -0.38434110269976385,
        "yaw": -87.95967045393508,
        "px": 0.06958801619530329,
        "py": -0.28251980028661544,
        "pz": -0.010306058948604074
    },
    "Error Stats": {
        "translation_error": 0.04085960836364045,
        "rotation_error": 0.7512576778920595,
        "reprojection_error": 27.18615944133744
    }
}
Key
Description
dataset_id
A unique value to identify the dataset. dataset_id can be used to retrieve the extrinsic parameters.
extrinsic_parameters
roll, pitch, and yaw are given in degrees
and px, py, and pz are given in meters.
error_stats
translation_error: Mean of difference between the centroid of points of checkerboard/charucoboard in the LiDAR and the projected corners in 3-D from an image
rotation_error: Mean of difference between the normals of the checkerboard/charucoboard in the point cloud and the projected corners in 3-D from an image
reprojection_error: Mean of difference between the centroid of image corners and projected lidar checkerboard/charucoboard points on the image in 3-D

Parameter name	Parameter type	Description
clientId	string	ClientId obtained from Deepen AI

Key	Value	Description
file	.zip file	Zip file containing config and images in a suitable format

Parameter name	Parameter type	Description
datasetId	string	datasetId obtained from the response of Upload file and calibrate API.

Key	Description
dataset_id	A unique value to identify the dataset. dataset_id can be used to retrieve the extrinsic parameters.
extrinsic_parameters	roll, pitch, and yaw are given in degrees and px, py, and pz are given in meters.
error_stats	translation_error: Mean of difference between the centroid of points of checkerboard/charucoboard in the LiDAR and the projected corners in 3-D from an image rotation_error: Mean of difference between the normals of the checkerboard/charucoboard in the point cloud and the projected corners in 3-D from an image reprojection_error: Mean of difference between the centroid of image corners and projected lidar checkerboard/charucoboard points on the image in 3-D

Overlapping Camera Calibration API

LiDAR-LiDAR Calibration API

Last modified 30d ago

Key	Value type	Description
calibration_name	string	Name of the calibration
calibration_type	string	Non-editable field.*Value should be *lidar_camera_calibration*
get_initial_estimates_from_lidar_autodetection	Boolean	parameter to specify if we want to do autodetection of boards in lidar
mapping_pair_to_use_for_initial_estimates	Integer	Index (0 based) of the mapping data i.e the files pair to use for calculating initial estimates when initial estimates are not provided
target_matching_the_chosen_board	String	String corresponding to the target configuration. The target configuration should be the we want to use for initial estimates calculation
board_to_chose_from_left	number	optional parameter. When we are using "get_initial_estimates_from_lidar_autodetection" as true and the boards multiple boards we are using are of same size then we can specify the board to chose from left that should be considered for initial estimates calculation

multi_target	boolean	true: if multiple targets are used false: if single target is used
max_correspondence	double	Accepted range is from 0 to 1
deep_optimisation	Boolean	Performs optimisation for the board edges. true: If tilted = true and deep optimisation is needed false: If deep optimisation is not required or the tilted = false
lidar_name	string	It is the name given by the client to the lidar. The client can modify it as willed.
camera_name	string	It is the name given by the client to the camera. The client can modify it as willed.
lens_model	string	Describes the type of lens used by the camera. Accepted values 1. pinhole 2. fisheye
fx	double	Focal length of the cameras in the X-axis. Value in pixels.
fy	double	Focal length of the camera in the Y-axis. Value in pixels.
cx	double	Optical centre of the camera in the X-axis. Value in pixels.
cy	double	Optical centre of the camera in the Y-axis. Value in pixels.
distortion_enabled	boolean	Makes use of distortion coefficients (k1, k2, k3, k4, p1, p2) for the calibration algorithm when set true. Distortion coefficients (k1, k2, k3, k4, p1, p2) are not required if it is false.
k1, k2, k3, k4, p1, p2	double	These are the values for distortion coefficients of the camera lens.Note: 1. If the lens_model is pinhole we require k1, k2, k3, p1, and p2 values (no need of k4) 2. If the lens_model is fisheye then we require the k1, k2, k3, and k4 values. (p1 and p2 are not needed) 3. These parameters are not required if distortion_enabled is false.
targets	Object	It is a dictionary of dictionary with each dictionary having target properties
type	string	Describes the type of target used. Accepted values 1. checkerboard 2. charucoboard
x	integer	number of horizontaol corners in the checkerboard (this property is needed if the type = checkerboard)
y	integer	number of vertical corners in the checkerboar (this property is needed if the type = checkerboard)
rows	integer	number of horizontaol squares in the charucoboard (this property is needed if the type is charucoboard)
columns	integer	number of vertical squares in the charucoboard (this property is needed if the type is charcuboard)
square_size	double	Size of each square in meters
marker_size	double	The size of marker in a charucoboard in meters ( Normally it is 0.8 times of square size ) (this property is needed if the type is charucoboard)
dictionary	string	It is the string that defines the charuco dictionary of the target. We support 1. 5X5 2. 6X6 3. 7X7 4. original This property is needed if the type is charucoboard
padding_right	double	padding to the right of the board
padding_left	double	padding to the left of the board
padding_top	double	padding to the top of the board
padding_bottom	double	padding to the bottom of the board
on_ground	Boolean	true: if the board is kept on ground false: if the board is not on the ground
tilted	Boolean	true: if the board is tilted false: if the board is not tilted
data	Object	It stores the data related to mapping of the camera and the lidar files
mappings	List of lists	It is a list of lists, where each sub-list is a tuple containing names of the image and pcd paired together. Note: 1. The first element in the tuple should be the image path 2. The second element in the tuple should be the lidar frame path from the lidar 3. The client can name their image and lidar frame as they want, but they must have the same name in the mapping list and be present in the provided path
extrinsic_params_initial_estimates	Object with all values as double	The estimated extrinsic parameters which will be optimised during calibration process. 1. roll 2. pitch 3. yaw 4. px 5. py 6. pz

Key	Status
dataset_id	A unique value to identify the dataset. dataset_id can be used to retrieve the extrinsic parameters.
status	Current status of the dataset. 1. ready: Files are uploaded, and the dataset is ready for Calibration. 2. in_progress: The calibration process has started 3. done: Calibration is done. Users can query for extrinsics.