Lidar Camera Calibration API

Introduction

The API requires the client to upload the images, PCD (pcap, csv, and bin are also supported), and configuration for camera setup in a zip file (.zip extension) in the format defined below. The contents of the zip file are called a dataset.

1. The client makes an Upload and calibrate API call, which uploads their files and runs the calibration algorithm on the images and lidar files for the given configuration.

2. The calibration process is completed without errors if the Upload and calibrate API call response contains dataset_id, extrinsic_camera_coordinate_system, extrinsic_parameters, error_stats, and projected_images.

3. The client can call the Get Extrinsic Parameters API using the dataset_id obtained from the Upload and calibrate API. This API responds with dataset_id, extrinsic_camera_coordinate_system, extrinsic_parameters, error_stats, and projected_images.

Folder Structure

We require image and lidar frame pairs from the camera and lidar for a given calibration.

1. Place the images captured from the camera in a folder.

2. Place the Lidar data captured from the LiDAR in a folder.

3. config.json contains configuration details of the calibration (intrinsic parameters, calibration name, etc.)

Note: Folder structure is optional. Users can place all files in the main directory and zip it.

Note

1. The names of the folders and the images shown here are for demonstration purposes. Users should avoid using space in the folder, lidar, and image filenames.

2. The name of the JSON file should be config.json (case sensitive)

config.json for checkerboard

{
    "calibration_name": "Lidar camera calibration",
    "calibration_type": "lidar_camera_calibration",
    "calibration_group_id": "xxxxxxxxxxxxxxxxxx",
    "multi_target": false,
    "max_correspondence": 0.05,
    "deep_optimization": false,
    "deep_optimization_approach": "custom_ransac",
    "is_lidar_inverted": false,
    "get_initial_estimates_from_lidar_autodetection": true,
    "mapping_pair_to_use_for_initial_estimates": 0,
    "target_matching_the_chosen_board": "0",
    "board_to_chose_from_left": 0,
    "lidar":
    {
        "name": "lidar"
    },
    "extrinsic_camera_coordinate_system": "OPTICAL",
    "intrinsics":
    {
        "camera_name": "camera name",
        "fx": 4809.13303863791,
        "fy": 4804.6573641098275,
        "cx": 1994.0408528062305,
        "cy": 1441.0395643417517,
        "k1": -0.03563526645635081,
        "k2": 0.2338404159941449,
        "k3": -1.3671429904044254,
        "k4": 0,
        "k5": 0,
        "k6": 0,
        "p1": -0.002478228973939787,
        "p2": -0.0026861612981927407,
        "distortion_enabled": false,
        "lens_model": "pinhole"
    },
    "targets":
    {
        "0":
        {
            "horizontal_corners": 7,
            "vertical_corners": 8,
            "type": "checkerboard",
            "square_size": 0.12,
            "padding_right": 0.343,
            "padding_left": 0.22,
            "padding_top": 0.22,
            "padding_bottom": 0.22,
            "on_ground": false,
            "tilted": true,
            "ignore_top_edge": true
        }
    },
    "data":
    {
        "mappings":
        [
            [
                "camera/1.png",
                "lidar/1.pcd"
            ],
            [
                "camera/2.png",
                "lidar/2.pcd"
            ],
            [
                "camera/3.png",
                "lidar/3.pcd"
            ],
            [
                "camera/4.png",
                "lidar/4.pcd"
            ],
            [
                "camera/5.png",
                "lidar/5.pcd"
            ],
            [
                "camera/6.png",
                "lidar/6.pcd"
            ]
        ]
    },
    "extrinsic_params_initial_estimates":
    {
        "roll": -91.22985012342338,
        "pitch": -1.8101400401363152,
        "yaw": -87.84825901836496,
        "px": 0.06356787067597357,
        "py": -0.28854421270970754,
        "pz": -0.015338954542810408
    }
}

Sample config.json

config.json for charucoboard

{
    "calibration_name": "Lidar camera calibration",
    "calibration_type": "lidar_camera_calibration",
    "calibration_group_id": "xxxxxxxxxxxxxxxxxx",
    "multi_target": false,
    "max_correspondence": 0.05,
    "deep_optimization": false,
    "deep_optimization_approach": "custom_ransac",
    "is_lidar_inverted": false,
    "get_initial_estimates_from_lidar_autodetection": true,
    "mapping_pair_to_use_for_initial_estimates": 0,
    "target_matching_the_chosen_board": "0",
    "board_to_chose_from_left": 0,
    "lidar":
    {
        "name": "lidar"
    },
    "extrinsic_camera_coordinate_system": "OPTICAL",
    "intrinsics":
    {
        "camera_name": "camera name",
        "fx": 4809.13303863791,
        "fy": 4804.6573641098275,
        "cx": 1994.0408528062305,
        "cy": 1441.0395643417517,
        "k1": -0.03563526645635081,
        "k2": 0.2338404159941449,
        "k3": -1.3671429904044254,
        "k4": 0,
        "k5": 0,
        "k6": 0,
        "p1": -0.002478228973939787,
        "p2": -0.0026861612981927407,
        "distortion_enabled": false,
        "lens_model": "pinhole"
    },
    "targets":
    {
        "0":
        {
            "rows": 14,
            "columns": 14,
            "type": "charucoboard",
            "square_size": 0.08708571428,
            "marker_size": 0.06966857142,
            "dictionary": "5X5",
            "padding_right": 0,
            "padding_left": 0,
            "padding_top": 0,
            "padding_bottom": 0,
            "on_ground": true,
            "tilted": false,
            "ignore_top_edge": true
        },
        "1":
        {
            "rows": 13,
            "columns": 13,
            "type": "charucoboard",
            "square_size": 0.09378461538,
            "marker_size": 0.0750276923,
            "dictionary": "6X6",
            "padding_right": 0,
            "padding_left": 0,
            "padding_top": 0,
            "padding_bottom": 0,
            "on_ground": true,
            "tilted": false,
            "ignore_top_edge": true
        },
        "2":
        {
            "rows": 12,
            "columns": 12,
            "type": "charucoboard",
            "square_size": 0.1016,
            "marker_size": 0.08128,
            "dictionary": "7X7",
            "padding_right": 0,
            "padding_left": 0,
            "padding_top": 0,
            "padding_bottom": 0,
            "on_ground": true,
            "tilted": false,
            "ignore_top_edge": true
        },
        "3":
        {
            "rows": 13,
            "columns": 13,
            "type": "charucoboard",
            "square_size": 0.09378461538,
            "marker_size": 0.0750276923,
            "dictionary": "original",
            "padding_right": 0,
            "padding_left": 0,
            "padding_top": 0,
            "padding_bottom": 0,
            "on_ground": true,
            "tilted": false,
            "ignore_top_edge": true
        }
    },
    "data":
    {
        "mappings":
        [
            [
                "Images/charuco_2_1.jpg",
                "PCDs/charuco_2_1.pcd"
            ],
            [
                "Images/charuco_2_2.jpg",
                "PCDs/charuco_2_2.pcd"
            ],
            [
                "Images/charuco_2_3.jpg",
                "PCDs/charuco_2_3.pcd"
            ]
        ]
    },
    "extrinsic_params_initial_estimates":
    {
        "roll": -91.22985012342338,
        "pitch": -1.8101400401363152,
        "yaw": -87.84825901836496,
        "px": 0.06356787067597357,
        "py": -0.28854421270970754,
        "pz": -0.015338954542810408
    }
}

Sample config.json

config.json key description

Quickstart

Before invoking the APIs, the client must obtain the clientId and auth token from Deepen AI. If you are a calibration admin, you can create different Access Tokens using the UI and use those instead. clientId is part of the path parameters in most API calls, and the auth token should be prefixed with “Bearer“ and passed to the ‘Authorization’ header in all API requests. How to get Access Tokens can be found on the following link: Access token for APIs

Upload file and calibrate

This POST api call sends a zip file to the server and runs the calibration algorithm. Returns dataset_id, extrinsic_camera_coordinate_system, extrinsic parameters, error_stats, and projected_images to the user as the response.

https://tools.calibrate.deepen.ai/api/v2/external/clients/{clientId}/calibration_dataset

Request

Path parameters

Body

Response

{
    "dataset_id": "XXXXXXXXXXXXXXXXX",
    "calibration_algorithm_version": "XXXXXXXXXXXXXXXXX",
    "extrinsic_camera_coordinate_system": "OPTICAL",
    "extrinsic_parameters": {
        "roll": -90.47755237974575,
        "pitch": -0.38434110269976385,
        "yaw": -87.95967045393508,
        "px": 0.06958801619530329,
        "py": -0.28251980028661544,
        "pz": -0.010306058948604074
    },
    "error_stats": {
        "translation_error": 0.04085960836364045,
        "plane_translation_error": 0.00832952204,
        "rotation_error": 0.7512576778920595,
        "reprojection_error": 27.18615944133744
    },
    "projected_images": "XXXXXXXXXXurl_to_download_projected_imagesXXXXXX"
    
}

Get Extrinsic Parameters

This GET api call returns dataset_id, extrinsic_camera_coordinate_system, extrinsic parameters, error_stats, and projected_images to the user as the response.

https://tools.calibrate.deepen.ai/api/v2/external/datasets/{dataset_id}/extrinsic_parameters

https://tools.calibrate.deepen.ai/api/v2/external/datasets/{dataset_id}/extrinsic_parameters/{extrinsic_camera_coordinate_system}

Request

Path parameters

Response

{
    "dataset_id": "XXXXXXXXXXXXXXXXX",
    "calibration_algorithm_version": "XXXXXXXXXXXXXXXXX",
    "extrinsic_camera_coordinate_system": "OPTICAL",
    "extrinsic_parameters": {
        "roll": -90.47755237974575,
        "pitch": -0.38434110269976385,
        "yaw": -87.95967045393508,
        "px": 0.06958801619530329,
        "py": -0.28251980028661544,
        "pz": -0.010306058948604074
    },
    "error_stats": {
        "translation_error": 0.04085960836364045,
        "plane_translation_error": 0.00832952204,
        "rotation_error": 0.7512576778920595,
        "reprojection_error": 27.18615944133744
    },
    "projected_images": "XXXXXXXXXXurl_to_download_projected_imagesXXXXXX"
}