How to upload dataset for PCD Point Cloud type project

To create a 3d dataset using the PCD dataset format, a zip file needs to be uploaded to the tool. The zip file should contain a set of JSON files namely config.json, lidar_timestamps.json, lidar_transforms.json, view_matrices.json; a text file named lidar_counts.txt and folders for images and lidar data. The content of each of these is described below.

Config.json (required)

This is the configuration file for the dataset. This contains the information about the primary sensor, cameras, the view matrix for each camera, and the directory the images and lidar files are present in. A sample config.json data is given below:

{

"primary_sensor_id": "lidar",

"sensors": {

"lidar": {

"content": "lidar",

"sensor_type": "lidar",

"sensor_fusion": {

"camera_0": {

"view_matrix": [

[1.0, 0.0, 0.0, 0.0],

[0.0, 1.0, 0.0, 0.0],

[0.0, 0.0, 0.0, 1.0],

[0.0, 0.0, 1.0, 0.0]

]

},

"camera_1": {

"view_matrix": [

[1.0, 0.0, 0.0, 0.0],

[0.0, 1.0, 0.0, 0.0],

[0.0, 0.0, 0.0, 1.0],

[0.0, 0.0, 1.0, 0.0]

]

}

}

},

"camera_0": {

"content": "images/camera_0_img",

"sensor_type": "camera",

"extension": "jpg"

},

"camera_1": {

"content": "images/camera_1_img",

"sensor_type": "camera",

"extension": "jpg"

}

}

}

The description of each field and its content is given below

primary_sensor_id: the name/id of the lidar sensor

sensors: an object containing the details of all the sensors (lidar and cameras) present in the dataset. Each key in the ‘sensors’ object depicts the sensor_id of the sensor and the values contain an object with the below information.

content: The relative path of the folder where the data for the sensor is present. A point to note here is that the image names should follow the convention {pcd_file_name}.pcd.jpg or {pcd_file_name}.pcd.png. We use this convention to map the image file with the pcd file.

sensor_type: type of the sensor. Supported values are ‘camera’ and ‘lidar’.

extension: valid only for sensor_type ‘camera’, this represents the extension/format of the image files present in the data. Supported formats are ‘jpg’ and ‘png’.

sensor_fusion: valid only for sensor_type ‘lidar’, this contains the transformations used to transform the points in lidar coordinates to respective camera coordinates. Each key in the sensor_fusion object depicts the sensor_id of the camera and the value contains an object with the below information:

view_matrix: the transformation matrix used to transform the points from lidar coordinates to image coordinates (sometimes also referred to as calibration matrix). If the points are present in world coordinates (a global frame of reference) and the vehicle is not stationary, then the frame-level view_matrices can be provided in view_matrices.json. The view_matrix from view_matrices.json will override the one present in config.json

Images folder/s (optional)

Images folders contain the images for the dataset. The image folders should follow the path provided in the ‘content’ field in config.json. The supported formats are png and jpg. The image name needs to follow the naming convention {pcd_file_name}.pcd.{jpg/png} This naming is used to map the image files with the correct pcd file.

Lidar folder (required)

Lidar folder contains the pcd files for the dataset. This folder should follow the path provided in the ‘content’ field in config.json. We support xyz, xyzi and xyzrgb point clouds in the pcd.

lidar_transforms.json (optional)

This file contains the transformation matrices used to convert the lidar points in the lidar (local) frame of reference to the world (global) frame of reference. The JSON object has two keys ‘forward_transforms’ and ‘inverse_transforms’. The key ‘forward_transforms’ contains the transformation matrix used to transform the points in the lidar frame to the world frame for each frame index. The key ‘inverse_transforms’ contains the inverse of these transformations.

lidar_timestamps.json (optional)

This file contains the timestamp of each lidar frame with frame index as the key and timestamp as the value.

view_matrices.json (optional)

This file contains the view_matrices of each camera for each frame. The JSON object has frame index as the key and the value similar to the sensor_fusion object as mentioned in the config.json

lidar_counts.txt (optional)

This file contains the point count of each lidar file in a text format. Each line of text file represents the points’ count for each frame index.