Data Input Format-3D PCD upload(Advanced users only)
Dataset Upload - PCD
In general, we found that the json format is easier to start. The 2 formats end up the same in our system, but the json format is more verbose and users have fewer problems with it. We recommend it for new users: How to upload dataset for JSON Point Cloud type project
To create a 3d dataset using the PCD dataset format, a zip file needs to be uploaded to the tool. The zip file should contain a set of json files namely config.json, lidar_timestamps.json, lidar_transforms.json, view_matrices.json; a text file named lidar_counts.txt and folders for images and lidar data. The content of each of these is described below.

Config.json (required)

This is the configuration file for the dataset. This contains the information about the primary sensor, cameras, the view matrix for each camera and the directory the images and Lidar files are present in. A sample config.json data is given below:
{
"primary_sensor_id": "lidar",
"sensors": {
"lidar": {
"content": "lidar",
"sensor_type": "lidar",
"sensor_fusion": {
"camera_0": {
"view_matrix": [
[1.0, 0.0, 0.0, 0.0],
[0.0, 1.0, 0.0, 0.0],
[0.0, 0.0, 0.0, 1.0],
[0.0, 0.0, 1.0, 0.0]
]
},
"camera_1": {
"view_matrix": [
[1.0, 0.0, 0.0, 0.0],
[0.0, 1.0, 0.0, 0.0],
[0.0, 0.0, 0.0, 1.0],
[0.0, 0.0, 1.0, 0.0]
]
}
}
},
"camera_0": {
"content": "images/camera_0_img",
"sensor_type": "camera",
"extension": "jpg"
},
"camera_1": {
"content": "images/camera_1_img",
"sensor_type": "camera",
"extension": "jpg"
}
}
}
The description of each field and its content is given below
  • primary_sensor_id: the name/id of the lidar sensor
  • sensors: an object containing the details of all the sensors (lidar and cameras) present in the dataset. Each key in the ‘sensors’ object depicts the sensor_id of the sensor, and the values contain an object with the below information.
  • content: the relative path of the folder where the data for the sensor is present. A point to note here is that the image names should follow the convention {pcd_file_name}.pcd.jpg or {pcd_file_name}.pcd.png. We use this convention to map the image file with the pcd file.
  • sensor_type: type of the sensor. Supported values are ‘camera’ and ‘lidar’.
  • extension : valid only for sensor_type ‘camera’, this represents the extension/format of the image files present in the data. Supported formats are ‘jpg’ and ‘png’.
  • sensor_fusion valid only for sensor_type ‘lidar’, this contains the transformations used to transform the points in lidar coordinates to respective camera coordinates. Each key in the sensor_fusion object depicts the sensor_id of the camera and the value contains an object with the below information:
  • view_matrix: the transformation matrix used to transform the points from lidar coordinates to image coordinates (sometimes also referred as calibration matrix). If the points are present in world coordinates (a global frame of reference) and the vehicle is not stationary, then the frame level view_matrices can be provided in view_matrices.json. The view_matrix from view_matrices.json will override the one present in config.json

Images folder/s (optional)

Images folders contain the images for the dataset. The image folders should follow the path provided in the ‘content’ field in config.json. The supported formats are png and jpg. The image name needs to follow the naming convention {pcd_file_name}.pcd.{jpg/png} This naming is used to map the image files with the correct pcd file.

Lidar folder (required)

Lidar folder contains the pcd files for the dataset. This folder should follow the path provided in the ‘content’ field in config.json. We support xyz, xyzi and xyzrgb point clouds in the pcd.

lidar_transforms.json (optional)

This file contains the transformation matrices used to convert the lidar points in the lidar (local) frame of reference to the world (global) frame of reference. The json object has two keys ‘forward_transforms’ and ‘inverse_transforms’. The key ‘forward_transforms’ contains the transformation matrix used to transform the points in the lidar frame to world frame for each frame index. The key ‘inverse_transforms’ contains the inverse of these transformations.

lidar_timestamps.json (optional)

This file contains the timestamp of each lidar frame with frame index as the key and timestamp as the value.

view_matrices.json (optional)

This file contains the view_matrices of each camera for each frame. The json object has frame index as the key and the value similar to sensor_fusion object as mentioned in the config.json
If your vehicle is stationary, the view_matrix has the intrinsic and extrinsic parameters combined. (No distortion parameters allowed.)
The camera coordinate frame is the optical one. If you're using REP103 (ENU), it needs to be converted to the ROS optical frame. Do you need the matrix for this conversion?
The camera is the parent frame and the lidar is the child frame. The extrinsic matrix converts from the lidar frame to the camera frame. If you have it the opposite way, you'd need to invert the entire 4x4 matrix
The product is a 3x4 matrix. We just add a [0 0 0 1] row at the bottom to make the view_matrix 4x4.
view_matrix = camera_intrinsic_matrix * extrinsic_calibration_matrix
Input camera intrinsic matrix
extrinsic_calibration_matrix
If your vehicle is moving, there is one view-matrix per camera per frame. Each view matrix is the product of intrinsic, extrinsic, and localization matrices.
Here is an example for view matrices.json file:
view_matrices.json
30KB
Code

lidar_counts.txt (optional)

This file contains the point count of each lidar file in a text format. Each line of text file represents the points’ count for each frame index.
Here is an example for PCD format:
PCD format.zip
65MB
Binary
Last modified 1mo ago