How to upload dataset for JSON Point Cloud type project

The zip file should contain a set of JSON files and camera images. Each JSON file will correspond to one point cloud frame of the dataset. JSON files should be at the root directory level. The order of the frames is decided by order of JSON filename sorted in ascending order. For instance, filenames can be 0001.json, 0002.json, 0003.json, ... Filenames can also be 0.json, 1.json, 2.json, …

Each JSON file should be an object with the following 5 fields:

1. Images

2. Timestamp

3. Points

4. Device position

5. Device heading

1. Images

An images (array) - of all camera images corresponding to one point cloud frame. Usually, number of images corresponds to the number of cameras in the system. If there are no images, please use an empty array. Each element is a JSON object. Fields in image object are as follows:

1. fx (float) - focal length in x direction.

2. fy (float) - focal length in y direction.

3. cx (float) - x coordinate of principal point.

4. cy (float) - y coordinate of principal point.

5. timestamp (float) - time in seconds when image was captured.

6. image_url (string) - corresponds to an image path inside the zip file, e.g. “images/0001.png”. It can also be an external URL. Image types supported are .jpeg and .png.

7. position (object) - position of camera with respect to world frame. Details of JSON object can be found below.

8. heading (object) - orientation of camera with respect to world frame. Please find details of JSON object below.

9. camera_model (string) - the camera model to be used for undistorting the image. Supported values for camera_model : pinhole (default) - uses k1, k2, p1, p2, k3, k4 distortion coefficients fisheye - uses k1, k2, k3, k4 distortion coefficients

10. k1 (float) - distortion coefficient.

11. k2 (float) - distortion coefficient.

12. p1 (float) - distortion coefficient.

13. p2 (float) - distortion coefficient.

14. k3 (float) - distortion coefficient.

15. k4 (float) - distortion coefficient.

If images are already undistorted, k1, k2, p1, p2, etc. should be all 0's. You can find more details on the camera parameters here.

A sample image JSON is as following:

2. Timestamp

Timestamp (float) - time in seconds at which the point cloud frame was captured.

3. Points

Points can be given in 2 formats - first format is an array of JSON objects and the second format is base64 encoded strings of points and intensities. Points in JSON object array format: A Points array of JSON objects of all LiDAR points having their x, y, z, i, r, g, b values. x, y and z values are mandatory and i, r, g and b values are optional for each point. In general, the “up” direction towards the sky should be in the positive z direction for the visualization to work correctly. Each element of array is a JSON object as shown in this section. rgb value in xyzrgb type point will be supported in a future release. Each point can have other values like velocity, as well, for which we can add custom support. Fields in point object are as follows:

1. x (float) - x coordinate of the point, in meters.

2. y (float) - y coordinate of the point, in meters.

3. z (float) - z coordinate of the point, in meters.

x, y, z values are in world coordinates. If you are unable to put the point cloud in world coordinate, you can fall back to local LiDAR coordinate and let us know. We will contact you about the issue. A sample point JSON object is as follows:

Points as a base64 encoded string: In this compact format, points and intensities should be given as two separate base64 encoded strings of the 4 byte floating point values (float32) of coordinates or intensities. Each point coordinates x, y, z should be present in that order then followed by next point coordinates.

Sample format for two points [1.0, 2.0, 3.0], [4.0, 5.0, 6.0] with intensities 123 and 254 is as follows:

4. Device position

A device_position (object) - position of LiDAR or camera with respect to world frame. Similar to point cloud, if you are unable to put the device position in world coordinate, you can fall back to local LiDAR coordinate and let us know. We will contact you about the issue. For camera, if you do not have any position information, please use (0, 0, 0) and let us know. Fields in position object are as follows:

1. x (float) - x coordinate of device/camera position, in meters.

2. y (float) - y coordinate of device/camera position, in meters.

3. z (float) - z coordinate of device/camera position, in meters.

Sample position JSON object:

5. Device heading

A device_heading (object) - orientation parameters of LiDAR or camera with respect to world frame. If you are unable to put LiDAR heading in world coordinate, please use the identity quaternion (x = 0, y = 0, z = 0, w = 1). If you are unable to obtain extrinsic camera calibration parameters, please also use the identity quaternion. We will contact you about this issue. Fields in heading object are as follows, the 4 components are quaternions:

1. x (float) - x component of device/camera orientation.

2. y (float) - y component of device/camera orientation.

3. z (float) - z component of device/camera orientation.

4. w (float) - w component of device/camera orientation.

A sample heading JSON object is as follows:

Please note that in JSON, the order of the dictionary values of quaternion doesn't matter. Following two JSONs will give exact same result:

A sample JSON can be found below,

​

{
  "images": [
    {
      "fx": 561.997914,
      "timestamp": 1541186225.8394644,
      "p2": 0.00251,
      "k1": -0.142792,
      "p1": 0.001203,
      "k3": 0,
      "k2": 0.022846,
      "cy": 361.97667,
      "cx": 664.16411,
      "image_url": "0.png",
      "fy": 561.585651,
      "position": {
        "y": -152.77584902657554,
        "x": 311.21505956090624,
        "z": -10.854137529636024
      },
      "heading": {
        "y": -0.7046155108831117,
        "x": 0.034278837280808494,
        "z": 0.7070617895701465,
        "w": -0.04904659893885366
      }
      "camera_model": "pinhole"
    },
    {
      "fx": 537.74122,
      "timestamp": 1541186225.8499014,
      "p2": -0.000507,
      "k1": -0.133161,
      "p1": -0.0007,
      "k3": 0,
      "k2": 0.020764,
      "cy": 353.596887,
      "cx": 687.798477,
      "image_url": "1.png",
      "fy": 541.411032,
      "position": {
        "y": -152.7458074214421,
        "x": 311.168923367011,
        "z": -10.855340458227541
      },
      "heading": {
        "y": -0.571381519522144,
        "x": -0.4283386878183726,
        "z": 0.5635977900941452,
        "w": 0.4152188081814165
      }
    }
  ],
  "timestamp": 1541186225.848686,
  "device_heading": {
    "y": -0.014390929501214435,
    "x": -0.006511549504752948,
    "z": -0.8798010889637369,
    "w": 0.4750795141124911
  },
  "points": [
    {
      "y": -147.6858459726749,
      "x": 319.51523557465174,
      "z": -11.55716049374703,
      "i": 0.32
    },
    {
      "y": -147.709804574419,
      "x": 319.5387083489352,
      "z": -11.559980704176585,
      "i": 0.32
    },
    {
      "y": -147.8861052361809,
      "x": 319.3026396838094,
      "z": -11.536266484496409,
      "i": 0.32
    }
  ],
  "device_position": {
    "y": -152.4309172401979,
    "x": 311.42759643080274,
    "z": -11.704321251954227
  }
}

Another sample:

​

{
  "images": [
    {
      "fx": 561.997914,
      "timestamp": 1541186225.8394644,
      "p2": 0.00251,
      "k1": -0.142792,
      "p1": 0.001203,
      "k3": 0,
      "k2": 0.022846,
      "cy": 361.97667,
      "cx": 664.16411,
      "image_url": "camera_1.jpg",
      "fy": 561.585651,
      "position": {
        "y": -152.77584902657554,
        "x": 311.21505956090624,
        "z": -10.854137529636024
      },
      "heading": {
        "y": -0.7046155108831117,
        "x": 0.034278837280808494,
        "z": 0.7070617895701465,
        "w": -0.04904659893885366
      }
    },
{
      Another 5 cameras will have similar settings then followed by 
        "heading": {
        "y": -0.3485952869673183,
        "x": -0.7973743859088446,
        "z": 0.1897018750524244,
        "w": 0.45463019389638054
      }
    },
    {
      "fx": 354.848025,
      "timestamp": 1541186225.832587,
      "cy": 331.448458,
      "cx": 812.600868,
      "image_url": "another_camera_1.jpg",
      "fy": 356.521296,
      "position": {
        "y": -152.11720409699208,
        "x": 311.6143516201334,
        "z": -10.845064295267472
      },
      "heading": {
        "y": 0.2603210255938851,
        "x": -0.8843903286926823,
        "z": -0.11886794907638565,
        "w": 0.36872363747252623
      }
    }
  ],
  "timestamp": 1541186225.848686,
  "device_heading": {
    "y": -0.014390929501214435,
    "x": -0.006511549504752948,
    "z": -0.8798010889637369,
    "w": 0.4750795141124911
  },
  "points": [
    {
      "y": -147.6858459726749,
      "x": 319.51523557465174,
      "z": -11.55716049374703,
      "i": 0.32
    },
    {
      "y": -147.709804574419,
      "x": 319.5387083489352,
      "z": -11.559980704176585,
      "i": 0.32
    },
    {
      "y": -147.8861052361809,
      "x": 319.3026396838094,
      "z": -11.536266484496409,
      "i": 0.32
    },
    {
      "y": -147.97751024914913,
      "x": 319.21297474760536,
      "z": -11.527578302408955,
      "i": 0.32
    },
    ....
​
   ],
  "device_position": {
    "y": -152.4309172401979,
    "x": 311.42759643080274,
    "z": -11.704321251954227
  }
}