Datasets

Datasets

Get all datasets

Returns all the existing datasets in a client.

URL

GET https://tools.deepen.ai/api/v2/clients/{clientId}/datasets

Request

Path parameters

Parameter name

Parameter type

Description

clientId

string

Client Id obtained from Deepen AI

Response

Returns a json with each field representing a dataset Id and its corresponding value being a DatasetResource.

Get all datasetIds

Returns all the existing datasetIds in a client.

URL GET https://tools.deepen.ai/api/v2/clients/{clientId}/datasets_ids

Request

Path parameters

Parameter name

Parameter type

Description

clientId

string

Client Id obtained from Deepen AI

Response

Returns a json with key “datasets” and value as a list of dataset details object.

Dataset details object has field names ‘dataset_id’ and ‘dataset_name’ with corresponding values.

Response Object:

{“datasets”:

[

{“dataset_id”: “testdatasetId1”, “dataset_name”:”testdatasetname1”},

{“dataset_id”: “testdatasetId1”, “dataset_name”:”testdatasetname1”}

]

}

Get dataset

Get the metadata related to a dataset.

URL

GET https://tools.deepen.ai/api/v2/datasets/{datasetId}

Request

Path parameters

Parameter name

Parameter type

Description

datasetId

string

Dataset Id of the dataset

**Note: To get the dataset details of old format datasets (created using v1 APIs) using this api please pass the dataset_id as client_id:project_id:dataset_id

Response

Returns a DatasetResource object.

Delete dataset

Delete a dataset

URL

DELETE https://tools.deepen.ai/api/v2/datasets/{datasetId}

Request

Path parameters

Parameter name

Parameter type

Description

datasetId

string

Dataset Id obtained of the dataset

Response

Returns empty response on successful deletion.

Create new dataset

Create a new dataset in a client.

** From 1st January 2020, one needs to call Process Upload API after this API call to complete preprocessing.

URL

POST https://tools.deepen.ai/api/v2/clients/{clientId}/datasets

Request

Path parameters

Parameter name

Parameter type

Description

clientId

string

Client Id obtained from Deepen AI

Request body

Property name

Property type

Description

files

array

Array of metadata objects, each representing a file to be uploaded. Please see the example object below for the structure of the metadata object. For datasets of dataset_type ‘images’’, the metadata should be of a zip file of all images to be uploaded as part of the dataset. For datasets of dataset_type ‘video’, the metadata should be of an mp4 of the video to be uploaded as part of the dataset.

dataset_format

string

The format in which the dataset is uploaded for ‘3d’ dataset types. Can take one of the following values :

  1. json

  2. default

Not required for dataset_type other than ‘3d’

dataset_type

string

The type of uploaded dataset:

  • 3d

  • video

  • images

dataset_name

string

Name of the uploaded dataset

labelling_mode

string

Labelling format in the dataset:

  • label_by_label (For sequence datasets)

  • frame_by_frame (For single frame datasets - default)

labelling_profile_id

string

Labelling profile id

This is the profile id of the profile from which to get the labelling instructions.

labelling_profile

object

LabellingProfileResource indicating the labelling instructions. Either labelling_profile or labelling_profile_id is necessary to include labelling instructions with the dataset.

tags

array

Array of TagResource

user_scopes

array

Array of DatasetUserScopeResource

user_group_scopes

array

Array of DatasetUserGroupScopeResource

Example body

{ 'dataset_name': 'testDataset',

'dataset_type': '3d',

'fps': 10,

'labelling_profile': {'pipeline_spec': [{'stage_id': 'Labelling', 'can_edit': True, 'skills': {'box': {}, 'point': {}, 'polygon': {}, 'lane': {}, '3d_bbox': {}, '3d_point': {}, '3d_instance_point': {}, '3d_polygon': {}, '3d_polyline': {}}}, {'stage_id': 'QA', 'can_edit': False, 'skills': {'box': {}, 'point': {}, 'polygon': {}, 'lane': {}, '3d_bbox': {}, '3d_point': {}, '3d_instance_point': {}, '3d_polygon': {}, '3d_polyline': {}}}], 'features': {'box': {'attributes': {}, 'label_category_ids': []}, 'lane': {'attributes': {}, 'label_category_ids': []}, 'polygon': {'attributes': {}, 'label_category_ids': []}, 'point': {'attributes': {}, 'label_category_ids': []}, '3d_bbox': {'attributes': {}, 'label_category_ids': []}, '3d_polyline': {'attributes': {}, 'label_category_ids': []}, '3d_polygon': {'attributes': {}, 'label_category_ids': []}, '3d_point': {'attributes': {}, 'label_category_ids': []}, '3d_instance_point': {'attributes': {}, 'label_category_ids': []}}, 'instructions': {'text': ''}, 'notification_url': ''},

'labelling_mode': 'frame_by_frame',

'dataset_format': 'default',

'sensor_maker': None,

'files': [{'file_size': 16513471, 'file_type': 'application/zip'}] }

Response

Returns an array of resumable upload urls on successful completion

Response body

Property name

Property type

Description

files

array

Array of metadata objects, each representing a file to be uploaded. Please see the example object below for the structure of the metadata object. Each metadata object has a Google cloud storage resumable upload url, to which the file has to be uploaded. Please see google cloud storage documentation at https://cloud.google.com/storage/docs/json_api/v2/how-tos/resumable-upload to learn how to upload the file using the url.

dataset_id

string

The new dataset id created for the uploaded dataset

Example

{

'dataset_id': 'BBZztvi0Z9TC',

'files': [{'resumable_upload_url': 'https://www.googleapis.com/upload/storage/v2/b/deepenai-dev-rannotateclientdata/o?uploadType=resumable&name=rannotate_client_uploaded_data/8cz4s0r1/defaultproject/BBZztvi0Z9TC/3d_data.zip&upload_id=AAANsUnQh_s03FsS-iGuvfD9n3ABpmvLxV8-Yi86g-QwGrtntj3HfYE8e_l1iDbeKAp06MXNGqvbotpgti1n3yjQEF0', 'signed_form_data': None, 'file_size': 16513471, 'file_type': 'application/zip'}] }

Files

Get all files

Returns all the files in a dataset.

URL

GET https://tools.deepen.ai/api/v2/datasets/{datasetId}/files

Request

Path parameters

Parameter name

Parameter type

Description

datasetId

string

Dataset Id of the dataset

Response

Returns a json with field ‘files’ that represents an array of FileResource objects.

Get file

Get the metadata related to a file.

URL

GET https://tools.deepen.ai/api/v2/datasets/{datasetId}/files/{fileId}

Request

Path parameters

Parameter name

Parameter type

Description

datasetId

string

Dataset Id of the file

fileId

string

File Id of the file

Response

Returns a FileResource object.