Create new dataset

Create a new dataset in a client.

** From 1st January 2020, one needs to call Process Upload API after this API call to complete preprocessing.

URL

POST https://tools.deepen.ai/api/v2/clients/{clientId}/datasets

Request

Path parameters

Parameter name	Parameter type	Description
clientId	string	Client Id obtained from Deepen AI

Request body

Property name	Property type	Description
files	array	Array of metadata objects, each representing a file to be uploaded. Please see the example object below for the structure of the metadata object. For datasets of dataset_type ‘images’’, the metadata should be of a zip file of all images to be uploaded as part of the dataset. For datasets of dataset_type ‘video’, the metadata should be of an mp4 of the video to be uploaded as part of the dataset.
dataset_format	string	The format in which the dataset is uploaded for ‘3d’ dataset types. Can take one of the following values : json default Not required for dataset_type other than ‘3d’
dataset_type	string	The type of uploaded dataset: 3d video images
dataset_name	string	Name of the uploaded dataset
labelling_mode	string	Labelling format in the dataset: label_by_label (For sequence datasets) frame_by_frame (For single frame datasets - default)
labelling_profile_id	string	Labelling profile id This is the profile id of the profile from which to get the labelling instructions.
labelling_profile	object	LabellingProfileResource indicating the labelling instructions. Either labelling_profile or labelling_profile_id is necessary to include labelling instructions with the dataset.
tags	array	Array of TagResource
user_scopes	array	Array of DatasetUserScopeResource
user_group_scopes	array	Array of DatasetUserGroupScopeResource

Example body

{ 'dataset_name': 'testDataset', 'dataset_type': '3d', 'fps': 10, 'labelling_profile': {'pipeline_spec': [{'stage_id': 'Labelling', 'can_edit': True, 'skills': {'box': {}, 'point': {}, 'polygon': {}, 'lane': {}, '3d_bbox': {}, '3d_point': {}, '3d_instance_point': {}, '3d_polygon': {}, '3d_polyline': {}}}, {'stage_id': 'QA', 'can_edit': False, 'skills': {'box': {}, 'point': {}, 'polygon': {}, 'lane': {}, '3d_bbox': {}, '3d_point': {}, '3d_instance_point': {}, '3d_polygon': {}, '3d_polyline': {}}}], 'features': {'box': {'attributes': {}, 'label_category_ids': []}, 'lane': {'attributes': {}, 'label_category_ids': []}, 'polygon': {'attributes': {}, 'label_category_ids': []}, 'point': {'attributes': {}, 'label_category_ids': []}, '3d_bbox': {'attributes': {}, 'label_category_ids': []}, '3d_polyline': {'attributes': {}, 'label_category_ids': []}, '3d_polygon': {'attributes': {}, 'label_category_ids': []}, '3d_point': {'attributes': {}, 'label_category_ids': []}, '3d_instance_point': {'attributes': {}, 'label_category_ids': []}}, 'instructions': {'text': ''}, 'notification_url': ''}, 'labelling_mode': 'frame_by_frame', 'dataset_format': 'default', 'sensor_maker': None, 'files': [{'file_size': 16513471, 'file_type': 'application/zip'}] }

Response

Returns an array of resumable upload urls on successful completion

Response body

Property name	Property type	Description
files	array	Array of metadata objects, each representing a file to be uploaded. Please see the example object below for the structure of the metadata object. Each metadata object has a Google cloud storage resumable upload url, to which the file has to be uploaded. Please see google cloud storage documentation at https://cloud.google.com/storage/docs/json_api/v2/how-tos/resumable-upload to learn how to upload the file using the url.
dataset_id	string	The new dataset id created for the uploaded dataset

Example

{ 'dataset_id': 'BBZztvi0Z9TC', 'files': [{'resumable_upload_url': 'https://www.googleapis.com/upload/storage/v2/b/deepenai-dev-rannotateclientdata/o?uploadType=resumable&name=rannotate_client_uploaded_data/8cz4s0r1/defaultproject/BBZztvi0Z9TC/3d_data.zip&upload_id=AAANsUnQh_s03FsS-iGuvfD9n3ABpmvLxV8-Yi86g-QwGrtntj3HfYE8e_l1iDbeKAp06MXNGqvbotpgti1n3yjQEF0', 'signed_form_data': None, 'file_size': 16513471, 'file_type': 'application/zip'}] }

Please use the below command or follow this link to upload the data

curl -v -X PUT -H 'Expect:' --upload-file '<path to zip file>' '<resumable upload url>'

Once the dataset zip file is uploaded, you need to call the process uploaded data api to process the zip file and make it ready for labelling.