Create new dataset
Create a new dataset in a client.
** From 1st January 2020, one needs to call Process Upload API after this API call to complete preprocessing.
URL
POST https://tools.deepen.ai/api/v2/clients/{clientId}/datasets
Request
Path parameters
Parameter name | Parameter type | Description |
clientId | string | Client Id obtained from Deepen AI |
Request body
Property name | Property type | Description |
files | array | Array of metadata objects, each representing a file to be uploaded. Please see the example object below for the structure of the metadata object. For datasets of dataset_type ‘images’’, the metadata should be of a zip file of all images to be uploaded as part of the dataset. For datasets of dataset_type ‘video’, the metadata should be of an mp4 of the video to be uploaded as part of the dataset. |
dataset_format | string | The format in which the dataset is uploaded for ‘3d’ dataset types. Can take one of the following values :
Not required for dataset_type other than ‘3d’ |
dataset_type | string | The type of uploaded dataset:
|
dataset_name | string | Name of the uploaded dataset |
labelling_mode | string | Labelling format in the dataset:
|
labelling_profile_id | string | Labelling profile id This is the profile id of the profile from which to get the labelling instructions. |
labelling_profile | object | LabellingProfileResource indicating the labelling instructions. Either labelling_profile or labelling_profile_id is necessary to include labelling instructions with the dataset. |
tags | array | Array of TagResource |
user_scopes | array | Array of DatasetUserScopeResource |
user_group_scopes | array | Array of DatasetUserGroupScopeResource |
Example body
{ 'dataset_name': 'testDataset', 'dataset_type': '3d', 'fps': 10, 'labelling_profile': {'pipeline_spec': [{'stage_id': 'Labelling', 'can_edit': True, 'skills': {'box': {}, 'point': {}, 'polygon': {}, 'lane': {}, '3d_bbox': {}, '3d_point': {}, '3d_instance_point': {}, '3d_polygon': {}, '3d_polyline': {}}}, {'stage_id': 'QA', 'can_edit': False, 'skills': {'box': {}, 'point': {}, 'polygon': {}, 'lane': {}, '3d_bbox': {}, '3d_point': {}, '3d_instance_point': {}, '3d_polygon': {}, '3d_polyline': {}}}], 'features': {'box': {'attributes': {}, 'label_category_ids': []}, 'lane': {'attributes': {}, 'label_category_ids': []}, 'polygon': {'attributes': {}, 'label_category_ids': []}, 'point': {'attributes': {}, 'label_category_ids': []}, '3d_bbox': {'attributes': {}, 'label_category_ids': []}, '3d_polyline': {'attributes': {}, 'label_category_ids': []}, '3d_polygon': {'attributes': {}, 'label_category_ids': []}, '3d_point': {'attributes': {}, 'label_category_ids': []}, '3d_instance_point': {'attributes': {}, 'label_category_ids': []}}, 'instructions': {'text': ''}, 'notification_url': ''}, 'labelling_mode': 'frame_by_frame', 'dataset_format': 'default', 'sensor_maker': None, 'files': [{'file_size': 16513471, 'file_type': 'application/zip'}] } |
Response
Returns an array of resumable upload urls on successful completion
Response body
Property name | Property type | Description |
files | array | Array of metadata objects, each representing a file to be uploaded. Please see the example object below for the structure of the metadata object. Each metadata object has a Google cloud storage resumable upload url, to which the file has to be uploaded. Please see google cloud storage documentation at https://cloud.google.com/storage/docs/json_api/v2/how-tos/resumable-upload to learn how to upload the file using the url. |
dataset_id | string | The new dataset id created for the uploaded dataset |
Example
{ 'dataset_id': 'BBZztvi0Z9TC', 'files': [{'resumable_upload_url': 'https://www.googleapis.com/upload/storage/v2/b/deepenai-dev-rannotateclientdata/o?uploadType=resumable&name=rannotate_client_uploaded_data/8cz4s0r1/defaultproject/BBZztvi0Z9TC/3d_data.zip&upload_id=AAANsUnQh_s03FsS-iGuvfD9n3ABpmvLxV8-Yi86g-QwGrtntj3HfYE8e_l1iDbeKAp06MXNGqvbotpgti1n3yjQEF0', 'signed_form_data': None, 'file_size': 16513471, 'file_type': 'application/zip'}] } |
Please use the below command or follow this link to upload the data
curl -v -X PUT -H 'Expect:' --upload-file '<path to zip file>' '<resumable upload url>'
Once the dataset zip file is uploaded, you need to call the process uploaded data api to process the zip file and make it ready for labelling.
Last updated