检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Prepare the tenant ID and IAM ID for OBS bucket configuration. Send the prepared information to Huawei technical support, who will configure an OBS bucket policy based on your information. You can upload the collected logs to the corresponding OBS bucket.
The value can be OBS or PFS. Storage Path: Set the OBS path for storing notebook data. If you want to use existing files or data, upload them to the specified OBS path.
Options: If type is set to OBS, source is an OBS path. If type is set to DATASET, source is a dataset ID. type String Type of a working path. Options: OBS: OBS path DATASET: dataset version_id String Version of a dataset. version_name String Name of a dataset version.
Obtain the manifest files of the two datasets from the OBS path set for Output Dataset Path. Create an empty dataset C and select an empty OBS folder for Input Dataset Path. Import the manifest files of datasets A and B to dataset C.
<obs path> can be obs:// or s3://. Share link generated by OBS, including signature information. It applies to accessing OBS data of other users. The link has a validity period. Perform operations within the period.
# Define a dataset. dataset = wf.data.DatasetPlaceholder(name="input_dataset") # Define OBS data. obs = wf.data.OBSPlaceholder(name = "obs_placeholder_name", object_type = "directory" ) # object_type must be file or directory.
Verify OBS permissions. In the service list in the upper left corner, select OBS. The OBS management console is displayed. Click Create Bucket in the upper right corner. If this operation is successful, you have obtained OBS operation permissions. Verify SWR permissions.
Select an OBS path as Input Dataset PathImport Path, and select another OBS path as Output Dataset Path. Figure 1 Data import and output paths After setting the parameters, click CreateSubmit in the lower right corner to create a dataset.
Configuring Pipelines A preset image-based algorithm obtains data from an OBS bucket or dataset for model training. The training output is stored in an OBS bucket.
Options: If type is set to OBS, source is an OBS path. If type is set to DATASET, source is a dataset ID. type String Type of a working path. Options: OBS: OBS path DATASET: dataset version_id String Version of a dataset. version_name String Name of a dataset version.
Prerequisites The data used for training has been uploaded to an OBS directory. At least one empty folder has been created in OBS for storing the training output. OBS buckets are not encrypted. ModelArts does not support encrypted OBS buckets.
For details about how to upload a file using OBS Browser, see Uploading a File. Use MoXing to copy the source code uploaded to OBS to a notebook instance in the development environment.
Figure 3 Example Copying Data to OBS On the Notebook JupyterLab page, copy the yolov8_train_ascend.zip file to an OBS bucket.
The value must be a specific directory in an OBS bucket rather than the root directory of an OBS bucket. If type is set to obsfs, the parameter value must be a valid bucket name of the OBS parallel file system. (CCE does not support subdirectory mounting.)
MapReduce Service (MRS) MRS Administrator GaussDB(DWS) DWS Administrator Cloud Trace Service (CTS) CTS Administrator ModelArts ModelArts CommonOperations ModelArts Dependency Access Development environment notebook/Image management/Elastic node server OBS OBS Administrator Cloud
OBS Users are agnostic to it. Yearly/Monthly Pay-per-use Token-based billing ModelArts Standard NPU GPU Users are agnostic to it. Public resource pool: OBS Dedicated resource pool: SFS Turbo and OBS Users are agnostic to it.
The path can be either a local or OBS path.
Delete the OBS bucket and data stored in OBS. Real-time service: Stop the real-time service. Delete the OBS bucket and data stored in OBS. Dedicated resource pool: Delete the dedicated resource pool.
Exporting Data ModelArts training management allows you to create training jobs using ModelArts datasets or files in an OBS directory. If you create a training job using an OBS directory, export the prepared data to OBS. Export data to OBS.
Model Management Failed to Create a Model Suspended Account or Insufficient Permission to Import Models Failed to Build an Image or Import a File During Model Creation Failed to Obtain the Directory Structure in the Target Image When Creating a Model Through OBS Failed to Obtain Certain
