检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
ModelArts data management provides the following functions for you to obtain high-quality AI data: Data acquisition Allows you to import data from OBS, MRS, DLI, and GaussDB(DWS). Provides 18+ data augmentation operators to increase data volume for training.
Overview End-to-End O&M Process During algorithm development, store service data in Object Storage Service (OBS), and then label and manage the data using ModelArts data management.
Do not delete any objects from the OBS directory while MoXing is downloading them. This will cause the download to fail.
If the model is imported from OBS, the requirements on the body are reflected in inference code preprocessing, which will convert the input HTTP body into the input required by the model. For details, see Specifications for Model Inference Coding.
OBS Not supported Not supported Method 1: Modify the Service Information on the Service Management Page Log in to the ModelArts console and choose Model Deployment from the navigation pane. Go to the service management page of the target service.
NOTE: Creating an auto labeling job is free, but you will be billed for OBS storage based on usage. For details, see Product Pricing Details. To avoid wasting resources, clear your OBS bucket after labeling jobs and their subsequent tasks are complete.
Figure 1 Obtaining images OBS Object Storage Service (OBS) is a cloud storage service optimized for storing massive amounts of data. It provides unlimited, secure, and highly reliable storage capabilities at a relatively low cost. ModelArts exchanges data with OBS.
The work directory of a table dataset cannot be an OBS path in a KMS-encrypted bucket. work_path Yes Table 6 Output dataset path, which is used to store output files such as label files. labels No List of Table 7 Dataset labels.
In this way, the logs will be automatically stored in the specified OBS path. If a training job is created on Ascend compute nodes, certain system logs cannot be downloaded in the training log pane.
If you need to use the OBS SDK, use ModelArts SDK instead to copy files. For details, see Transferring Files.
To meet the routine training requirements of algorithm engineers, SFS and OBS can be used to store and read data. Public Resource Pool: provides large-scale public computing clusters, which are allocated based on job parameter settings. Resources are isolated by job.
resource ID of a visualization job job_id Long ID of a visualization job job_desc String Description of a visualization job duration Long Visualization job running duration, in milliseconds create_time Long Time when a visualization job is created, in timestamp format train_url String OBS
Result data can be asynchronously exported to associated OBS for long-term, low-cost storage, thereby accelerating data access in training scenarios in OBS.
You can use SFS or OBS for data storage and retrieval operations, meeting the needs of algorithm engineers for daily training. Refer to Elastic BMS Lite Server. ModelArts Lite Cluster is tailored for users focused on Kubernetes resources.
After the writing is completed, upload the file to the specified OBS directory. If the meta model is from a container image, ensure the size of the meta model complies with Restrictions on the Size of an Image for Importing an AI Application.
By default, this parameter is left blank. src_path Yes String OBS path of the input data of a batch job dest_path Yes String OBS path of the output data of a batch job req_uri Yes String Inference API called in batch tasks.
Call the API for creating a ModelArts agency to create an agency for ModelArts-dependent services, such as OBS, SWR, and IEF.
Connecting to a Notebook Instance Through PyCharm Toolkit OBS-based upload and download Local files or folders can be uploaded to OBS and files or folders can be downloaded from OBS to a local directory.
Users granted with these permissions can also access OBS and SWR of all users in the current IAM project.
By default, this parameter is left blank. inputs No String OBS storage path of a training job dataset_id No String Dataset ID of a training job.
