检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
With OBS, you can access various tools like SDK, API, OBS console, and OBS Browser. ModelArts mox.file offers a set of APIs that mimic a local file system, enabling easy management of OBS files.
--data-url TEXT OBS path for training data. --log-url TEXT OBS path for training log. --code-dir TEXT OBS path for source code. --output TEXT Training output parameter with OBS path.
If the summary file is stored in OBS, you will be charged for the storage. After a job is complete, stop the notebook instance and clear OBS data to stop billing.
OBS ModelArts uses Object Storage Service (OBS) to securely and reliably store data and models at low costs. For more information about OBS, see Object Storage Service Documentation.
modelarts:processTask:getProcessTask obs:bucket:HeadBucket obs:bucket:ListBucket obs:object:PutObject obs:object:PutObject obs:bucket:ListAllMyBuckets obs:bucket:PutBucketAcl obs:bucket:GetBucketAcl obs:bucket:GetBucketPolicy obs:bucket:DeleteBucketPolicy √ √ Creating a Data Processing
Insufficient OBS permissions For details, see Error "stat:403 reason:Forbidden" Is Displayed in Logs When a Training Job Accesses OBS. OBS traffic limiting For details, see Error Message "BrokenPipeError: Broken pipe" Is Displayed When OBS Data Is Copied.
for heterogeneous jobs Table 30 remote Parameter Type Description obs obs object OBS in which data input and output are stored Table 31 obs Parameter Type Description obs_url String OBS URL of the dataset for a training job, for example, /usr/data/ Table 32 outputs Parameter Type
Service deployment The models stored in OBS can be deployed as services. Settings - Authorizes ModelArts to access OBS (using an agency or access key) so that ModelArts can use OBS to store data and create notebook instances.
OBS buckets and folders for storing data are available. In addition, the OBS buckets and ModelArts are in the same region. OBS parallel file systems are not supported. Select object storage. OBS buckets are not encrypted. ModelArts does not support encrypted OBS buckets.
Use OBS or ModelArts SDK to download the file from OBS to the local path. Method 1: Use OBS to download the file. Download obs_file.txt from OBS to the local path. If a large amount of data is to be downloaded, use OBS Browser+ to download.
Model Management Permissions Table 1 Fine-grained permissions for model management Permission API Action Related Action IAM Project Enterprise Project Importing a Model POST /v1/{project_id}/models modelarts:model:create obs:bucket:ListAllMybuckets obs:object:PutObjectAcl obs:bucket
The possible causes are as follows: You have uploaded data to OBS for storage when using ModelArts, and the OBS storage is billed. In this case, go to the OBS management console and delete the data, folders, and OBS buckets that are no longer needed.
Log in to the OBS console, access your OBS bucket, and delete the data stored in the OBS bucket. After these operations are complete, the billing is stopped. Parent topic: Billing FAQs
Client Overview of OBS Management How to call the SDK APIs of Object Storage Service (OBS), including the APIs for creating OBS buckets, uploading and downloading files and folders, as well as deleting OBS objects and buckets ModelArts SDK operations: Data Management Training Management
The possible causes are as follows: You have uploaded data to OBS for storage when using ModelArts, and the OBS storage is billed. In this case, go to the OBS management console and delete the data, folders, and OBS buckets that are no longer needed.
Solution Use the current account to log in to the OBS console and check whether the current account can access OBS. If the account can access OBS, rectify the fault by referring to 2. If the account cannot access OBS, rectify the fault by referring to 3.
Checking Whether the Affected Path Is an OBS Path When using ModelArts, store data in an OBS bucket. However, the OBS path cannot be used to read data during the execution of the training code.
OBS: Import data from an OBS path or a manifest file. Local file: Import local data that has been uploaded to an OBS path. Table Data Sources You can import data from OBS, MRS, and local files. Import Mode There are five modes for importing data to a dataset.
If the folder exists in the OBS bucket, the upload is successful. import moxing as mox mox.file.copy_parallel('/home/ma-user/work/sub_dir_0', 'obs://bucket_name/sub_dir_0') Use OBS or ModelArts SDK to download the file from OBS to the local path.
For details, see How Do I Upload a File from a Notebook Instance to OBS or Download a File from OBS to a Notebook Instance? Parent topic: Code Running Failures
