检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
If ModelArts is no longer used, stop or delete the services running in ModelArts and delete the data stored in OBS and EVS. Clearing Storage Data ModelArts data is stored in OBS. To stop storage billing, switch to the OBS console and delete the data and directories.
environment variables Variable Description Example S3_ENDPOINT OBS endpoint N/A S3_VERIFY_SSL Whether to use SSL to access OBS S3_VERIFY_SSL=0 S3_USE_HTTPS Whether to use HTTPS to access OBS S3_USE_HTTPS=1 Table 5 Environment variables of the pip source and API Gateway address Variable
Reference ModelArts CLI Commands (Optional) Installing ma-cli Locally Autocompletion for ma-cli Commands ma-cli Authentication ma-cli image Commands for Building Images ma-cli ma-job Commands for Training Jobs ma-cli dli-job Commands for Submitting DLI Spark Jobs Using ma-cli to Copy OBS
The value must be a specific directory in an OBS bucket rather than the root directory of an OBS bucket. If type is set to obsfs, the parameter value must be a valid bucket name of the OBS parallel file system. (CCE does not support subdirectory mounting.)
The recommended method is to use the OBS path. OBS path (recommended) Call the copy_parallel API of MoXing to copy the corresponding OBS path.
The recommended method is to use the OBS path. OBS path (recommended) Call the copy_parallel API of MoXing to copy the corresponding OBS path.
Code Directory OBS directory where the training code file is stored. Configure this parameter only if your custom image does not contain training code. Upload code to the OBS bucket beforehand.
Synchronize New Data: You can upload images to the OBS directory specified during project creation and click Synchronize New Data to quickly add the new images in the OBS directory to ModelArts.
To delete the files, delete them in the OBS parallel file system.
/YOLOX obs://your_bucket/ -f -r Log in to the ECS and use obsutil to migrate the dataset to SFS. The sample code is as follows: # Upload code from OBS to SFS. ./obsutil cp obs://your_bucket/YOLOX/ /mnt/sfs_turbo/code/ -f -r In this example, obsutils is used to upload files.
Figure 1 Obtaining the image list OBS OBS is a cloud storage service optimized for storing massive amounts of data. It provides unlimited, secure, and highly reliable storage capabilities at a relatively low cost.
How Do I Improve Training Efficiency While Reducing Interaction with OBS? Why the Data Read Efficiency Is Low When a Large Number of Data Files Are Read During Training? How Do I Define Path Variables When Using MoXing? Parent topic: Training Jobs
If ModelArts is no longer used, stop or delete the services running in ModelArts and delete the data stored in OBS and EVS. Clearing Storage Data ModelArts data is stored in OBS. To stop storage billing, switch to the OBS console and delete the data and directories.
Therefore, MoXing is more convenient than ModelArts and OBS SDKs. Parent topic: AI Development Basics
Go to OBS and delete the OBS bucket and files used by the training job. Searching for a Training Job If you log in to ModelArts using an IAM account, all training jobs under this account are displayed in the training job list.
Prepare the tenant ID and IAM ID for OBS bucket configuration. Send the prepared information to technical support, who will configure an OBS bucket policy based on your information. You can upload the collected logs to the corresponding OBS bucket.
Notebook Instances Creating a Notebook Instance Accessing a Notebook Instance Searching for, Starting, Stopping, or Deleting a Notebook Instance Changing a Notebook Instance Image Changing the Flavor of a Notebook Instance Selecting Storage in DevEnviron Dynamically Mounting an OBS
Then, access the OBS path from ModelArts to obtain the data. Parent topic: Internet Access Issues
//fake/be462ea9c5abc09f.jpg", "annotation":[ { "annotation-loc":"obs://fake/be462ea9c5abc09f.xml", "type":"modelarts/object_detection", "annotation-format":"PASCAL VOC", "annotated-by":"modelarts/hard_example_algo" } ] } Parent topic: Built-in Operators
For example, if the OBS path of the training code file is obs://obs-bucket/new/train.py and the code directory is obs://obs-bucket/new/, the local path of the container is /home/work/user-job-dir/new/.
