检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Using the ma-cli dli-job Command to Submit a DLI Spark Job Overview Querying DLI Spark Jobs Submitting a DLI Spark Job Querying DLI Spark Run Logs Querying DLI Queues Obtaining DLI Group Resources Uploading Local Files or OBS Files to a DLI Group Stopping a DLI Spark Job Parent topic
Managing Notebook Instances Searching for a Notebook Instance Updating a Notebook Instance Starting, Stopping, or Deleting a Notebook Instance Saving a Notebook Instance Dynamically Expanding EVS Disk Capacity Dynamically Mounting an OBS Parallel File System Viewing Notebook Events
The recommended method is to use the OBS path. OBS path (recommended) Call the copy_parallel API of MoXing to copy the corresponding OBS path.
The value must be a specific directory in an OBS bucket rather than the root directory of an OBS bucket. If type is set to obsfs, the parameter value must be a valid bucket name of the OBS parallel file system. (CCE does not support subdirectory mounting.)
The recommended method is to use the OBS path. OBS path (recommended) Call the copy_parallel API of MoXing to copy the corresponding OBS path.
Code Directory OBS directory where the training code file is stored. Configure this parameter only if your custom image does not contain training code. Upload code to the OBS bucket beforehand.
N/A N/A Storage resource Object Storage Service (OBS) OBS is used to store the input and output data for service deployment. For details, see Object Storage Service Pricing Details. CAUTION: OBS resources for storing data are continuously billed.
Reference ModelArts CLI Commands (Optional) Installing ma-cli Locally Autocompletion for ma-cli Commands ma-cli Authentication ma-cli image Commands for Building Images ma-cli ma-job Commands for Training Jobs ma-cli dli-job Commands for Submitting DLI Spark Jobs Using ma-cli to Copy OBS
Prepare the tenant ID and IAM ID for OBS bucket configuration. Send the prepared information to technical support, who will configure an OBS bucket policy based on your information. You can upload the collected logs to the corresponding OBS bucket.
Therefore, MoXing is more convenient than ModelArts and OBS SDKs. Parent topic: AI Development Basics
Go to OBS and delete the OBS bucket and files used by the training job. Searching for a Training Job If you log in to ModelArts using an IAM account, all training jobs under this account are displayed in the training job list.
An OBS URL is required. For example, obs://bucket/file.yaml. Range: N/A autosearch_framework_path String Definition: Framework code directory of an auto search job. An OBS URL is required. For example, obs://bucket/files/.
Notebook Instances Creating a Notebook Instance Accessing a Notebook Instance Searching for, Starting, Stopping, or Deleting a Notebook Instance Changing a Notebook Instance Image Changing the Flavor of a Notebook Instance Selecting Storage in DevEnviron Dynamically Mounting an OBS
Then, access the OBS path from ModelArts to obtain the data. Parent topic: Internet Access Issues
Data from OBS is used. dataset: Data from a specified dataset is used. data_url String OBS bucket path Table 4 model_metric_list parameters Parameter Type Description metric JSON Array Validation metrics of a class of a training job total_metric JSON Array All validation metrics
Symptom: After the labeled data is uploaded to OBS and synchronized, the data is displayed as unlabeled. Possible causes: Automatic encryption is enabled in the OBS bucket. Solution: Create an OBS bucket and upload data again, or disable bucket encryption and upload data again.
Options: obs: Data from OBS is used. dataset: Data from a specified dataset is used. data_url String OBS bucket path Table 5 volumes parameters Parameter Type Description nfs Object Storage volume of the shared file system type.
Call the API for querying the logs of a specified task in a training job (OBS link) to obtain the OBS path of the training job logs.
environment variables Variable Description Example MA_S3_ENDPOINT OBS endpoint N/A S3_VERIFY_SSL Whether to use SSL to access OBS S3_VERIFY_SSL=0 S3_USE_HTTPS Whether to use HTTPS to access OBS S3_USE_HTTPS=1 Table 5 Environment variables of the pip source and API Gateway address
If a model is deployed by a training job or OBS model file, this parameter is left blank. model_type String Model type.
