检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Take OBS path obs://obs-bucket/training-test/demo-code as an example. The content in the OBS path will be automatically downloaded to ${MA_JOB_DIR}/demo-code in the training container, and demo-code (customizable) is the last-level directory of the OBS path.
In addition to the billing items displayed on the Dashboard page of ModelArts, OBS and EVS storage will be separately billed if they are used. To stop billing the OBS resources, go to the OBS management console and clear the data in OBS.
Engine Package Address Engine package address if the meta model comes from OBS and AI Engine is Custom. Runtime Environment Runtime environment on which the meta model depends if the meta model comes from a training job or OBS and a preset AI engine is used.
The storage fee is based on the OBS billing rules.
ModelArts mox.file provides a set of APIs for accessing OBS more conveniently, allowing users to operate OBS files by simulating a series of APIs for operating the local file system. For details, see Using MoXing Commands in a Notebook Instance.
Go to OBS and delete the OBS bucket and files used by the training job. After the resources are released, check the resource usage on the Dashboard page. Figure 1 Checking the resource usage Parent topic: Performing a Training
Data OBS Path OBS path for storing training data, for example, /test-modelarts2/mnist/dataset-mnist/, in which test-modelarts2 indicates a bucket name. Training OBS Path OBS path. A directory is automatically created in the path for storing a trained model and training logs.
To quickly obtain the latest data in the OBS bucket, in the Labeled or Unlabeled tab of the labeling job details page, click Synchronize Data Source. Symptom: After the labeled data is uploaded to OBS and synchronized, the data is displayed as unlabeled.
Imported from OBS: If you use a mainstream framework to develop and train a model locally, you can upload the model to an OBS bucket based on the model package specifications, import the model from OBS to ModelArts, and use the model to create an AI application for service deployment
When running a training job on ModelArts, read data and code files from the OBS bucket. Log in to OBS management console and create an OBS bucket and folder.
When running a training job on ModelArts, read data and code files from the OBS bucket. Log in to OBS console and create an OBS bucket and folder.
For details about specifications for importing labeled data from an OBS directory, see Specifications for Importing Data from an OBS Directory. from modelarts import workflow as wf # Use CreateDatasetStep to create a dataset of the new version using OBS data.
Deleting a Pod does not clear its storage. hostPath OBS Training dataset storage Object storage. The OBS SDKs are used to download sample data. Due to the large storage capacity being far from nodes, direct training speed is slow.
/ directory in the OBS bucket.
The field value is the OBS URL of the file in any of the following formats: Bucket path <obs path>{{Bucket name}}/{{Object name}}/File name, which is used to access your OBS data. You can access the path for obtaining an object in OBS. <obs path> can be obs:// or s3://.
algorithm = wf.steps.MrsJobAlgorithm( boot_file="obs://spark-sql/wordcount.py", # OBS path to the boot script parameters=[wf.AlgorithmParameters(name="run_args", value="--master,yarn-cluster")] ) inputs = wf.steps.MrsJobInput(name="mrs_input", data=wf.data.OBSPath
The input data, output data, and cached data generated during AI development using ModelArts Standard can be stored in OBS buckets. Therefore, you are advised to create an OBS bucket before using ModelArts. You can also create an OBS bucket later when needed.
Solution Obtain the size of an OBS folder. mox.file.get_size('obs://bucket_name/sub_dir_0/sub_dir_1', recursive=True) Obtain the size of an OBS file. mox.file.get_size('obs://bucket_name/obs_file.txt') Parent topic: MoXing
Yes str, Placeholder input Template input path, which can be an OBS file path or OBS directory path.
Training Jobs OBS Operation Issues In-Cloud Migration Adaptation Issues Hard Faults Due to Space Limit Internet Access Issues Permission Issues GP Issues Service Code Issues Training Job Suspensions Running a Training Job Failed Training Jobs Created in a Dedicated Resource Pool Training
