AI开发平台ModelArts-提交ModelArts训练作业:基于自定义镜像创建训练作业

时间：2023-11-01 16:20:20

AI开发平台ModelArts 使用ma-cli ma-job命令提交ModelArts训练作业

基于自定义镜像创建训练作业

指定命令行options参数提交训练作业

ma-cli ma-job submit --image-url atelier/pytorch_1_8:pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64-20220926104358-041ba2e \                  --code-dir obs://your-bucket/mnist/code/ \                  --user-command "export LD_LIBRARY_PATH=/usr/local/cuda/compat:$LD_LIBRARY_PATH && cd /home/ma-user/modelarts/user-job-dir/code && /home/ma-user/anaconda3/envs/PyTorch-1.8/bin/python main.py" \                  --data-url obs://your-bucket/mnist/dataset/MNIST/ \                  --log-url obs://your-bucket/mnist/logs/ \                  --train-instance-type modelarts.vm.cpu.8u \                  --train-instance-count 1  \                  -q

使用自定义镜像的train.yaml样例：

# .ma/train.yaml样例（自定义镜像）image-url: atelier/pytorch_1_8:pytorch_1.8.0-cuda_10.2-py_3.7-ubuntu_18.04-x86_64-20220926104358-041ba2euser-command: export LD_LIBRARY_PATH=/usr/local/cuda/compat:$LD_LIBRARY_PATH && cd /home/ma-user/modelarts/user-job-dir/code && /home/ma-user/anaconda3/envs/PyTorch-1.8/bin/python main.pytrain-instance-type: modelarts.vm.cpu.8utrain-instance-count: 1data-url: obs://your-bucket/mnist/dataset/MNIST/code-dir: obs://your-bucket/mnist/code/log-url: obs://your-bucket/mnist/logs/##[Optional] Uncomment to set uid when use custom image modeuid: 1000##[Optional] Uncomment to upload output file/dir to OBS from training platformoutput:    - name: output_dir      obs_path: obs://your-bucket/mnist/output1/##[Optional] Uncomment to download input file/dir from OBS to training platforminput:    - name: data_url      obs_path: obs://your-bucket/mnist/dataset/MNIST/##[Optional] Uncomment pass hyperparametersparameters:    - epoch: 10    - learning_rate: 0.01    - pretrained:##[Optional] Uncomment to use dedicated poolpool_id: pool_xxxx##[Optional] Uncomment to use volumes attached to the training jobvolumes:  - efs:      local_path: /xx/yy/zz      read_only: false      nfs_server_path: xxx.xxx.xxx.xxx:/

上一篇：AI开发平台ModelArts-提交ModelArts训练作业:基于ModelArts预置镜像提交训练作业

下一篇：AI开发平台ModelArts-提交ModelArts训练作业:基于ModelArts预置镜像提交训练作业