检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
No Bucket Name The name of an OBS bucket. The bucket to be added must be available. OBS buckets configured with server-side encryption and OBS parallel file systems cannot be added as storage backends. No OBS Endpoint The OBS domain name of the region.
Click the Storage Backends tab and then click Add OBS Bucket. Figure 1 Add OBS Bucket On the Add OBS Bucket page displayed on the right, configure the parameters described in Table 1.
Configuring Auto Data Export from SFS Turbo to OBS After auto export is configured, checkpoint files periodically written to the SFS Turbo file system during training will be automatically exported to the OBS bucket for long-term storage.
Uploading Data to OBS and Preloading the Data to SFS Turbo Uploading Data to OBS An OBS bucket has been created by referring to Creating a Bucket. obsutil has been installed by referring to Downloading and Installing obsutil.
Use the correct OBS bucket name. 400 SFS.TURBO.0110 OBS agent error The import or export task failed. Contact technical support. 400 SFS.TURBO.0111 The OBS configuration list is empty The OBS configuration list is empty. Use valid OBS configuration information.
What Are the Differences Between SFS, OBS, and EVS? Table 1 shows the comparison between SFS, OBS, and EVS. Table 1 Comparison between SFS, OBS, and EVS Dimension SFS OBS EVS Concept SFS provides on-demand high-performance file storage, which can be shared by multiple ECSs.
Can I Use OBS to Support AI Autonomous Driving and Large Model Training? OBS is capacity-oriented storage. Its latency and bandwidth cannot meet the requirements of high-performance AI training.
For example, if you want to migrate data from SFS to OBS, use the OBS upload function on the server or container to upload data to OBS.
Configuring the SFS Turbo Data Eviction Policy After an OBS bucket is added as the storage backend of an SFS Turbo HPC file system, you are advised to configure a cold data eviction duration.
Monitoring OBS Performance You can monitor the performance of the associated OBS bucket on Cloud Eye. The file import and export speeds are affected by the maximum read and write bandwidth of OBS. The default maximum bandwidth is 16 Gbit/s.
Configure SFS Turbo and OBS interworking. Configure auto data export from SFS Turbo to OBS. Configure the SFS Turbo data eviction policy. 4 Perform training Upload data to OBS and preload the data to SFS Turbo.
Any data or metadata modifications made will be automatically synchronized to the OBS bucket. CHANGED: Files previously imported from the OBS bucket and then modified in the SFS Turbo interworking directory.
It cannot contain slashes (/), and cannot be the name of a multi-level directory. obs Yes ObsDataRepository object The OBS storage backend.
Basic Configurations Configuring Network Passthrough Between ModelArts and SFS Turbo Configuring SFS Turbo and OBS Interworking Configuring Auto Data Export from SFS Turbo to OBS Configuring the SFS Turbo Data Eviction Policy Parent topic: Implementation Procedure
Creating an OBS Bucket Create an OBS bucket. Set Storage Class to Standard and Bucket Policy to Private. For details, see Creating an OBS Bucket. Creating a ModelArts Resource Pool Take a dedicated resource pool as an example.
You can use SFS Turbo HPC file systems to speed up data access to OBS and asynchronously store the generated data to OBS for low-cost, long term persistent storage.
Any data or metadata modifications made will be automatically synchronized to the OBS bucket. CHANGED: Files previously imported from the OBS bucket and then modified in the SFS Turbo interworking directory.
Any data or metadata modifications made will be automatically synchronized to the OBS bucket. CHANGED: Files previously imported from the OBS bucket and then modified in the SFS Turbo interworking directory.
Costs OBS price: See OBS Billing. SFS Turbo price: See SFS Turbo Billing. ModelArts price: See ModelArts Billing. NOTICE: Prices provided are for reference only. Refer to the prices displayed on the Huawei Cloud console.
Example Request DELETE / HTTP/1.1 User-Agent: curl/7.29.0 Accept: */* Date: WED, 01 Jul 2015 02:31:25 GMT Authorization: OBS H4IPJX0TQTHTHEBQQCEC:jZiAT8Vx4azWEvPRMWi0X5BpJMA= Example Response HTTP/1.1 204 No Content Server: OBS X-Obs-Request-Id: 0000018893B8081DC047305E783867DD X-Obs-Id
