检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
OBS path format: obs://OBS parallel file system name/XXX hadoop fs -ls "obs://obs-example/job/hadoop-mapreduce-examples-3.1.2.jarobs-example/job/hadoop-mapreduce-examples-3.1.2.jar" To delete files on OBS using hadoop fs commands, run hadoop fs -rm -skipTrash.
ModifyObjectMetaData", "obs:object:DeleteObject", "obs:object:ListMultipartUploadParts", "obs:bucket:HeadBucket", "obs:object:AbortMultipartUpload", "obs:bucket:ListBucket", "obs:object:PutObject", "obs:object:GetAccessLabel", "obs:object:DeleteAccessLabel"
Spark on OBS Performance Tuning Scenario In the scenario where a small number of requests are frequently sent from Spark on OBS to OBS, you can disable OBS monitoring to improve performance. Configuration Modify the configuration in the core-site.xml file on the Spark client.
Parent topic: Accessing OBS
Does OBS Support the ListObjectsV2 Protocol? Question: Does OBS support the ListObjectsV2 protocol? Answer: No it doesn't. Parent topic: Peripheral Service Interconnection
In the displayed dialog box, select OBS for Connector, and click Next to set OBS link parameters. See Figure 2. Name: Enter a custom link name, for example, obslink. OBS Endpoint and Port: Enter the actual OBS address information. OBS Bucket Type: Use the default option.
In this example, before you analyze data stored in OBS, bind an IAM agency to the MRS cluster so that cluster components can connect to the OBS file system and have operation permissions on file system directories.
What Are the Differences Between OBS and HDFS in Data Storage? The data processed by MRS is from OBS or HDFS. OBS is an object-based storage service that provides secure, reliable, and cost-effective storage of huge amounts of data. MRS can directly process data in OBS.
For details about how to obtain the OBS SK, see How Do I Obtain the AK/SK? Typically, the URL shared by OBS contains HTTPS.
For details about how to obtain the OBS SK, see How Do I Obtain the AK/SK? Typically, the URL shared by OBS contains HTTPS.
How Do I Get My Data into OBS or HDFS? MRS can process data in OBS and HDFS. You can get your data into OBS or HDFS as follows: Upload local data to OBS. Log in to the OBS console.
table starts with obs://OBS bucket name/.
Interconnecting an MRS Cluster with OBS Through Guardian Interconnecting Guardian with OBS Example for Interconnecting an MRS Cluster with OBS Parent topic: Configuring Storage-Compute Decoupling for an MRS Cluster
Contact OBS O&M personnel to check whether the OBS service is normal. If yes, go to 4. If no, contact OBS O&M personnel to restore the OBS service. Collect fault information. On FusionInsight Manager, choose Cluster > Services > meta.
Procedure Upload service data to your OBS file system. Obtain the AK/SK information and create an OBS and HDFS link. For details, see Loader Link Configuration. Access the Loader page.
Using HDFS to Access OBS Function Description The OBS access process is as follows: Set fs.obs.access.key and fs.obs.secret.key. Use the FileSystem instance to read, add, and delete various resources. The appending operation is not supported.
Example for Interconnecting an MRS Cluster with OBS Accessing OBS Using Flink Through Guardian Accessing OBS Using HDFS Through Guardian Accessing OBS Using HetuEngine Through Guardian Accessing OBS Using Hive Through Guardian Accessing OBS Using Hudi Through Guardian Accessing OBS
Example for Interconnecting an MRS Cluster with OBS Interconnecting Flink with OBS Using an IAM Agency Interconnecting Flume with OBS Using an IAM Agency Interconnecting HDFS with OBS Using an IAM Agency Interconnecting Hive with OBS Using an IAM Agency Interconnecting Hudi with OBS
What Should I Do If a Job Fails to Be Submitted and the Error Is Related to OBS? Question: Why Does a Job Fail to Be Submitted and the Error Is Related to OBS? Answer: OBS errors are generally related to read and write.
Interconnecting an MRS Cluster with OBS Using an IAM Agency Interconnecting an MRS Cluster with OBS Using an IAM Agency Configuring the Policy for Clearing Recycle Bin Directories of MRS Cluster Components Example for Interconnecting an MRS Cluster with OBS Configuring Fine-Grained
