检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Interconnecting an MRS Cluster with OBS Using an IAM Agency MRS allows you to store data in OBS and use an MRS cluster for data computing only. In this way, storage and compute are decoupled.
Run the hadoop fs -rm obs://<obs_path> command again. Parent topic: Accessing OBS
OBS path format: obs://OBS parallel file system name/XXX hadoop fs -ls "obs://obs-example/job/hadoop-mapreduce-examples-3.1.2.jarobs-example/job/hadoop-mapreduce-examples-3.1.2.jar" To delete files on OBS using hadoop fs commands, run hadoop fs -rm -skipTrash.
Spark on OBS Performance Tuning Scenarios In the scenario where a small number of requests are frequently sent from Spark on OBS to OBS, you can disable OBS monitoring to improve performance. Configuration Install the Spark client. For details, see Installing a Client.
Does OBS Support the ListObjectsV2 Protocol? Question: Does OBS support the ListObjectsV2 protocol? Answer: No it doesn't. Parent topic: Peripheral Service Interconnection
For details about how to obtain the OBS SK, see How Do I Obtain the AK/SK? Typically, the URL shared by OBS contains HTTPS.
Prerequisites You have uploaded the table data files to be read by ClickHouse to the OBS file system. For details, see Creating a Parallel File System. Interconnecting with OBS Using the S3 Table Function Obtain the OBS file path. Log in to the OBS console.
In this example, before you analyze data stored in OBS, bind an IAM agency to the MRS cluster so that cluster components can connect to the OBS file system and have operation permissions on file system directories.
What Are the Differences Between OBS and HDFS in Data Storage? The data processed by MRS is from OBS or HDFS. OBS is an object-based storage service that provides secure, reliable, and cost-effective storage of huge amounts of data. MRS can directly process data in OBS.
How Do I Get My Data into OBS or HDFS? MRS can process data in OBS and HDFS. You can get your data into OBS or HDFS as follows: Upload local data to OBS. Log in to the OBS console.
table starts with obs://OBS bucket name/.
In the displayed dialog box, select OBS for Connector, and click Next to set OBS link parameters. See Figure 2. Name: Enter a custom link name, for example, obslink. OBS Endpoint and Port: Enter the actual OBS address information. OBS Bucket Type: Use the default option.
Interconnecting an MRS Cluster with OBS Through Guardian Disabling Ranger OBS Path Authentication for Guardian Enabling Ranger OBS Path Authentication for Guardian Example for Interconnecting an MRS Cluster with OBS Parent Topic: Configuring Storage-Compute Decoupling for an MRS Cluster
Using HDFS to Access OBS Function Description The OBS access process is as follows: Set fs.obs.access.key and fs.obs.secret.key. Use the FileSystem instance to read, add, and delete various resources. The appending operation is not supported.
Example for Interconnecting an MRS Cluster with OBS Accessing OBS Using Flink Through Guardian Accessing OBS Using HDFS Through Guardian Accessing OBS Using HetuEngine Through Guardian Accessing OBS Using Hive Through Guardian Accessing OBS Using Hudi Through Guardian Accessing OBS
Example for Interconnecting an MRS Cluster with OBS Interconnecting Flink with OBS Using an IAM Agency Interconnecting Flume with OBS Using an IAM Agency Interconnecting HDFS with OBS Using an IAM Agency Interconnecting Hive with OBS Using an IAM Agency Interconnecting Hudi with OBS
Contact OBS O&M personnel to check whether the OBS service is normal. If yes, go to Step 4. If no, contact OBS O&M personnel to restore the OBS service. Collect fault information. On FusionInsight Manager, choose Cluster > Services > meta.
Procedure Upload service data to your OBS file system. Obtain the AK/SK information and create an OBS and HDFS link. For details, see Loader Link Configuration. Access the Loader page.
What Should I Do If a Job Fails to Be Submitted and the Error Is Related to OBS? Question: Why Does a Job Fail to Be Submitted and the Error Is Related to OBS? Answer: OBS errors are generally related to read and write.
Storing Hive Table Partitions to OBS and HDFS Scenario In the scenario where storage and compute resources are separated, you can specify different storage sources, for example, OBS or HDFS, for partitions in a Hive partitioned table.
