检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Run the hadoop fs -rm obs://<obs_path> command again. Parent topic: Accessing OBS
Delete the OBS certificate.
Parent topic: Example for Interconnecting an MRS Cluster with OBS
To interconnect MRS with OBS using an IAM agency, perform the following tasks: Creating an ECS Agency with OBS Access Permissions Creating a Decoupled Storage and Compute Cluster Creating an OBS File System for Storing Data Creating a Lifecycle Rule Creating an ECS Agency with OBS
Interconnecting Sqoop with OBS Using an IAM Agency After connecting the Sqoop client to an OBS file system by referring to Interconnecting an MRS Cluster with OBS Using an IAM Agency, you can import tables from a relational database to OBS or export tables from OBS to a relational
/WordCount.jar -input obs://Parallel file system name/tmp/flinkjob/test -output obs://Parallel file system name/tmp/flinkjob/output Before interconnecting Flink with OBS, ensure that YARN is connected to OBS as Flink jobs run on YARN.
If the file list is returned, OBS is successfully accessed. hadoop fs -ls obs://mrs-word001/ Figure 1 Returned file list Run the following command to upload the /opt/test.txt file from the client node to the OBS file system path: hdfs dfs -put /opt/test.txt obs://OBS parallel file
Accessing OBS Using MapReduce Through Guardian After interconnecting Guardian with OBS by referring to Interconnecting Guardian with OBS, you need to refer to the information in this section to add custom configurations for the MapReduce component.
For example, set it to obs://hivetest/user/hive/warehouse/, where hivetest is the name of the OBS parallel file system.
Interconnecting Flume with OBS Create an OBS folder for storing data. Log in to the OBS console. In the navigation pane on the left, choose Resources > Parallel File Systems.
Parent topic: Example for Interconnecting an MRS Cluster with OBS
Accessing OBS Using HetuEngine Through Guardian After interconnecting Guardian with OBS by referring to Interconnecting Guardian with OBS, you can create a table with a specified Location as the OBS file system path on the HetuEngine client.
Interconnecting Spark with OBS Using an IAM Agency After configuring decoupled storage and compute for a cluster by referring to Interconnecting an MRS Cluster with OBS Using an IAM Agency, you can create tables with OBS paths as their location on the Spark client.
Accessing OBS Using Flink Through Guardian After interconnecting Guardian with OBS by referring to Interconnecting Guardian with OBS, you can access the OBS parallel file system and run jobs on the Flink client.
Access the OBS file system. hdfs dfs -ls obs://OBS parallel file system name/Path Create a directory in the OBS file system. hdfs dfs -mkdir obs://OBS parallel file system name/hadoop1 Execute the YARN task to access OBS. yarn jar Client installation directory/HDFS/hadoop/share/hadoop
Parent topic: Example for Interconnecting an MRS Cluster with OBS
Accessing OBS Using Spark Through Guardian After interconnecting Guardian with OBS by referring to Interconnecting Guardian with OBS, you can create a table whose location is the OBS path on the Spark client.
Accessing OBS Using Hive Through Guardian After interconnecting Guardian with OBS by referring to Interconnecting Guardian with OBS, you can create tables stored in the OBS parallel file system on the Hive client.
Access the OBS file system. hdfs dfs -ls obs://OBS parallel file system name/Path Create a directory in the OBS file system. hdfs dfs -mkdir obs://OBS parallel file system name/hadoop Upload the /opt/test.txt file on the client node to the obs://OBS parallel file system name/hadoop
//testhudi/cow_table/" is the OBS path, and testhudi is the name of the OBS parallel system file.
