检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Why Does an OBS Quickly Deleted Directory Not Take Effect After Being Added to the Customized Hive Configuration? The error message "java.lang.OutOfMemoryError: Java heap space." is displayed during Hive SQL execution.
You have had the permission to access the HDFS or OBS directories and data involved in job execution. You have had the permission to access the HBase tables or phoenix tables that are used during job execution.
You have had the permission to access the HDFS or OBS directories and data involved in job execution. You have obtained the username and password of the relational database.
Using Loader to Import Data from an SFTP Server to HDFS or OBS Scenario Use Loader to import data from an SFTP server to HDFS or OBS. Prerequisites You have obtained the service username and password for creating a Loader job.
Directory Log in to the OBS console.
You have had the permission to access the HDFS or OBS directories and data involved in job execution. You have obtained the username and password of the relational database.
Typical Scenario: Importing Data from an SFTP Server to HDFS or OBS Scenario Use Loader to import data from an SFTP server to HDFS or OBS. Prerequisites You have obtained the service username and password for creating a Loader job.
Relationships Between MemArtsCC and Other Components OBS OBS provides a new InputStream: OBSMemArtsCCInputStream. This InputStream reads data from the MemArtsCC cluster deployed on the compute side to reduce OBS server pressure and improve data read performance.
Add the endpoint configuration item fs.obs.endpoint of OBS and enter the endpoint corresponding to OBS in Value. For details, see Endpoints. For MRS 3.x or later, a file path on OBS can start with obs://.
Symptom When Spark SQL is used to access Hive partitioned tables stored in OBS, the access speed is slow and a large number of OBS query APIs are called.
Click Import Data and configure the HDFS and OBS paths correctly. When configuring the OBS or HDFS path, click Browse, select a file directory, and click OK. Figure 3 Importing data OBS path description: The path must start with obs://.
Question Can Hive tables be stored in OBS or HDFS? Answer The location of a common Hive table stored on OBS can be set to an HDFS path. In the same Hive service, you can create tables stored in OBS and HDFS, respectively.
HDFS Client Failed to Delete Overlong Directories Symptom When a user runs the hadoop fs -rm -r -f obs://<obs_path> command to delete an OBS directory with an overlong path name, the following error message is displayed: 2022-02-28 17:12:45,605 INFO internal.RestStorageService: OkHttp
OBS determines whether the current user has the access permission based on the credential. Figure 2 Relationships between Guardian and other components Parent topic: Components
Log in to the Hive client node, use Beeline to create a table, and ensure that Location is an OBS path. For details, see Accessing OBS Using Hive Through Guardian. Run the following statement in Beeline to execute MapReduce tasks: select count(*) from tablename; Repeat 1 and 2.
Use HetuEngine to create an OBS table and query table data. For details, see Accessing OBS Using HetuEngine Through Guardian. Repeat 1 and 2. If there are more shards in the cluster than there were in 2, the interconnection is successful. Parent topic: Using MemArtsCC
Log in to the Spark client node, create a table whose Location is an OBS path, and query the table. For details, see Accessing OBS Using Spark Through Guardian. Repeat 1 and 2. If there are more shards in the cluster than there were in 2, the interconnection is successful.
For details about OBS permission configuration, see Granting OBS Permissions. If you only need to configure Guardian to connect to OBS, but do not need to use Ranger to configure permission policies for OBS paths, see Disabling Ranger OBS Path Authentication for Guardian.
Run the following command to mount the internal directory of an OBS container to the /obs directory of Alluxio: alluxio fs mount /obs obs://<OBS_BUCKET>/<OBS_DIRECTORY>/ Parent topic: Using Alluxio
An Error Is Reported When DistCP Is Used to Copy an Empty Folder Symptom When a user runs the following distcp commands on the MRS client, empty folders cannot be copied from HDFS to OBS: hadoop distcp -Dfs.obs.endpoint=xxx -Dfs.obs.access.key=xxx -Dfs.obs.secret.key=xxx -update hdfs
