检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Introduction to Impala Application Development Introduction to Impala Impala provides fast, interactive SQL queries directly on your Apache Hadoop data stored in HDFS, HBase, or Object Storage Service (OBS).
Impala Impala Impala provides fast, interactive SQL queries directly on your Apache Hadoop data stored in HDFS, HBase, or the Object Storage Service (OBS).
Configuring YARN Big Job Scanning YARN's big job scanning function monitors local temporary files (such as shuffle files) and key HDFS directories (OBS is not supported) for Hive, HetuEngine, and Spark jobs.
Answer: If you have stored data on OBS, you can use Spark in MRS to read Data Lake Insight (DLI) tables, flexibly process table data, and save the result to another DLI table. If you have not stored data on OBS, you cannot use MRS to read or write DLI tables.
Search for hive.metastore.warehouse.dir in the search box and change the parameter value to an OBS path, for example, obs://hivetest/user/hive/warehouse/. hivetest indicates the OBS file system name.
Therefore, MRS VMs cannot download OBS files from other regions. Use a script for downloading files from the OBS file system. You can specify the file to be downloaded from OBS in the script.
Replace the file system name with the actual one. alluxio fs mount /mnt/obs obs://obs-mrstest/data Mounted obs://obs-mrstest/data at /mnt/obs List files in the OBS file system using the Alluxio namespace.
Set the path of the executable program to the address of the program stored on the OBS, for example, obs://wordcount01/program/hadoop-mapreduce-examples-x.x.x.jar. Enter wordcount obs://wordcount01/input/ obs://wordcount01/output/ in the Parameter pane.
Set the path of the executable program to the address of the program stored on the OBS, for example, obs://wordcount01/program/hadoop-mapreduce-examples-x.x.x.jar. Enter wordcount obs://wordcount01/input/ obs://wordcount01/output/ in the Parameter pane.
Search for and modify the following parameters: fs.coldFS: OBS file system name, for example, obs://OBS parallel file system name hbase.fs.hot.cold.enabled: The default value is false.
FAQ About Decoupled Storage and Compute How Do I Read Encrypted OBS Data When Running an MRS Job? Example Application Development for Interconnecting HDFS with OBS How Do I Connect an MRS Cluster Client to OBS Using an AK/SK Pair?
Folder name: name of the folder created in the OBS parallel file system disk_s3 indicates the OBS disk name, which can be customized. Figure 3 Configuring an OBS disk with clickhouse-config-customize (Optional) Cache data stored on OBS to local storage to accelerate data query.
The OBS program path should start with obs://, for example, obs://wordcount/program/XXX.jar. The HDFS program path should start with hdfs://, for example, hdfs://hacluster/user/XXX.jar.
OBS Because the NTP Time of Cluster Nodes Is Not Synchronized
Table 1 obs-connector configuration Parameter Description Name Name of a Loader connection OBS Server Enter an OBS endpoint. The common format is OBS.Region.DomainName.
the OBS object name.
Check the executable programs in the OBS bucket. The executable programs can be stored in HDFS or OBS. The paths vary according to file systems. OBS storage path: starts with obs://, for example, obs://wordcount/program/hadoop-mapreduce-examples-2.7.x.jar.
HDFS or OBS to a Relational Database Typical Scenario: Exporting Data from HBase to a Relational Database Typical Scenario: Exporting Data from Hive to a Relational Database Typical Scenario: Importing Data from HBase to HDFS or OBS Parent topic: Using Loader
Data from HDFS or OBS to a Relational Database Using Loader to Export Data from HDFS to MOTService Using Loader to Export Data from HBase to a Relational Database Using Loader to Export Data from Hive to a Relational Database Using Loader to Export Data from HBase to HDFS or OBS Using
Script Path obs://mrs-samples/xxx.sh Script path. The value can be an OBS file system path or a local VM path. An OBS file system path must start with obs:// and end with .sh, for example, obs://mrs-samples/xxx.sh. A local VM path must start with a slash (/) and end with .sh.
