检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
For OBS billing, see OBS Billing Overview. Table 2 Parameters for creating a lifecycle rule Parameter Description Example Value Status Whether to enable the lifecycle rule.
Permission Policy for Hive Adding a Ranger Access Permission Policy for Yarn Adding a Ranger Access Permission Policy for Spark2x Adding a Ranger Access Permission Policy for Kafka Adding a Ranger Access Permission Policy for HetuEngine Adding a Ranger Access Permission Policy for OBS
Spark Tasks Not to Obtain HBase Token Information Spark Core Enterprise-Class Enhancements Spark SQL Enterprise-Class Enhancements Spark Streaming Enterprise-Class Enhancements Spark Core Performance Tuning Spark SQL Performance Tuning Spark Streaming Performance Tuning Spark on OBS
If you want to back up data to OBS, you have connected the current cluster to OBS and have the permission to access OBS.
MRS allows you to store data in OBS and use an MRS cluster for data computing only. In this way, storage and compute are decoupled. For details about the components supported by each MRS version, see List of MRS Component Versions. Parent topic: MRS Basics
Import Data from HBase to HDFS A Data Format Error Is Reported When Data Is Exported from Hive to MySQL 8.0 Using Sqoop An Error Is Reported When the sqoop import Command Is Executed to Extract Data from PgSQL to Hive Failed to Use Sqoop to Read MySQL Data and Write Parquet Files to OBS
How Do I Migrate Data from OBS/S3 to ClickHouse? An Error Is Reported in Logs When the Auxiliary ZooKeeper or Replica Data Is Used to Synchronize Table Data How Do I Grant the Select Permission at the Database Level to ClickHouse Users?
Application Development How Do I Get My Data into OBS or HDFS? Can MRS Write Data to HBase Through an HBase External Table of Hive? Where Can I Download the Dependency Package (com.huawei.gaussc10) in the Hive Sample Project? Does MRS Support Python Code?
Link Configurations of Loader Jobs Destination Link Configurations of Loader Jobs Managing Loader Jobs Preparing a Driver for MySQL Database Link Importing Data Exporting Data Managing Jobs Operator Help Client Tools Loader Log Overview Example: Using Loader to Import Data from OBS
Parameter Description Table 1 HDFS parameters Parameter Description Default Value fs.obs.security.provider Implementation method of obtaining the key for accessing the OBS file system Value options are as follows: com.huawei.mrs.MrsObsCredentialsProvider: obtains a credential through
Parameter Description Table 1 HDFS parameters Parameter Description Default Value fs.obs.security.provider Implementation method of obtaining the key for accessing the OBS file system Value options are as follows: com.huawei.mrs.MrsObsCredentialsProvider: obtains a credential through
If HBase data is stored in OBS, data backup is not supported. If you want to back up data to OBS, you have connected the current cluster to OBS and have the permission to access OBS.
LocalDir LocalHDFS RemoteHDFS NFS CIFS SFTP OBS Flink (Applicable to MRS 3.2.0 and later versions) Flink metadata. LocalDir LocalHDFS RemoteHDFS OBS (available in MRS 3.5.0 and later) Kafka Kafka metadata. LocalDir LocalHDFS RemoteHDFS NFS CIFS OBS NameNode HDFS metadata.
File Format Loader supports the following file formats of data stored in OBS: CSV_FILE: Specifies a text file. When the destination link is a database link, only the text file is supported. BINARY_FILE: Specifies binary files excluding text files.
Enable OBS Local Cache OBS provides local cache that meets your data storage demands, improving the read speed. For example, you can configure a single-disk 100 GB local cache with data_cache=/srv/BigData/data1/impala:100 GB.
OBS Permission Control Mapping between MRS users and OBS permissions. Data Connection Type of the data connection associated with the cluster. Agency Agency bound to or modified by the cluster. Key Pair Name of a key pair, which is set during cluster creation.
What Should I Do If Data Failed to Be Synchronized to a Hive Table on the OBS Using hive-table? What Should I Do If Data Failed to Be Synchronized to an ORC or Parquet Table Using hive-table? What Should I Do If Data Failed to Be Synchronized Using hive-table?
In the environmental protection industry, climate data is stored on OBS and periodically dumped into HDFS for batch analysis. 10 TB of climate data can be analyzed in 1 hour.
If a low-permission user lacks OBS path access permission for the default database, Spark will display a permission error message but will still successfully create the database.
OBS Indicates that backup files are stored in an OBS directory. This option is available for MRS 3.3.0-LTS.1 or later only. Target Path: indicates the OBS directory for storing backup data.
