检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
It provides a data abstraction layer for computing frameworks including Apache Spark, Presto, MapReduce, and Apache Hive, so that upper-layer computing applications can access persistent storage systems including HDFS and OBS through unified client APIs and a global namespace.
Set this parameter to a correct OBS path. The OBS path does not support files or programs encrypted by KMS. The value can contain a maximum of 1,023 characters, excluding special characters such as ;|&>'<$, and can be left blank.
You can obtain the AK and SK by logging in to the OBS console and go to the My Credentials page.
Enterprise-Class Enhancements of Hive Storing Hive Table Partitions to OBS and HDFS Configuring Automatic Removal of Old Data in the Hive Directory to the Recycle Bin Configuring Hive to Insert Data to a Directory That Does Not Exist Forbidding Location Specification When Hive Internal
For storage-compute decoupled system, OBS may fail to be accessed. As a result, component services cannot be processed properly. Possible Causes The meta role of the MRS cluster is abnormal.
Data is stored in OBS, which features low cost and unlimited storage capacity. The clusters can be deleted at any time in OBS. The computing performance is determined by OBS access performance and is lower than that of HDFS.
Script Path obs://mrs-samples/test.sh Script path. The value can be an OBS file system path or a local VM path. An OBS file system path must start with obs:// and end with .sh, for example, obs://mrs-samples/xxx.sh. A local VM path must start with a slash (/) and end with .sh.
OBS: indicates that backup files are stored in OBS. If you select this option, set the following parameters: Target Path: indicates the OBS directory for storing backup data.
File Format Loader supports the following file formats of data stored in OBS: CSV_FILE: Specifies a text file. When the destination link is a database link, only the text file is supported. BINARY_FILE: Specifies binary files excluding text files.
import command sqoop import --connect jdbc:mysql://10.100.231.134:3306/test --username root --password xxx --table component --target-dir obs://obs-file-bucket/xx --delete-target-dir --fields-terminated-by "," -m 1 --as-textfile Importing data from MySQL to OBS tables outside Hive
import command sqoop import --connect jdbc:mysql://10.100.231.134:3306/test --username root --password xxx --table component --target-dir obs://obs-file-bucket/xx --delete-target-dir --fields-terminated-by "," -m 1 --as-textfile Importing data from MySQL to OBS tables outside Hive
Parameter Default Value New Value --conf spark.shuffle.readHostLocalDisk false true --conf spark.io.encryption.enabled true false --conf spark.shuffle.service.enabled false true Adjusting Spark Scheduling Parameters to Shorten the Spark Scheduling Delay in the OBS Enabling OBS storage
If you want to back up data to OBS, you have connected the current cluster to OBS and have the permission to access OBS. Backing Up CDL Service Data On FusionInsight Manager, choose O&M > Backup and Restoration > Backup Management. Click Create.
OBS: indicates that backup files are stored in OBS. If you select this option, set the following parameters: Target Path: indicates the OBS directory for storing backup data.
OBS: indicates that backup files are stored in OBS. If you select this option, set the following parameters: Target Path: indicates the OBS directory for storing backup data.
OBS: indicates that backup files are stored in OBS. If you select this option, set the following parameters: Target Path: indicates the OBS directory for storing backup data.
Set this parameter to an OBS bucket path or a local VM path. OBS bucket path: Enter a script path, for example, enter the path of the public sample script provided by MRS. Example: obs://bootstrap/presto/presto-install.sh.
Why Do I Fail to Create a Table in the Specified Location on OBS After Logging to spark-beeline? Spark Shuffle Exception Handling Parent topic: Using Spark2x (for MRS 3.x or Later)
As a result, OBS cannot be accessed. Possible Causes The IAM service is abnormal. Handling Procedure Collect fault information. On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
Impact on the System If the average RPC processing time of the Guardian TokenServer instance exceeds the threshold, service access to OBS may slow down or even OBS cannot be accessed. Possible Causes The alarm threshold is improperly configured.
