检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Creating a VPC and Subnet Object Storage Service (OBS) OBS stores the following user data: MRS job input data, such as user programs and data files MRS job output data, such as result files and log files of jobs In MRS clusters, HDFS, Hive, MapReduce, YARN, Spark, Flume, and Loader
Question: When data is exported from a DistCP job, if some files already exist in OBS, how will the job process the files? Answer: DistCP jobs will overwrite the files in OBS. Parent topic: Job Management
The submission command example is as follows (the topology name is obs-test): storm jar /opt/jartarget/storm-examples-1.0.jar com.huawei.storm.example.obs.SimpleOBSTopology obs://my-bucket obs-test After the topology is submitted successfully, log in to OBS Browser to view the topology
OBS Indicates that backup files are stored in an OBS directory. You need to set the Target Path to the OBS directory for storing backup data. Set Maximum Number of Recovery Points to the number of snapshots that can be retained in the cluster.
Interconnection with Other Cloud Services Using MRS Spark SQL to Access GaussDB(DWS) Connecting to the OBS File System with an MRS Hive Table Interconnecting Hive with CSS
Importing Doris Data Importing Data to Doris with Broker Load Importing OBS Data to Doris with Broker Load Importing Data to Doris with Stream Load Parent topic: Using Doris
ClickHouse Data Import Interconnecting ClickHouse with RDS for MySQL Interconnecting ClickHouse with OBS Synchronizing Kafka Data to ClickHouse Importing DWS Table Data to ClickHouse Using ClickHouse to Import and Export Data Parent topic: Using ClickHouse
Loader supports the following data export modes: Exporting data from HDFS or OBS to an SFTP server Exporting data from HDFS or OBS to a relational database Exporting data from HBase to an SFTP server Exporting data from HBase to a relational database Exporting data from Phoenix tables
Set Program Path to the path where programs are stored on OBS, for example, obs://sparkpi/program/spark-examples_2.11-2.1.0.jar. In Program Parameter, select --class for Parameter and set Value to org.apache.spark.examples.SparkPi. Set Parameters to 10.
Developing an HDFS Application HDFS Development Plan Initializing HDFS Writing Data to an HDFS File Appending HDFS File Content Reading an HDFS File Deleting an HDFS File HDFS Colocation Setting HDFS Storage Policies Using HDFS to Access OBS Parent topic: HDFS Development Guide
to MRS Preparing the Network Before Data Migration to MRS Migrating Data from Hadoop to MRS Migrating Data from HBase to MRS Migrating Data from Hive to MRS Using BulkLoad to Import Data to HBase in Batches Migrating MySQL Data to MRS Hive with CDM Migrating Data from MRS HDFS to OBS
ClickHouse Data Import Interconnecting ClickHouse with RDS for MySQL Interconnecting ClickHouse with OBS Interconnecting ClickHouse with HDFS (MRS 3.2.0-LTS) Interconnecting ClickHouse with HDFS (MRS 3.3.0-LTS or later) Configuring Interconnection Between ClickHouse and Kafka Synchronizing
ECS BMS VPC EVS Image Management Service (IMS) OBS EIP SMN IAM For details about how to view and modify quotas, see Quotas.
In addition, MRS supports fine-grained permission management of OBS for multiple tenants. Permissions to access OBS buckets and objects in the buckets are differentiated based on user roles, so that MRS users can each control a different directory in OBS buckets.
What Are the Differences Between OBS and HDFS in Data Storage? What Are the Solutions for Processing 1 Billion Data Records? What are the advantages of the compression ratio of zstd?
If you want to restore data from OBS, you have connected the Doris cluster to OBS and have the permission to access OBS.
For OBS billing, see OBS Billing Overview. Table 2 Parameters for creating a lifecycle rule Parameter Description Example Value Status Whether to enable the lifecycle rule.
Permission Policy for Hive Adding a Ranger Access Permission Policy for Yarn Adding a Ranger Access Permission Policy for Spark2x Adding a Ranger Access Permission Policy for Kafka Adding a Ranger Access Permission Policy for HetuEngine Adding a Ranger Access Permission Policy for OBS
Spark Tasks Not to Obtain HBase Token Information Spark Core Enterprise-Class Enhancements Spark SQL Enterprise-Class Enhancements Spark Streaming Enterprise-Class Enhancements Spark Core Performance Tuning Spark SQL Performance Tuning Spark Streaming Performance Tuning Spark on OBS
If you want to back up data to OBS, you have connected the current cluster to OBS and have the permission to access OBS.
