检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
You can select an HDFS or OBS path. Output Data Path No Set the output data path. You can select an HDFS or OBS path.
You can enter an HDFS or OBS path. The path format is as follows: OBS path: obs://bucket/warehouse HDFS path: /tmp/warehouse Global Configuration of Hudi Table Attributes Some advanced functions can be configured using parameters. For details, see Hudi advanced parameters.
You can enter an HDFS or OBS path. The path format is as follows: OBS path: obs://bucket/warehouse HDFS path: /tmp/warehouse Global Configuration of Hudi Table Attributes N/A N/A Some advanced functions can be configured using parameters.
The value can be original_version (parameters of the original job) or latest_version (parameters of the latest job). ignore_obs_monitor No boolean Whether to ignore OBS listening node_name No String Name of the node from which the job rerun starts when retry_location is set to specified_node
The migration path is as follows: CDH → DES → CDM (HUAWEI CLOUD) → OBS → CDM (Huawei Cloud) → MRS DES is well suited to the scenario where a large amount of data is to be transmitted, no private line is set up between the private cloud and HUAWEI CLOUD, and the bandwidth from the
Minimizing Agency Permissions Scenario and Objectives In earlier versions, the dlg_agency automatically created by DataArts Studio has the administrator permissions of the following services by default: GaussDB(DWS), MRS, RDS, OBS, SMN, and KMS.
Cdm.0800 400 Failed to connect to the OBS server. The OBS server fails to be connected.
In this example, data needs to be migrated from OBS to the database. Understand the scale of database migration and whether the migration duration of the largest table is acceptable.
How Do I Upload a File from a Notebook Instance to OBS or Download a File from OBS to a Notebook Instance in ModelArts? How Do I Handle OBS-related Errors Displayed in DataArts Studio? (General)
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
Prepare an OBS bucket to store the generated metadata. The OBS bucket must be in the same region as the CDM cluster in the DataArts Studio instance, and the enterprise project of the OBS bucket must be the same as that of the CDM cluster.
Driver File Path Yes It specifies the OBS path where the driver file is located. You need to download a .jar driver file from the corresponding official website and upload it to OBS. MySQL driver: Download it from https://downloads.mysql.com/archives/c-j/.
Scenario Description In this case, DataArts Studio is used to convert the user and rating data of a movie website stored in MySQL database to standard vertex and edge data sets, and to synchronize the data to OBS and MRS Hive.
table or in the partition directory of an OBS partition table.
You can enter an HDFS or OBS path. The path format is as follows: OBS path: obs://bucket/warehouse HDFS path: /tmp/warehouse Global Configuration of Hudi Table Attributes N/A N/A Some advanced functions can be configured using parameters.
You can enter an HDFS or OBS path. The path format is as follows: OBS path: obs://bucket/warehouse HDFS path: /tmp/warehouse Global Configuration of Hudi Table Attributes N/A N/A Some advanced functions can be configured using parameters.
Write Dirty Data: Specify this parameter if data that fails to be processed or filtered out during job execution needs to be written to OBS for future viewing. Before writing dirty data, create an OBS link on the CDM console.
Entire DB Configuring a Job for Synchronizing Data from Oracle to MRS Hudi Supported Oracle database versions: 10, 11, 12, and 19 Supported Hudi version: 0.11.0 Message system DMS for Kafka Object-based storage: Object Storage Service (OBS) Single table Entire DB Configuring a Job
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
Archive to OBS: Dirty data is stored in OBS and printed in job logs. Do not archive Write Dirty Data Link This parameter is displayed when Dirty Data Policy is set to Archive to OBS.
