检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
You can select an HDFS or OBS path. Output Data Path No Output data path. You can select an HDFS or OBS path. Table 2 Advanced settings Parameter Mandatory Description Job Status Polling Interval (s) Yes Set the interval at which the system checks whether the job is complete.
Constraints on OBS During file migration, the system automatically transfers the files concurrently. In this case, Concurrent Extractors in the task configuration is invalid. Resumable transmission is not supported. If CDM fails to transfer files, OBS fragments are generated.
You can save the data in Table 1 to a CSV file in the OBS bucket. Then use a DLI SQL or DWS SQL statement to create an OBS foreign table, associate it with the CSV file, and use the query result of the OBS foreign table as the dataset.
You can save the data in Table 1 to a CSV file in the OBS bucket. Then use a DLI SQL or DWS SQL statement to create an OBS foreign table, associate it with the CSV file, and use the query result of the OBS foreign table as the dataset.
For details about how to create a bucket on the OBS console, see Creating a Bucket in Object Storage Service Console Operation Guide. Upload data to OBS bucket fast-demo.
Driver File Path Yes It specifies the OBS path where the driver file is located. You need to download a .jar driver file from the corresponding official website and upload it to OBS. MySQL driver: Download it from https://downloads.mysql.com/archives/c-j/.
Driver File Path Yes It specifies the OBS path where the driver file is located. You need to download a .jar driver file from the corresponding official website and upload it to OBS.
You can enter an HDFS or OBS path. The path format is as follows: OBS path: obs://bucket/warehouse HDFS path: /tmp/warehouse Global Configuration of Hudi Table Attributes N/A N/A Some advanced functions can be configured using parameters.
Driver File Path Yes This parameter is mandatory when Driver file source is set to OBS path. Path of the driver file in the OBS bucket. You need to download the .jar driver file from the corresponding official website and upload it to the OBS bucket.
The default value is false. resources No Array of longs Resource ID array export_mode No String Export mode. null indicates that data is to be exported to a local file system, and obs indicates that data is to be exported to OBS.
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
The EL expression is as follows: #{"obs://test/"+DateUtil.format(DateUtil.addDays(Job.planTime,-1),"yyyy-MM-dd")} Parent topic: EL Expression Reference
Driver File Path Yes It specifies the OBS path where the driver file is located. You need to download a .jar driver file from the corresponding official website and upload it to OBS.
If data is migrated between file-related data sources, such as FTP, SFTP, OBS, and HDFS, the value of File Format must the same as the source file format.
You can select an HDFS or OBS path. Output Data Path No Set the output data path. You can select an HDFS or OBS path.
Only links to OBS support dirty data writes. obslink OBS Bucket This parameter is only displayed when Write Dirty Data Link is a link to OBS.
You can enter an HDFS or OBS path. The path format is as follows: OBS path: obs://bucket/warehouse HDFS path: /tmp/warehouse Global Configuration of Hudi Table Attributes Some advanced functions can be configured using parameters. For details, see Hudi advanced parameters.
The migration path is as follows: CDH → DES → CDM (HUAWEI CLOUD) → OBS → CDM (Huawei Cloud) → MRS DES is well suited to the scenario where a large amount of data is to be transmitted, no private line is set up between the private cloud and HUAWEI CLOUD, and the bandwidth from the
