检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Possible values: OBS: After the streaming data is stored to DIS, it is then periodically imported to OBS. After the real-time file data is stored to DIS, it is imported to OBS immediately. Click . In the dialog box that is displayed, set dump parameters.
Select File from Local/OBS: Select a local path or an OBS bucket path. You are advised to obtain a file to import by exporting tags. The first row of the file is the tag name, and the first column is the job name.
NOTE: The OBS path of the driver file cannot contain Chinese characters. For security purposes, the system checks the SHA512 value and byte size of the selected driver file. If the driver file is not in the allowed list, an error message will be displayed.
Column names are displayed when the source of the migration job is OBS, CSV files are to be migrated, and parameter Extract first row as columns is set to Yes. Field mapping is not involved when the binary format is used to migrate files to files.
OBS Manager: manages OBS paths, including creating and deleting paths. RestClient: sends REST API requests. SMN: sends short messages or emails. MRS Spark: runs Spark jobs of MRS. MapReduce: Runs MapReduce jobs of MRS. MRSFlinkJob: runs FlinkJob jobs of MRS.
Ignore OBS Listening Yes: OBS listening is ignored in PatchData scenarios. No: The system listens to the OBS path in PatchData scenarios. Set Running Period Whether a running period can be set for the PatchData task.
Open a created job and drag a Create OBS node from the node library to the canvas. On the Node Properties tab page, configure the node properties. Figure 2 Create OBS Click Save and then Monitor to monitor the running status of the job.
Task to DataArts Migration Configuring a Job for Synchronizing Data from MySQL to MRS Hudi Configuring a Job for Synchronizing Data from MySQL to GaussDB(DWS) Configuring a Job for Synchronizing Data from MySQL to Kafka Configuring a Job for Synchronizing Data from DMS for Kafka to OBS
Currently, CDM supports the following incremental migration modes: Exporting the files in a specified directory Application scenarios: The migration source is a file system (OBS/HDFS/FTP/SFTP).
If Key Pair is the login mode of the host, you need to obtain the private key file, upload it to OBS, and select an OBS path. NOTE: The uploaded private key must match the public key configured on the host. For details, see Application Scenarios for Using Key Pairs.
Currently, CDM supports the following incremental migration modes: Exporting the files in a specified directory Application scenarios: The migration source is a file system (OBS/HDFS/FTP/SFTP).
Archive to OBS: Dirty data is stored in OBS and printed in job logs. Do not archive Write Dirty Data Link This parameter is displayed when Dirty Data Policy is set to Archive to OBS. Dirty data can only be written to OBS links.
SELECT * FROM top_bad_comment_product Figure 6 Viewing the data in the top_bad_comment_product table Developing and Scheduling a Job Assume that the BI reports in the OBS bucket are changing every day.
Information for creating a database connection, including the database IP address, port number, username, password, and key pair password Information for creating a DIS or OBS connection, including the Access Key ID (AK) and Secret Access Key (SK) Data Storage Security DataArts Studio
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
Driver File Path Yes It specifies the OBS path where the driver file is located. You need to download a .jar driver file from the corresponding official website and upload it to OBS.
When you use DataArts Studio, you may need to pay for the following services: OBS: During batch data migration or data development, DataArts Studio may write dirty data or logs to OBS, which incurs data storage fees. For details, see OBS Pricing Details.
Only links to OBS support dirty data writes. obs_link OBS Bucket This parameter is only displayed when Write Dirty Data Link is a link to OBS.
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
