检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
OBS Manager: manages OBS paths, including creating and deleting paths. RESTAPI: sends REST API requests. SMN: sends short messages or emails. MRS Spark: runs Spark jobs of MRS. MapReduce: runs MapReduce jobs of MRS.
Column names are displayed when the source of the migration job is OBS, CSV files are to be migrated, and parameter Extract first row as columns is set to Yes. Field mapping is not involved when the binary format is used to migrate files to files.
Column names are displayed when the source of the migration job is OBS, CSV files are to be migrated, and parameter Extract first row as columns is set to Yes. Field mapping is not involved when the binary format is used to migrate files to files.
Data Source Authentication and Other Function Configuration AK Yes OBS AK Example: HCXUET8G37MWF SK No SK corresponding to the OBS AK Parent topic: Configuring DataArts Studio Data Connection Parameters
OBS path: The resource package file will not be uploaded to DLI resource management system before the job is executed. The OBS path where the file is located is part of the message body for starting the job. This type is recommended.
For example, migrate files from the FTP server in the on-premises data center to OBS and migrate relational databases from the third-party cloud to RDS. Parent topic: DataArts Migration (CDM Jobs)
Column names are displayed when the source of the migration job is OBS, CSV files are to be migrated, and parameter Extract first row as columns is set to Yes. Field mapping is not involved when the binary format is used to migrate files to files.
Possible values: OBS: After the streaming data is stored to DIS, it is then periodically imported to OBS. After the real-time file data is stored to DIS, it is imported to OBS immediately. Click . In the dialog box that is displayed, set dump parameters.
OBS Manager: manages OBS paths, including creating and deleting paths. RestClient: sends REST API requests. SMN: sends short messages or emails. MRS Spark: runs Spark jobs of MRS. MapReduce: Runs MapReduce jobs of MRS. MRSFlinkJob: runs FlinkJob jobs of MRS.
Driver Source Yes Source of the driver file Driver File Path Yes This parameter is mandatory when Driver File Source is set to OBS path. It specifies the OBS path where the driver file is located.
Ignore OBS Listening The default value is Yes. Yes: The system does not listen to the OBS path when rerunning the job instance. No: The system listens to the OBS path when rerunning the job instance. NOTE: If this parameter is not used, ignore it.
Open a created job and drag a Create OBS node from the node library to the canvas. On the Node Properties tab page, configure the node properties. Figure 2 Create OBS Click Save and then Monitor to monitor the running status of the job.
Currently, CDM supports the following incremental migration modes: Exporting the files in a specified directory Application scenarios: The migration source is a file system (OBS/HDFS/FTP/SFTP).
If Key Pair is the login mode of the host, you need to obtain the private key file, upload it to OBS, and select an OBS path. NOTE: The uploaded private key must match the public key configured on the host. For details, see Application Scenarios for Using Key Pairs.
Currently, CDM supports the following incremental migration modes: Exporting the files in a specified directory Application scenarios: The migration source is a file system (OBS/HDFS/FTP/SFTP).
Select File from Local/OBS: Select a local path or an OBS bucket path. You are advised to obtain a file to import by exporting tags. The first row of the file is the tag name, and the first column is the job name.
) Configuring a Job for Synchronizing Data from DMS for Kafka to OBS Apache Kafka Hadoop: MRS Kafka NOTE: This connection is available only after you apply for the whitelist membership.
Archive to OBS: Dirty data is stored in OBS and printed in job logs. Do not archive Write Dirty Data Link This parameter is displayed when Dirty Data Policy is set to Archive to OBS. Dirty data can only be written to OBS links.
SELECT * FROM top_bad_comment_product Figure 6 Viewing the data in the top_bad_comment_product table Developing and Scheduling a Job Assume that the BI reports in the OBS bucket are changing every day.
Information for creating a database connection, including the database IP address, port number, username, password, and key pair password Information for creating a DIS or OBS connection, including the Access Key ID (AK) and Secret Access Key (SK) Data Storage Security DataArts Studio
