检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
DataArts Factory You can enable backup of assets such as scripts and jobs to OBS buckets. For details, see Managing Backups. The execution history of scripts, jobs, and nodes is stored in OBS buckets. If no OBS bucket is available, you cannot view the execution history.
", "obs:object:PutObject", "obs:object:DeleteObject", "obs:bucket:GetBucketStorage", "obs:bucket:GetBucketLocation", "obs:bucket:ListAllMyBuckets", "obs:bucket:ListBucket",
Log Path Selects the OBS path to save job logs. By default, logs are stored in a bucket named dlf-log-{Projectid}.
The following links are supported: DMS for Kafka to Hudi or DMS for Kafka to OBS Figure 3 Single table synchronization Entire database synchronization Multiple tables in multiple databases in an instance can be synchronized to another instance in real time.
Data Source Authentication and Other Function Configuration AK Yes OBS AK Example: HCXUET8G37MWF SK No SK corresponding to the OBS AK Parent topic: Configuring DataArts Studio Data Connection Parameters
Cloud Search Service (CSS) Graph Engine Service (GES) Object Storage Service (OBS) MRS Hudi (MRS Hudi is a data format. The metadata is stored in Hive, and operations are performed using Spark.)
Configuring Link Parameters OBS Link Parameters PostgreSQL/SQLServer Link Parameters GaussDB(DWS) Link Parameters RDS for MySQL/MySQL Database Link Parameters Oracle Database Link Parameters DLI Link Parameters Hive Link Parameters HBase Link Parameters HDFS Link Parameters FTP/SFTP
- OBS storage support The server must support OBS storage. When creating a Hive table, you can store the table in OBS. No AK This parameter is mandatory when OBS storage support is enabled.
Public services, such as Elastic Cloud Server (ECS), Elastic Volume Service (EVS), Object Storage Service (OBS), Virtual Private Cloud (VPC), Elastic IP (EIP), and Image Management Service (IMS), are shared within the same region.
Log in to the management console and choose Storage > Object Storage Service to access the OBS console. Click Create Bucket and set parameters as prompted to create an OBS bucket named fast-demo.
Parent topic: DataArts Migration (CDM Jobs)
for Migrating Data from MySQL to MRS Hudi Optimizing the Parameters of a Job for Migrating Data from MySQL to GaussDB(DWS) Optimizing the Parameters of a Job for Migrating Data from MySQL to DMS for Kafka Optimizing the Parameters of a Job for Migrating Data from DMS for Kafka to OBS
The value can be DWS, OBS, CSS, HIVE, DLI, or CUSTOM. OK Click OK to save the parameter settings. Cancel Click Cancel to cancel the parameter settings. Modify Click to modify the parameter settings. After the modification, save the settings.
Input Notebook File Yes Select the notebook file in the OBS input directory. The file is in .ipynb format. The absolute path can contain a maximum of 2,048 characters.
Column names are displayed when the source of the migration job is OBS, CSV files are to be migrated, and parameter Extract first row as columns is set to Yes. Field mapping is not involved when the binary format is used to migrate files to files.
Column names are displayed when the source of the migration job is OBS, CSV files are to be migrated, and parameter Extract first row as columns is set to Yes. Field mapping is not involved when the binary format is used to migrate files to files.
OBS Manager: manages OBS paths, including creating and deleting paths. RESTAPI: sends REST API requests. SMN: sends short messages or emails. MRS Spark: runs Spark jobs of MRS. MapReduce: runs MapReduce jobs of MRS.
OBS path: The resource package file will not be uploaded to DLI resource management system before the job is executed. The OBS path where the file is located is part of the message body for starting the job. This type is recommended.
For example, migrate files from the FTP server in the on-premises data center to OBS and migrate relational databases from the third-party cloud to RDS. Parent topic: DataArts Migration (CDM Jobs)
Ignore OBS Listening The default value is Yes. Yes: The system does not listen to the OBS path when rerunning the job instance. No: The system listens to the OBS path when rerunning the job instance. NOTE: If this parameter is not used, ignore it.
