检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
This parameter is mandatory when OBS is supported. You are advised to store it in ciphertext in the configuration file or an environment variable and decrypt it when needed to ensure security. linkConfig.securityKey No String SK.
Migrating Data from MySQL to OBS CDM supports migration of tables to OBS. This best practice describes how to migrate tables from a MySQL database to OBS.
- OBS storage support The server must support OBS storage. When creating a Hive table, you can store the table in OBS. No AK This parameter is mandatory when OBS storage support is enabled.
The default value is |. fromJobConfig.useQuery No Boolean If you set this parameter to true, the name of the objects uploaded to OBS does not carry the query parameter.
OMS OMS helps you migrate data from object storage on other clouds online to the OBS on HUAWEI CLOUD. For details, see Object Storage Migration Service.
Configuring CDM Source Job Parameters From OBS From HDFS From HBase/CloudTable From Hive From DLI From FTP/SFTP From HTTP From PostgreSQL/SQL Server From DWS From SAP HANA From MySQL From Oracle From a Database Shard From MongoDB/DDS From Redis From DIS From Kafka/DMS Kafka From Elasticsearch
The job is successfully executed, and the dirty data type in the OBS bucket is incorrect. Parent topic: Troubleshooting
The job is successfully executed, and the dirty data type in the OBS bucket is incorrect. Parent topic: Troubleshooting
Configuring Link Parameters OBS Link Parameters PostgreSQL/SQLServer Link Parameters GaussDB(DWS) Link Parameters RDS for MySQL/MySQL Database Link Parameters Oracle Database Link Parameters DLI Link Parameters Hive Link Parameters HBase Link Parameters HDFS Link Parameters FTP/SFTP
- OBS storage support The server must support OBS storage. When creating a Hive table, you can store the table in OBS. No AK This parameter is mandatory when OBS storage support is enabled.
- OBS storage support The server must support OBS storage. When creating a Hive table, you can store the table in OBS. No AK This parameter is mandatory when OBS storage support is enabled.
For example, migrate files from the FTP server in the on-premises data center to OBS and migrate relational databases from the third-party cloud to RDS. Parent topic: General
Data is first written to an OBS file and then imported to a DLI table through foreign table mapping. The data has been basically imported, but an error is reported during the final mapping because an incorrect queue is used (the DLI SQL queue should be used).
OBS CDM supports data import and export using OBS, which also stores backup files and logs of CDM clusters. Cloud Eye CDM uses Cloud Eye to monitor cluster performance metrics, delivering status information in a concise and efficient manner, as shown in Table 1.
Column names are displayed when the source of the migration job is OBS, CSV files are to be migrated, and parameter Extract first row as columns is set to Yes. Field mapping is not involved when the binary format is used to migrate files to files.
Write Dirty Data: Specify this parameter if data that fails to be processed or filtered out during job execution needs to be written to OBS for future viewing. Before writing dirty data, create an OBS link on the CDM console.
Currently, CDM supports the following incremental migration modes: Exporting the files in a specified directory Application scenarios: The migration source is a file system (OBS/HDFS/FTP/SFTP).
Currently, CDM supports the following incremental migration modes: Exporting the files in a specified directory Application scenarios: The migration source is a file system (OBS/HDFS/FTP/SFTP).
Commercial use None 2 File migration You can migrate files from OBS to HDFS. Commercial use Table/File Migration Jobs Earlier Feature Releases View PDF of earlier feature releases.
Only links to OBS support dirty data writes. obs_link OBS Bucket This parameter is only displayed when Write Dirty Data Link is a link to OBS.
