检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Why Is Error Invalid OBS Bucket Name Reported After a Flink Job Submission Failed? Why Does the Flink Submission Fail Due to Hadoop JAR File Conflict? How Do I Locate a Flink Job Submission Error? Parent topic: Flink Jobs
There are the following ways to manage JAR files: Upload packages to OBS: Upload JAR files to an OBS bucket in advance and select the corresponding OBS path.
Flink Jar Jobs Using DEW to Acquire Access Credentials for Reading and Writing Data from and to OBS Guideline for Flink Jar jobs to acquire an AK/SK to read and write data from and to OBS.
and the storage path of the table in OBS will not change.
The error information is as follows: DLI.0999: RuntimeException: org.apache.hadoop.fs.obs.OBSIOException: initializing on obs://xxx.csv: status [-1] - request id [null] - error code [null] - error message [null] - trace :com.obs.services.exception.ObsException: OBS servcie Error Message
Select Save Job Log, and specify the OBS bucket for saving job logs. For details about how to use data types, see section Format.
Use the OBS API to upload data to obs_path_to_write_tmp_data. You can configure a lifecycle policy in OBS to periodically delete these temporary data. # 2. Submit the Load Data statement to DLI to import OBS data to DLI.
Request Table 2 Request parameters Parameter Mandatory Type Description paths Yes Array of Strings List of OBS object paths. The OBS object path refers to the OBS object URL. group Yes String Name of a package group.
Creating an OBS Table DLI provides an API for creating OBS tables.
This example writes data to OBS by calling the OBS SDK. Currently, UploadJob offers an implementation where data is written in JSON format by default.
For DLI Spark jobs, the permissions allow downloading OBS objects and reading/writing OBS foreign tables.
Tables Creating an OBS Table Creating a DLI Table Deleting a Table Viewing a Table Modifying a Table Partition-related Syntax Backing Up and Restoring Data of Multiple Versions
Timestamp format: yyyy-MM-dd HH:mm:ss Precautions Currently, the multiversion function supports only OBS tables created using the Hive syntax. For details about the syntax for creating a table, see Creating an OBS Table Using the Hive Syntax.
`obs://bucket_name/db0/delta_table0` VERSION AS OF 2; System Response You can view command execution results in the driver log or on the client. Parent topic: Delta DML Syntax
In the DLI environment, the data files of the Hudi table are stored in OBS, so they can be inspected by viewing the OBS files.
Save Job Log No Whether job running logs are saved to OBS If this option is selected, you need to set the following parameters: OBS Bucket: Select an OBS bucket to store job logs. If the OBS bucket you selected is unauthorized, click Authorize.
In this example, the path after upload is obs://dli-test-obs01/spark-examples.jar. For more operations on the OBS console, see the Object Storage Service User Guide.
This configuration is only available for Flink OpenSource SQL jobs and Flink Jar jobs. Locate the desired Flink job, click More in the Operation column, and select Runtime Configuration.
Prerequisites All software packages must be uploaded to OBS for storage in advance. Procedure On the left of the management console, choose Data Management > Package Management. On the Package Management page, click Create in the upper right corner to create a package.
Connectors Overview BlackHole ClickHouse DataGen Doris GaussDB(DWS) Elasticsearch OBS HBase Hive Hudi JDBC Kafka MySQL CDC Print Redis Upsert Kafka Parent topic: Flink OpenSource SQL 1.15 Syntax Reference
