检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections. When compute resources in non-elastic resource pools are used, enhanced datasource connections can only be created for yearly/monthly and pay-per-use dedicated queues.
This parameter specifies the OBS address. Example: obs://bucket/path/CloudSearchService.cer Key Handling The Elasticsearch sink can work in either upsert mode or append mode, depending on whether a primary key is defined.
If no partition storage path is specified when running ADD PARTITION, the partition directory will be deleted from OBS, and the data will be moved to the .Trash/Current folder.
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
Write Dirty Data: Specify this parameter if data that fails to be processed or filtered out during job execution needs to be written to OBS for future viewing. Before writing dirty data, create an OBS link on the CDM console.
OBS Bucket: Select an OBS bucket for storing job logs and grant access permissions of the OBS bucket as prompted. Enable Checkpointing: Enable this function. Enter a SQL statement in the editing pane. The following is an example. Modify the parameters in bold as you need.
OBS Bucket: Select an OBS bucket for storing job logs and grant access permissions of the OBS bucket as prompted. Enable Checkpointing: Enable this function. Enter a SQL statement in the editing pane. The following is an example. Modify the parameters in bold as you need.
Select Save Job Log, and specify the OBS bucket for saving job logs. Change the values of the parameters in bold as needed in the following script.
Resource Planning and Costs Table 1 Resource planning and costs Resource Description Cost OBS You need to create an OBS bucket and upload data to OBS for data analysis using DLI.
---------------------------- External FUNCTION example.namespace02.repeat ( s varchar, n integer ) RETURNS varchar COMMENT 'repeat' LANGUAGE JAVA DETERMINISTIC CALLED ON NULL INPUT SYMBOL com.test.udf.hetuengine.functions.repeat URI obs
sparkSession.sql("insert into opentsdb_new_test values('Penglai', 'abc', '2021-06-30 18:00:00', 30.0)"); Query data. 1 sparkSession.sql("select * from opentsdb_new_test").show(); Response Submitting a Spark job Generate a JAR file based on the code file and upload the JAR file to the OBS
password'='**')"); Insert data. 1 sparkSession.sql("insert into dli_to_dws values(3,'Liu'),(4,'Xie')"); Query data. 1 sparkSession.sql("select * from dli_to_dws").show(); Response: Submitting a Spark Job Generate a JAR file based on the code file and upload the JAR file to the OBS
Method 2: If you allow DLI to save job logs in OBS, view the output in the taskmanager.out file. +I(111,scooter,Big 2-wheel scooter,5.15) Parent topic: Format
Click the name of the corresponding Flink job, choose Run Log, click OBS Bucket, and locate the folder of the log you want to view according to the date. Go to the folder of the date, find the folder whose name contains taskmanager, download the .out file, and view result logs.
Click the name of the corresponding Flink job, choose Run Log, click OBS Bucket, and locate the folder of the log you want to view according to the date. Go to the folder of the date, find the folder whose name contains taskmanager, download the .out file, and view result logs.
Select Save Job Log, and specify the OBS bucket for saving job logs.
Select Save Job Log, and specify the OBS bucket for saving job logs. Change the values of the parameters in bold as needed in the following script.
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
