检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
Write Dirty Data: Specify this parameter if data that fails to be processed or filtered out during job execution needs to be written to OBS for future viewing. Before writing dirty data, create an OBS link on the CDM console.
OBS Bucket: Select an OBS bucket for storing job logs and grant access permissions of the OBS bucket as prompted. Enable Checkpointing: Enable this function. Enter a SQL statement in the editing pane. The following is an example. Modify the parameters in bold as you need.
OBS Bucket: Select an OBS bucket for storing job logs and grant access permissions of the OBS bucket as prompted. Enable Checkpointing: Enable this function. Enter a SQL statement in the editing pane. The following is an example. Modify the parameters in bold as you need.
Select Save Job Log, and specify the OBS bucket for saving job logs. Change the values of the parameters in bold as needed in the following script.
Resource Planning and Costs Table 1 Resource planning and costs Resource Description Cost OBS You need to create an OBS bucket and upload data to OBS for data analysis using DLI.
---------------------------- External FUNCTION example.namespace02.repeat ( s varchar, n integer ) RETURNS varchar COMMENT 'repeat' LANGUAGE JAVA DETERMINISTIC CALLED ON NULL INPUT SYMBOL com.test.udf.hetuengine.functions.repeat URI obs
Method 2: If you allow DLI to save job logs in OBS, view the output in the taskmanager.out file. +I(111,scooter,Big 2-wheel scooter,5.15) Parent topic: Format
Click the name of the corresponding Flink job, choose Run Log, click OBS Bucket, and locate the folder of the log you want to view according to the date. Go to the folder of the date, find the folder whose name contains taskmanager, download the .out file, and view result logs.
Click the name of the corresponding Flink job, choose Run Log, click OBS Bucket, and locate the folder of the log you want to view according to the date. Go to the folder of the date, find the folder whose name contains taskmanager, download the .out file, and view result logs.
sparkSession.sql("insert into opentsdb_new_test values('Penglai', 'abc', '2021-06-30 18:00:00', 30.0)"); Query data. 1 sparkSession.sql("select * from opentsdb_new_test").show(); Response Submitting a Spark job Generate a JAR file based on the code file and upload the JAR file to the OBS
password'='**')"); Insert data. 1 sparkSession.sql("insert into dli_to_dws values(3,'Liu'),(4,'Xie')"); Query data. 1 sparkSession.sql("select * from dli_to_dws").show(); Response: Submitting a Spark Job Generate a JAR file based on the code file and upload the JAR file to the OBS
Select Save Job Log, and specify the OBS bucket for saving job logs. Change the values of the parameters in bold as needed in the following script.
Select Save Job Log, and specify the OBS bucket for saving job logs.
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
For more logs, check the OBS bucket where the job logs are stored. Search for dependency information in the logs. Search for Classpath: in the logs to check the dependencies.
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
Method 2: If you allow DLI to save job logs in OBS, view the output in the taskmanager.out file.
Method 2: If you allow DLI to save job logs in OBS, view the output in the taskmanager.out file. -U(111,scooter,Big2-wheel scooter,5.18) +U(111,scooter,Big2-wheel scooter,5.15) Parent topic: Format
