检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Method 2: If you allow DLI to save job logs in OBS, view the output in the taskmanager.out file.
Method 2: If you allow DLI to save job logs in OBS, view the output in the taskmanager.out file. -U(111,scooter,Big2-wheel scooter,5.18) +U(111,scooter,Big2-wheel scooter,5.15) Parent topic: Format
Click the name of the corresponding Flink job, choose Run Log, click OBS Bucket, and locate the folder of the log you want to view according to the date. Go to the folder of the date, find the folder whose name contains taskmanager, download the .out file, and view result logs.
Select Save Job Log, and specify the OBS bucket for saving job logs.
OBS Bucket: Select an OBS bucket for storing job logs and grant access permissions of the OBS bucket as prompted. Enable Checkpointing: Enable this function. Enter a SQL statement in the editing pane. The following is an example. Modify the parameters in bold as you need.
Create a Kafka cluster for DMS, enable SASL_SSL, download the SSL certificate, and upload the downloaded certificate client.jks to an OBS bucket.
If it is set to false, only metadata is cleared and data files are moved to the OBS recycle bin. The default value is false. You are advised not to change the value. Otherwise, deleted data cannot be restored.
Before writing dirty data, create an OBS link. You can view the data on OBS later. Retain the default value No, meaning dirty data is not recorded. Click Save and Run. On the Job Management page, you can view the job execution progress and result.
Method 2: If you allow DLI to save job logs in OBS, view the output in the taskmanager.out file. -U(111,scooter,Big2-wheel scooter,5.15) +U(111,scooter,Big2-wheel scooter,5.18) Parent topic: Format
val sparkSession = SparkSession.builder().getOrCreate() val sc = sparkSession.sparkContext sc.addFile("OBS address of krb5.conf") sc.addFile("OBS address of user.keytab") Thread.sleep(10) val krb5_startfile = new File(SparkFiles.get("krb5.conf")) val keytab_startfile
The default value is false. obs_bucket No String Name of an OBS bucket. root_id No Integer Parent job ID. edge_group_ids No Array of Strings List of edge computing group IDs.
Select Save Job Log, and specify the OBS bucket for saving job logs. Storing authentication credentials such as usernames and passwords in code or plaintext poses significant security risks. It is recommended using DEW to manage credentials instead.
Table 1 Primitive data types Data Type Description Storage Space Value Range Support by OBS Table Support by DLI Table INT Signed integer 4 bytes –2147483648 to 2147483647 Yes Yes STRING String - - Yes Yes FLOAT Single-precision floating point 4 bytes - Yes Yes DOUBLE Double-precision
Click the name of the corresponding Flink job, choose Run Log, click OBS Bucket, and locate the folder of the log you want to view according to the date.
Select Save Job Log, and specify the OBS bucket for saving job logs.
Select Save Job Log, and specify the OBS bucket for saving job logs.
Select Save Job Log, and specify the OBS bucket for saving job logs. Set the values of the parameters in bold in the following script as needed.
OBS Bucket: Select an OBS bucket for storing job logs and grant access permissions of the OBS bucket as prompted. Enable Checkpointing: Enable this function. Enter a SQL statement in the editing pane. The following is an example. Modify the parameters in bold as you need.
Submitting a Spark job Upload the Java code file to the OBS bucket. In the Spark job editor, select the corresponding dependency module and execute the Spark job.
Click the name of the corresponding Flink job, choose Run Log, click OBS Bucket, and locate the folder of the log you want to view according to the date. Go to the folder of the date, find the folder whose name contains taskmanager, download the .out file, and view result logs.
