检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
OBS Path: Click . In the displayed dialog box, select an OBS path. CSS Cluster Name: Click . In the displayed dialog box, select a CSS cluster. Index: Enter a CSS index name. DLI Connection Name: Click . In the displayed dialog box, select a DLI data connection.
Example The previous day of the job scheduling plan time is used as the subdirectory name to generate an OBS path. The EL expression is as follows: #{"obs://test/"+DateUtil.format(DateUtil.addDays(Job.planTime,-1),"yyyy-MM-dd")} Parent topic: EL
Example The previous day of the job scheduling plan time is used as the subdirectory name to generate an OBS path. The EL expression is as follows: #{"obs://test/"+DateUtil.format(DateUtil.addDays(Job.planTime,-1),"yyyy-MM-dd")} Parent topic: EL
Import a resource file uploaded to OBS, click Next and then Close. Exporting a Resource To export a resource, perform the following operations: In the navigation tree of the Data Development console, choose Configuration > Manage Resource.
Import a resource file uploaded to OBS, click Next and then Close. Exporting a Resource To export a resource, perform the following operations: In the navigation tree of the Data Development console, choose Configuration > Manage Resource.
Preparations Object Storage Service (OBS) has been enabled and a bucket, for example, s3a://dlfexample, has been created for storing the JAR package of the Spark job.
Preparations Object Storage Service (OBS) has been enabled and a bucket, for example, s3a://dlfexample, has been created for storing the JAR package of the Spark job.
Click and choose Import Script in the script directory, select the script file that has been uploaded to OBS, and set Duplicate Name Policy. Figure 3 Importing a Script Click Next. Parent topic: Script Development
Click and choose Import Script in the script directory, select the script file that has been uploaded to OBS, and set Duplicate Name Policy. Figure 3 Importing a Script Click Next. Parent topic: Script Development
table or in the partition directory of an OBS partition table.
table or in the partition directory of an OBS partition table.
Log Path Selects the OBS path to save job logs. By default, logs are stored in a bucket named dlf-log-{Projectid}. NOTE: If you want to customize a storage path, select the bucket that you have created on OBS by referring to the instructions in Configuring a Log Storage Path.
Possible values: OBS: After the streaming data is stored to DIS, it is then periodically imported to OBS. After the real-time file data is stored to DIS, it is imported to OBS immediately. Click . In the dialog box that is displayed, set dump parameters.
Possible values: OBS: After the streaming data is stored to DIS, it is then periodically imported to OBS. After the real-time file data is stored to DIS, it is imported to OBS immediately. Click . In the dialog box that is displayed, set dump parameters.
Log Path Selects the OBS path to save job logs. By default, logs are stored in a bucket named dlf-log-{Projectid}. NOTE: If you want to customize a storage path, select the bucket that you have created on OBS by referring to the instructions in Configuring a Log Storage Path.
Figure 2 Scenario example Easy Analysis and Mining of Massive Amounts of Logs After ingesting logs into Object Storage Service (OBS) or Cloud Search Service through Data Ingestion Service (DIS), DLF helps you analyze and mine massive amounts of logs by compiling data development scripts
Key pair Password Key Pair Yes If Key Pair is the login mode of the host, the user needs to obtain the private key file, upload it to OBS, and select an OBS path.
Key pair Password Key Pair Yes If Key Pair is the login mode of the host, the user needs to obtain the private key file, upload it to OBS, and select an OBS path.
For example, use a data migration node to import data from OBS to MRS Hive. R R resource A resource is a self-defined code or text file that is uploaded by users and scheduled when a node is running.
For example, use a data migration node to import data from OBS to MRS Hive. R R resource A resource is a self-defined code or text file that is uploaded by users and scheduled when a node is running.
