检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
OBS Bucket for DLI Dirty Data The OBS bucket for storing dirty data generated during DLI SQL execution in DataArts Studio Data Development.
You can obtain the OBS bucket endpoint by either of the following means: To obtain the endpoint of an OBS bucket, go to the OBS console and click the bucket name to go to its details page.
or Download a File from OBS to a Notebook Instance in ModelArts Scheduling Notebook Tasks in Pipeline Jobs in DataArts Factory Parent topic: DataArts Factory
GET /v1/b384b9e9ab9b4ee8994c8633aabc9505/resources/3624d1c3-5df5-4f20-9af9-98eadad6c5f9 Example Response Success response { "name": "test", "type": "jar", "location": "obs://dlf-test/hadoop-mapreduce-examples-2.4.1.jar", "dependFiles": ["obs://dlf-test/depend1.jar","obs://dlf-test
Environment preparations OBS has been enabled and buckets have been created, for example, obs://dlfexample/alarm_info and obs://dlfexample/alarm_count_info, which are used to store the raw alarm table and alarm statistic table, respectively.
(OBS) NoSQL cannot serve as the destination.
You can select an existing OBS bucket. The selected OBS bucket is globally configured in the current workspace. If this parameter is not set, job logs generated during data development are stored in the OBS bucket named dlf-log-{projectId} by default.
Binary files cannot. obsutil is recommended for migrating data from HTTP servers to OBS. For details, see Introduction to obsutil.
No AK This parameter is mandatory when OBS storage support is enabled. The account corresponding to the AK/SK pair must have the OBS Buckets Viewer permission. Otherwise, OBS cannot be accessed and the "403 AccessDenied" error is reported.
For details, see Authorizing dlg_agency. obs:bucket:HeadBucket obs:bucket:CreateBucket obs:object:PutObject obs:object:DeleteObject obs:bucket:ListBucket obs:object:GetObject obs:bucket:GetEncryptionConfiguration obs:bucket:PutEncryptionConfiguration Constraints You need to select
Use the Create OBS node. If the preceding methods are unavailable, use the Create OBS node and create an OBS path with the value of the EL expression as its name. You can click Test and go to the OBS console to view the name of the created path.
Preparations Object Storage Service (OBS) has been enabled and a bucket, for example, obs://dlfexample, has been created for storing the JAR package of the Spark job.
//00000000dlf-test/hadoop-mapreduce-examples-2.4.1.jar", "dependFiles":[ "obs://00000000dlf-test/depend1.jar", "obs://00000000dlf-test/depend2.jar" ], "desc":"test", "directory":"/resource" }
Database Sharding Data warehouse: Data Lake Insight (DLI) Hadoop: MRS HBase and MRS Hive Search: Elasticsearch and Cloud Search Service (CSS) Object-based storage: Object Storage Service (OBS) Database shards cannot serve as the destination.
Importing Environment Variables This function is available only if the OBS service is available. If OBS is unavailable, variables can be imported from the local PC.
CSS, and GaussDB(DWS) ETL Job Data lineages generated by ETL tasks between DLI, OBS, MySQL, and DWS.
CSS, and GaussDB(DWS) ETL Job Data lineages generated by ETL tasks between DLI, OBS, MySQL, and DWS.
Table 1 Parameters for creating a notebook Category Parameter Description Basic Information Name Notebook name Description (Option) Notebook description OBS Configuration OBS Bucket OBS bucket for storing .ipynb files NOTE: You must have the permission to upload files to OBS.
You can only select an OBS link. obs_link OBS Bucket This parameter is displayed only when Write Dirty Data Link is a link to OBS.
--jars OBS path of the file. Separate multiple paths by pressing Enter. (optional) --py-files OBS path of the file. Separate multiple paths by pressing Enter. (optional) --files OBS path of the file. Separate multiple paths by pressing Enter.
