检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Connecting OBS to Big Data Components Supported Big Data Components Connecting Hadoop to OBS Connecting Hive to OBS Connecting Spark to OBS Connecting Presto to OBS Connecting Flume to OBS Connecting DataX to OBS Connecting Druid to OBS Connecting Flink to OBS Connecting Logstash
Automatically Creating an OBS Volume Through kubectl Scenario During the use of OBS, expected OBS buckets can be automatically created and mounted as volumes.
Authorizing Access to an OBS Bucket Function Authorizes VOD to access an OBS bucket or cancels the authorization URI PUT /v1.0/{project_id}/asset/authority Table 1 Path Parameters Parameter Mandatory Type Description project_id Yes String Project ID.
API Overview of OBS SDK for Python Table 1 describes the APIs provided by OBS SDK for Python. You can click an API name in the table to see its detailed information and sample code.
OBS Result Table Function The FileSystem result (sink) table is used to export data to the HDFS or OBS file system. It is applicable to scenarios such as data dumping, big data analysis, data backup, and active, deep, or cold archiving.
Delete OBS Constraints This function depends on OBS. Functions The Delete OBS node is used to delete a bucket or directory on OBS. Parameters Table 1 and Table 2 describe the parameters of the Delete OBS node.
Naming OBS Objects The OBS object names supported by GES can contain the following characters: Letters and digits 0-9, a-z, A-Z Special characters ! - _ . * ' ( ) The following characters are not supported: Special characters \{^}%`]">[~<#|&@:,$=+?
OBS Path Yes Path to the OBS bucket or directory. To create a bucket, enter //OBS bucket name. The OBS bucket name must be unique To create an OBS directory, select the path to the OBS directory to be created, and enter the /Directory name following the path.
Using OBS You can manage OBS resources in the ways listed in the table below. Tool Description How to Use Reference OBS Console OBS Console is a web-based GUI. You can manage all your OBS resources through this console. Create an account or an IAM user to log in to OBS Console.
You can only access OBS using the OBS domain name in the region where the VPC endpoint is located. Configure an OBS route from your on-premises data center to the Direct Connect or VPN gateway. The IP address of OBS belongs to 100.125.0.0/16.
OBS and HDFS differ in consistency assurance.
Parent Topic: Connecting OBS to Big Data Components
/conf/file2obs.conf Parent Topic: Connecting OBS to Big Data Components
Example: txtfilereader is the source, and OBS is the destination.
Compatibility Between OBS APIs and PFS You can call some OBS APIs to use PFS. There may be additional requirements when you call these APIs. For details about the OBS APIs, see Object Storage Service API Reference.
Parent Topic: Connecting Big Data Platforms to OBS
Check whether the connection is successful: $SPARK_HOME/bin/run-example org.apache.spark.examples.JavaWordCount obs://obs-bucket/input/test.txt Parent Topic: Connecting OBS to Big Data Components
by ","; insert into table student select 6,"yangdong",29; Parent Topic: Connecting OBS to Big Data Components
Where Is Data Stored in OBS? When creating a bucket on OBS, you can specify a region for the bucket. Then your data on OBS is stored on multiple storage devices in this region. Parent topic: OBS Basics
/test/ obs://your-bucket/copy-data/ # Download OBS file to local path ma-cli obs-copy obs://your-bucket/copy-data/test.zip ./test.zip # Download OBS directory to local path ma-cli obs-copy obs://your-bucket/copy-data/ .
