检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
You can configure a lifecycle policy in OBS to periodically delete these temporary data. * 2. Submit the LoadData statement to DLI to import data to DLI. For details, see Importing Data. * 3.
Delta Cleansing and Optimization Cleansing a Delta Table You can run the VACUUM command on a Delta table to remove data files that are no longer referenced and were created before the retention threshold. VACUUM delta_table0; VACUUM delta_table0 RETAIN 168 HOURS;--The unit can only
Delta DDL Syntax CREATE TABLE DROP TABLE DESCRIBE ADD CONSTRAINT DROP CONSTRAINT CONVERT TO DELTA SHALLOW CLONE Parent topic: Delta SQL Syntax Reference
DLI Delta Metadata For how to submit a Spark SQL job in DLI using the Delta SQL syntax, see Delta SQL Syntax Reference. For how to submit a Spark Jar job in DLI using Delta, see Using Delta to Submit a Spark Jar Job in DLI. DLI Delta Metadata Description When creating a Delta table
Typical Delta Configurations To set Delta parameters while submitting a DLI Spark SQL job, access the SQL Editor page and click Settings in the upper right corner. In the Parameter Settings area, set the parameters. Table 1 Typical Delta configurations Parameter Description Default
Delta Time Travel Viewing History Operation Records of a Delta Table Querying History Version Data of a Delta Table Restoring a Delta Table to an Earlier State
DLI Delta FAQ When Executing insert into/overwrite table_name partition(part_key='part_value') select ..., Error DLI.0005: DeltaAnalysisException: Partition column 'dt' not found in schema [id, name] Occurs Root cause analysis: The syntax insert into/overwrite table_name partition
DLI Delta Table Overview Delta tables use the Delta Lake technology to offer a robust data storage solution. By extending Parquet data files with file-based transaction logs, they support ACID transactions and scalable metadata. Delta Lake seamlessly integrates with Apache Spark APIs
Delta SQL Syntax Reference Delta DDL Syntax Delta DML Syntax Schema Evolution Syntax
Restoring a Delta Table to an Earlier State Syntax Restore a Delta table to the state at a specific point in time: RESTORE [TABLE] [database_name.]table_name|DELTA.`obs_path` [TO] TIMESTAMP AS OF timestamp_expression Restore a Delta table to the state at a specific historical version
Using Delta to Develop Jobs in DLI DLI Delta Metadata Using Delta to Submit a Spark Jar Job in DLI
Querying History Version Data of a Delta Table Syntax Query the state of a Delta table at a specific point in time: SELECT * FROM [database_name.]table_name TIMESTAMP AS OF timestamp_expression Query the state of a Delta table at a specific historical version: SELECT * FROM [database_name
When connecting DLI to a LakeFormation instance, you cannot specify filter criteria to delete partitions. When connecting DLI to a LakeFormation instance, you cannot create Truncate Datasource or Hive foreign tables.
Using Delta to Submit a Spark Jar Job in DLI 1. Add the following dependencies: <dependency> <groupId>io.delta</groupId> <artifactId>delta-core_2.12</artifactId> <version>2.3.0</version> </dependency> 2. Add the following parameters to SparkSession: .config("spark.sql.extensions
