Product Advantages

  • Compatibility and Openness

    DLI seamlessly migrates offline Spark applications to the cloud. Leveraging the open-source Apache Spark and Flink ecosystems and APIs, DLI makes migration easier.

  • Batch-Stream Convergence

    DLI provides a high-scalable framework integrating batch and stream processing, allowing you to handle data analysis requests in countless scenarios with ease.

  • Outstanding Performance

    With a deeply optimized kernel and architecture, DLI delivers 100-fold performance improvement compared with the MapReduce model. Your analysis is backed by an industry-vetted 99.95% SLA.

  • Lower Cost

    DLI is billed in compute units (CUs) based on time used. One CU contains one vCPU and 4 GB memory. DLI costs $0.35 USD per CU per hour.

Functions

  • Standard and Complex SQL Compute

    Standard ANSI SQL 2003 and CEP SQL are supported
    Standard ANSI SQL 2003 and CEP SQL are supported

  • Federated Analysis of Heterogeneous Data Sources

    Datasource analysis of multiple data formats and SQL on AI intelligent analysis are supported.
    Datasource analysis of multiple data formats and SQL on AI intelligent analysis are supported.
  • Standard ANSI SQL 2003
    Standard ANSI SQL 2003
    JDBC APIs and SDKs can be used to run standard ANSI SQL 2003. For both online and offline data, you only need to compile SQL statements to analyze and collect statistics, detect outliers, perform real-time clustering, and analyze time series. SQL engines' deployment and O&M will no longer be your concern.
    JDBC APIs and SDKs can be used to run standard ANSI SQL 2003. For both online and offline data, you only need to compile SQL statements to analyze and collect statistics, detect outliers, perform real-time clustering, and analyze time series. SQL engines' deployment and O&M will no longer be your concern.
  • CEP SQL
    CEP SQL
    Pattern matching and detection based on Match Recognize is supported, allowing you to perform anomaly detection based on complex event rules using SQL. You can apply this function in various scenarios, such as fraud detection, abnormal vehicle behavior detection, and abnormal running status detection for industrial devices.
    Pattern matching and detection based on Match Recognize is supported, allowing you to perform anomaly detection based on complex event rules using SQL. You can apply this function in various scenarios, such as fraud detection, abnormal vehicle behavior detection, and abnormal running status detection for industrial devices.
  • Federated Analysis of Heterogeneous Data Sources
    Federated Analysis of Heterogeneous Data Sources
    DLI supports various formats, such as CSV, JSON, Parquet, ORC, and CarbonData. It performs federated analysis of data from multiple cloud services (for example, OBS, DWS, CloudTable, and RDS) with data migration, helping you innovate faster and obtain valuable insights from your data.
    DLI supports various formats, such as CSV, JSON, Parquet, ORC, and CarbonData. It performs federated analysis of data from multiple cloud services (for example, OBS, DWS, CloudTable, and RDS) with data migration, helping you innovate faster and obtain valuable insights from your data.
  • SQL on AI
    SQL on AI
    DLI integrates the processing and analyzing of images, videos, and languages in SQL to offer convergent analysis for both structured and unstructured data.
    DLI integrates the processing and analyzing of images, videos, and languages in SQL to offer convergent analysis for both structured and unstructured data.

  • Serverless Services

    Cost-effective DLI with Serverless Spark, Auto Scaling, and Pay-per-use Billing
    Cost-effective DLI with Serverless Spark, Auto Scaling, and Pay-per-use Billing

  • Enterprise-Class Multi-Tenancy

    Computing resources are isolated between tenants to meet job SLAs. You can restrict your data permissions to a specific table or column for data sharing between departments and permissions management.
    Computing resources are isolated between tenants to meet job SLAs. You can restrict your data permissions to a specific table or column for data sharing between departments and permissions management.
  • Serverless Spark
    Serverless Spark
    DLI offers full-stack Spark capabilities, such as Spark SQL, Spark Streaming, and Spark Batch based on the Apache Spark ecosystem. You can use DLI to analyze data at the TB-EB scale with standard SQL statements or Spark APIs.
    DLI offers full-stack Spark capabilities, such as Spark SQL, Spark Streaming, and Spark Batch based on the Apache Spark ecosystem. You can use DLI to analyze data at the TB-EB scale with standard SQL statements or Spark APIs.
  • Auto Scaling
    Auto Scaling
    Auto scaling of storage and computing resources allows you to query data without worrying about whether you have sufficient resources.
    Auto scaling of storage and computing resources allows you to query data without worrying about whether you have sufficient resources.

Application Scenarios

  • Large-scale Log Analysis

  • Federated Analysis of Heterogeneous Data Sources

  • Big Data ETL

  • Geographic Big Data Analysis

Large-scale Log Analysis

Operational Data Analysis

Different departments of a company can analyze daily logs via the data analysis platform to obtain data required for intelligent decision making. For example, operational departments may use the platform to obtain data on new users, active users, the retention and churn rates, and the payment rates and determine follow-up actions based on the data. The delivery department can use the platform to obtain the channel sources of new and active users to help them determine what internal platforms to allocate resources to.

Advantages

Efficient Spark Programming

DLI uses Spark Streaming to directly ingest and preprocess data from DIS. You only need to edit the processing logic. There is no need to deal with multi-threading.

Easy to Use

You can use standard SQL statements to compile metric analysis logic. There is no need to navigate a complex distributed computing platform.

Pay-per-Use

Log analysis is scheduled periodically based on time-critical requirements. There is a long idle period between every two scheduling operations. DLI is billed for usage, which is at least 50% cheaper than purchasing exclusive clusters. You only pay for the resources actually used for scheduling.

Federated Analysis of Heterogeneous Data Sources

Digital Transformation for Car Companies

In the face of new competition pressures and changes in travel services, car companies build the IoV cloud platform and IVI OS to streamline Internet applications and vehicle use scenarios, completing digital service transformation for car companies. This delivers better travel experience for vehicle owners, increases the competitiveness of car companies, and promotes sales growth. For example, collect and analyze daily vehicle metric data (such as batteries, engines, tire pressure, and airbags), and send maintenance suggestions to vehicle owners in time.

Advantages

No Need for Migration in Multi-source Data Analysis

RDS stores the basic information about vehicles and vehicle owners, CloudTable stores real-time vehicle location data and health status details, while DWS stores periodic statistics. DLI allows for federated analysis of data from multiple sources without data migration.

Tiered Data Storage

Car companies need to archive all historical data for auditing and other services that require only occasional data access. Warm and cold data is stored in OBS and frequently accessed data is stored in CloudTable and DWS, reducing the overall storage cost.

Big Data ETL

Carrier Big Data Analysis

Carriers typically require petabytes, or even exabytes of data storage, for both structured (base station details) and unstructured (messages and communications) data. They need to be able to access the data with extremely low data latency. Extracting value from this data efficiently is a major challenge. DLI provides multi-mode engines such as batch processing and stream processing to break down data silos and perform unified data analysis.

Advantages

Big Data ETL

You can enjoy TB to EB-level data governance capabilities to quickly perform ETL processing on massive carrier data. Distributed datasets are provided for batch processing

High Throughput, Low Latency

DLI uses the Dataflow model of Apache Flink, a real-time computing framework. High-performance computing resources
are provided to consume data from your created Kafka, DMS Kafka, and MRS Kafka clusters. A single CU processes 1,000 to 20,000 messages per second

Fine-grained Permissions Management

Your company may have numerous departments, where data needs to be shared and isolated. Using DLI, you can apply for resource queues by tenant to isolate computing resources (CPUs and memory), ensuring job SLA. DLI supports table- or column-level data permission control, allowing for secure access for different departments 

Geographic Big Data Analysis

Geographic Big Data Analysis

Geographic big data has all the characteristics typical of big data. It features large data volume (for example, PB-scale global satellite remote sensing image data) and numerous data varieties (for example, structured remote sensing image raster data, vector data, unstructured spatial location data, and 3D modeling data). Users focus on how to use efficient mining tools or mining methods to get insights from the large volume of geographic big data.

Advantages

Spatial Data Analysis Operators

With full-stack Spark capabilities and rich Spark spatial data analysis algorithm operators, DLI delivers comprehensive support for real-time processing of dynamic streaming data with location attributes and offline batch processing. DLI can handle massive data, including structured remote sensing image data, unstructured 3D modeling, and laser point cloud data.

CEP SQL

DLI delivers geographical location analysis functions to analyze geospatial data in real time. You can fulfill yaw detection and geo-fencing through SQL statements.

Big Data Processing

DLI allows you to quickly migrate remote sensing image data at the TB to EB scale to the cloud and perform image data slicing to offer resilient distributed datasets (RDDs) for distributed batch computing.

Success Stories
alt-logo-car
Chengdu Longyuan Network works with HUAWEI CLOUD to query and analyze gaming data in an efficient manner. The analysis provides support for different departments launching new services. Data applications are integrated, benefitting the entire organization.
Chengdu Longyuan Network

New Features

Documentation

Provides Scalable, On-demand Computing Resources

Learn More