Product Advantages
-
Compatibility and Openness
DLI seamlessly migrates offline Spark applications to the cloud. Leveraging the open-source Apache Spark and Flink ecosystems and APIs, DLI makes migration easier. -
Batch-Stream Convergence
DLI provides a high-scalable framework integrating batch and stream processing, allowing you to handle data analysis requests in countless scenarios with ease. -
Outstanding Performance
With a deeply optimized kernel and architecture, DLI delivers 100-fold performance improvement compared with the MapReduce model. Your analysis is backed by an industry-vetted 99.95% SLA. -
Lower Cost
DLI is billed in compute units (CUs) based on time used. One CU contains one vCPU and 4 GB memory. DLI costs $0.35 USD per CU per hour.
Functions
-
Standard and Complex SQL Compute
Standard ANSI SQL 2003 and CEP SQL are supportedStandard ANSI SQL 2003 and CEP SQL are supported -
Federated Analysis of Heterogeneous Data Sources
Datasource analysis of multiple data formats and SQL on AI intelligent analysis are supported.Datasource analysis of multiple data formats and SQL on AI intelligent analysis are supported.
-
Standard ANSI SQL 2003Standard ANSI SQL 2003JDBC APIs and SDKs can be used to run standard ANSI SQL 2003. For both online and offline data, you only need to compile SQL statements to analyze and collect statistics, detect outliers, perform real-time clustering, and analyze time series. SQL engines' deployment and O&M will no longer be your concern.JDBC APIs and SDKs can be used to run standard ANSI SQL 2003. For both online and offline data, you only need to compile SQL statements to analyze and collect statistics, detect outliers, perform real-time clustering, and analyze time series. SQL engines' deployment and O&M will no longer be your concern.
-
CEP SQLCEP SQLPattern matching and detection based on Match Recognize is supported, allowing you to perform anomaly detection based on complex event rules using SQL. You can apply this function in various scenarios, such as fraud detection, abnormal vehicle behavior detection, and abnormal running status detection for industrial devices.Pattern matching and detection based on Match Recognize is supported, allowing you to perform anomaly detection based on complex event rules using SQL. You can apply this function in various scenarios, such as fraud detection, abnormal vehicle behavior detection, and abnormal running status detection for industrial devices.
-
Federated Analysis of Heterogeneous Data SourcesFederated Analysis of Heterogeneous Data SourcesDLI supports various formats, such as CSV, JSON, Parquet, ORC, and CarbonData. It performs federated analysis of data from multiple cloud services (for example, OBS, DWS, CloudTable, and RDS) with data migration, helping you innovate faster and obtain valuable insights from your data.DLI supports various formats, such as CSV, JSON, Parquet, ORC, and CarbonData. It performs federated analysis of data from multiple cloud services (for example, OBS, DWS, CloudTable, and RDS) with data migration, helping you innovate faster and obtain valuable insights from your data.
-
SQL on AISQL on AIDLI integrates the processing and analyzing of images, videos, and languages in SQL to offer convergent analysis for both structured and unstructured data.DLI integrates the processing and analyzing of images, videos, and languages in SQL to offer convergent analysis for both structured and unstructured data.
-
Serverless Services
Cost-effective DLI with Serverless Spark, Auto Scaling, and Pay-per-use BillingCost-effective DLI with Serverless Spark, Auto Scaling, and Pay-per-use Billing -
Enterprise-Class Multi-Tenancy
Computing resources are isolated between tenants to meet job SLAs. You can restrict your data permissions to a specific table or column for data sharing between departments and permissions management.Computing resources are isolated between tenants to meet job SLAs. You can restrict your data permissions to a specific table or column for data sharing between departments and permissions management.
-
Serverless SparkServerless SparkDLI offers full-stack Spark capabilities, such as Spark SQL, Spark Streaming, and Spark Batch based on the Apache Spark ecosystem. You can use DLI to analyze data at the TB-EB scale with standard SQL statements or Spark APIs.DLI offers full-stack Spark capabilities, such as Spark SQL, Spark Streaming, and Spark Batch based on the Apache Spark ecosystem. You can use DLI to analyze data at the TB-EB scale with standard SQL statements or Spark APIs.
-
Auto ScalingAuto ScalingAuto scaling of storage and computing resources allows you to query data without worrying about whether you have sufficient resources.Auto scaling of storage and computing resources allows you to query data without worrying about whether you have sufficient resources.
Application Scenarios
-
Large-scale Log Analysis
-
Federated Analysis of Heterogeneous Data Sources
-
Big Data ETL
-
Geographic Big Data Analysis
Operational Data Analysis
Different departments of a company can analyze daily logs via the data analysis platform to obtain data required for intelligent decision making. For example, operational departments may use the platform to obtain data on new users, active users, the retention and churn rates, and the payment rates and determine follow-up actions based on the data. The delivery department can use the platform to obtain the channel sources of new and active users to help them determine what internal platforms to allocate resources to.
Advantages
DLI uses Spark Streaming to directly ingest and preprocess data from DIS. You only need to edit the processing logic. There is no need to deal with multi-threading.
You can use standard SQL statements to compile metric analysis logic. There is no need to navigate a complex distributed computing platform.
Log analysis is scheduled periodically based on time-critical requirements. There is a long idle period between every two scheduling operations. DLI is billed for usage, which is at least 50% cheaper than purchasing exclusive clusters. You only pay for the resources actually used for scheduling.
Digital Transformation for Car Companies
In the face of new competition pressures and changes in travel services, car companies build the IoV cloud platform and IVI OS to streamline Internet applications and vehicle use scenarios, completing digital service transformation for car companies. This delivers better travel experience for vehicle owners, increases the competitiveness of car companies, and promotes sales growth. For example, collect and analyze daily vehicle metric data (such as batteries, engines, tire pressure, and airbags), and send maintenance suggestions to vehicle owners in time.
Advantages
RDS stores the basic information about vehicles and vehicle owners, CloudTable stores real-time vehicle location data and health status details, while DWS stores periodic statistics. DLI allows for federated analysis of data from multiple sources without data migration.
Car companies need to archive all historical data for auditing and other services that require only occasional data access. Warm and cold data is stored in OBS and frequently accessed data is stored in CloudTable and DWS, reducing the overall storage cost.
Carrier Big Data Analysis
Carriers typically require petabytes, or even exabytes of data storage, for both structured (base station details) and unstructured (messages and communications) data. They need to be able to access the data with extremely low data latency. Extracting value from this data efficiently is a major challenge. DLI provides multi-mode engines such as batch processing and stream processing to break down data silos and perform unified data analysis.
Advantages
You can enjoy TB to EB-level data governance capabilities to quickly perform ETL processing on massive carrier data. Distributed datasets are provided for batch processing
DLI uses the Dataflow model of Apache Flink, a real-time computing framework. High-performance computing resources
are provided to consume data from your created Kafka, DMS Kafka, and MRS Kafka clusters. A single CU processes 1,000 to 20,000 messages per second
Your company may have numerous departments, where data needs to be shared and isolated. Using DLI, you can apply for resource queues by tenant to isolate computing resources (CPUs and memory), ensuring job SLA. DLI supports table- or column-level data permission control, allowing for secure access for different departments
Geographic Big Data Analysis
Geographic big data has all the characteristics typical of big data. It features large data volume (for example, PB-scale global satellite remote sensing image data) and numerous data varieties (for example, structured remote sensing image raster data, vector data, unstructured spatial location data, and 3D modeling data). Users focus on how to use efficient mining tools or mining methods to get insights from the large volume of geographic big data.
Advantages
With full-stack Spark capabilities and rich Spark spatial data analysis algorithm operators, DLI delivers comprehensive support for real-time processing of dynamic streaming data with location attributes and offline batch processing. DLI can handle massive data, including structured remote sensing image data, unstructured 3D modeling, and laser point cloud data.
DLI delivers geographical location analysis functions to analyze geospatial data in real time. You can fulfill yaw detection and geo-fencing through SQL statements.
DLI allows you to quickly migrate remote sensing image data at the TB to EB scale to the cloud and perform image data slicing to offer resilient distributed datasets (RDDs) for distributed batch computing.
