Data Lake Factory

Data Lake Factory (DLF) is a one-stop Big Data collaboration development platform, enabling operations such as data modeling, data integration, script development, job scheduling, and job monitoring. Thanks to DLF, Big Data is more accessible than ever before, helping you quickly build Big Data processing centers.

DLF is free of charge before Dec 31, 2018.

Learn more
Product Advantages
  • One-Stop Data Warehouse Building

    DLF supports one-stop building of cloud data warehouses, where you can complete data integration, script development, job development, job scheduling, job monitoring, and data management without the need for multiple tools.

  • Data Lake Development

    DLF manages a variety of Big Data services such as DWS and DLI and allows data to be orchestrated and scheduled in different types of data services.

  • Diverse Data Types

    DLF allows online collaborative development, supports online editing of SQL and Shell scripts and real-time script query, and enables job development for types of data processing nodes such as Data Migration, SQL, MR, Shell, Machine Learning, and Spark.

  • Powerful Job Scheduling Capabilities

    DLF provides you with diverse scheduling policies and powerful scheduling capabilities, supporting manual, periodic, and event-driven scheduling.

Application Scenarios
  • Service Flow Automation

  • BI Report Automation

  • Cloud Data Warehouse Building

  • Log Analysis and Mining

Service Flow Automation

Automated Data Analysis Service Flow

DLF automates the E2E procedure from data import, data cleansing, machine learning, data backhaul, to report generation.

Advantages

Job Orchestration for Multiple Cloud Services

Orchestrates jobs of multiple cloud services, such as data migration, MR, Spark, Machine Learning, and SQL.

Easy to Use

Supports GUI-based orchestration and is used out of the box.

BI Report Automation

Automated Generation of Complex BI Reports

DLF enables flexible development and automated generation of BI reports.

Advantages

Comprehensive Script Development

Supports SQL and Shell script development and multiple types of data warehouses, addressing the rapid development requirements of the new reports.

Flexible Automated Scheduling

Develops workflows using drag-and-drop on a GUI, provides flexible dependency and scheduling policies, and automates generation of BI reports.

Cloud Data Warehouse Building

Quick Building of Cloud Data Warehouses

DLF can quickly migrate offline data to the cloud and integrate the data into the Big Data cloud services. With DLF Console, you can directly carry out data development, making data warehouse building easier than ever before.

Advantages

One-click data integration

With a unified configuration on the DLF Console, both online and offline data can be quickly integrated into cloud data warehouses with just one click.

Data warehouse building for required services

You can build data warehouses for whatever services you want, such as DWS, DLI, and HBase.

Secure, stable, and cost-effective

One-stop data warehouse building and stable service provisioning free you from building and maintaining Big Data clusters, dramatically reducing your costs in data warehouse building and improving the security of your data on the cloud.

Related Services

cdm

obs

dws

Log Analysis and Mining

Easy Analysis and Mining of Massive Amounts of Logs

After ingesting logs into OBS or Cloud Search Service through DIS, you can simply use DLF to analyze and mine massive amounts of logs by compiling data development scripts and data mining scripts.

Advantages

Easy to use

Using Big Data analysis and mining capabilities is as simple as using utilities like water or electricity, easily helping explore your business value.

High efficiency

Log data can be quickly analyzed and mined on the one-stop development interface, several times faster than before.

Flexible analysis and mining

You can leverage SQL scripts, MR scripts, Shell scripts, and Machine Learning scripts to perform data analysis and mining for different business.

Functions

  • Data Modeling

    Visually designs data models to adapt to personalized service requirements.

  • Data Integration

    Easily integrates various data sources into data warehouses.

Data Modeling

  • Supported data warehouse types including DWS, DLI, and MRS Hive

  • Visualized table building and data table management

Data Integration

  • Seamless integration of DLF and CDM

  • CDM-enabled data migration among 20+ heterogeneous data sources, driving smooth integration of data sources into data warehouses

  • Data Development

    Edits and debugs scripts online and orchestrates workflows using drag-and-drop.

  • Maintenance & Scheduling

    Supports flexible scheduling, real-time monitoring, simple management & maintenance, and prompt alarming of jobs.

Data Development

  • Online collaborative development of SQL and Shell scripts

  • Drag-and-drop workflow development

  • Multiple supported task types including data integration, SQL, MR, Spark, Shell, machine learning, and REST

Maintenance & Scheduling

  • Flexible scheduling cycles by minute, hour, day, and week

  • Multiple alarming modes and manual retry of jobs

Usage Guides

Create an Account and Experience HUAWEI CLOUD for Free

Register Now