数据湖探索 DLI-Spark作业相关:提交批处理作业
时间:2025-06-11 15:15:00
提交批处理作业
DLI 提供执行批处理作业的接口。您可以使用该接口执行批处理作业。示例代码如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
def submit_spark_batch_job(dli_client, batch_queue_name, batch_job_info): try: batch_job = dli_client.submit_spark_batch_job(batch_queue_name, batch_job_info) except DliException as e: print(e) return print(batch_job.job_id) while True: time.sleep(3) job_status = batch_job.get_job_status() print('Job status: {0}'.format(job_status)) if job_status == 'dead' or job_status == 'success': break logs = batch_job.get_driver_log(500) for log_line in logs: print(log_line) |
support.huaweicloud.com/sdkreference-dli/dli_04_0139.html