检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
检测到您已登录华为云国际站账号，为了您更好的体验，建议您访问国际站服务网站 https://www.huaweicloud.com/intl/zh-cn
不再显示此消息
Migrating Loader Jobs in Batches Scenario Loader allows jobs to be migrated in batches from a group (source group) to another group (target group). Prerequisites The source group and target group exist. The current user has the Group Edit permission for the source group and target
Deleting Loader Jobs in Batches Scenario Loader allows existing jobs to be deleted in batches. Prerequisites The current user has the Edit permission for the jobs to be deleted or the Jobs Edit permission for the group to which the jobs belong. Procedure Access the Loader web UI.
Importing Loader Jobs in Batches Scenario Loader allows all jobs of a configuration file to be imported in batches. Prerequisites The current user has the Jobs Edit permission of the group to which the jobs to be imported belong. If the group to which the jobs to be imported belong
Exporting Loader Jobs in Batches Scenario Loader allows existing jobs to be exported in batches. Prerequisites The current user has the Edit permission for the jobs to be exported or the Jobs Edit permission of the group to which the jobs belong. Procedure Access the Loader web UI
Using UDFs in FlinkServer Jobs This section applies to MRS 3.1.2 or later clusters. You can customize functions to extend SQL statements to meet personalized requirements. These functions are called user-defined functions (UDFs). You can upload and manage UDF JAR files on the Flink
Managing MRS Cluster Jobs Stopping and Deleting an MRS Cluster Job Configuring Notification Rules for MRS Jobs Parent topic: Managing Clusters
Modifying the OMS Service Configuration Based on the security requirements of the user environment, you can modify the Kerberos and LDAP configurations in the OMS on FusionInsight Manager. This section applies only to MRS 3.x or later. System Impacts After the OMS service configuration
Oozie Jobs Do Not Run When a Large Number of Jobs Are Submitted Concurrently Issue When a large number of Oozie jobs are submitted concurrently, the jobs do not run. Symptom When a large number of Oozie jobs are submitted concurrently, the jobs do not run. Cause Analysis When Oozie
RocksDB State Backend of Flink Jobs This topic is available for MRS 3.3.0 and later versions only. Flink Job RocksDB When RocksDB is enabled as the state backend for jobs, a large amount of state data causes poor read and write performance of RocksDB. You can perform the following
Configuring the Baseline Parameters for MapReduce Jobs Scenario The performance optimization effect is verified by comparing actual values with the baseline data. Therefore, determining optimal job baseline is critical to performance optimization. When determining the job baseline
Viewing the Health Status of FlinkServer Jobs This topic is available for MRS 3.3.0 or later only. Job Health Status When a large number of Flink jobs are running in a cluster, the FlinkServer web UI provides the Flink job health management function to help you evaluate the health
Optimizing Performance for Committing MR Jobs Scenario By default, if an MR job generates a large number of output files, it takes a long time for the job to commit the temporary outputs of a task to the final output directory in the commit phase. In large clusters, the time-consuming
Managing Oozie Jobs Using Hue Scenario Users can use the Hue web UI to query all jobs in an MRS cluster. Hue provides a UI for Oozie job management. The Hue UI is used to view and analyze data such as files and tables. Do not perform high-risk management operations such as deleting
Changing the Password for the OMS Database Administrator You are advised to regularly change the password for the OMS database administrator to enhance system O&M security. Log in to the active management node as user root. To ensure smooth cluster running, you are not allowed to
Configuring Notification Rules for MRS Jobs MRS uses SMN to offer a publish/subscribe model to achieve one-to-multiple message subscriptions and notifications in a variety of message types (SMSs and emails). You can configure job notification rules to receive notifications immediately
Failed to Run Jobs Related to the sftp-connector Connector Symptom The jobs related to the sftp-connector connector fail to be executed and "Failed to obtain the SFTP stream" is displayed. xxx (Failed to send channel request.) Error "subsystem request failed on channel 0. Connection
Configuring the Distributed Cache to Execute MapReduce Jobs Scenarios This section applies to MRS 3.x or later. Distributed caching is useful in the following scenarios: Rolling upgrade During the upgrade, applications must keep the text content (JAR file or configuration file) unchanged
Changing the Password for an OMS Database Access User This section describes how to regularly change the password for an OMS database access user to enhance system O&M security. Impact on the System The OMS service needs to be restarted for the new password to take effect. The service
DataArts Studio Failed to Schedule Spark Jobs Issue DataArts Studio fails to schedule jobs, and a message is displayed indicating that data in the /thriftserver/active_thriftserver directory cannot be read. Symptom DataArts Studio fails to schedule jobs, and the following error is
Where Are the Execution Logs of Spark Jobs Stored? Logs of unfinished Spark jobs are stored in the /srv/BigData/hadoop/data1/nm/containerlogs/ directory on the Core node. Logs of finished Spark jobs are stored in the /tmp/logs/Username/logs directory of the HDFS. Parent topic: Job
