配置多队列的容量调度器配置文件capacity-scheduler.xml报错

admin管理员组

文章数量:1611539

项目场景：

提示： ：

---

问题描述：配置完capacity-scheduler.xml之后启动集群，resourcemanager报错。

ERROR: Cannot set priority of resourcemanager process 5501

---

原因分析：

一般遇到这种情况，第一反应应该是去查看日志，因为bing得到的信息可能大家的情况并不是都一样，比如我这个小问题基本上就不会有人写出来。

---

解决方案：

首先查看出错的信息在哪一台服务器上，我的是在hadoop103上，就去这台服务器上找对应的日志

vim /opt/module/hadoop-3.1.3/logs/hadoop-root-resourcemanager-hadoop103.log

从日志中很容易看出来问题，是配置文件中的标签闭合问题，在修改配置文件的时候，不小心删掉了<导致这样的结果。修改好配置文件后，集群还好是不能正常启动。

2021-06-14 03:30:05,286 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler failed in state INITED
java.lang.IllegalArgumentException: Illegal capacity of 0.4 for children of queue root
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.setChildQueues(ParentQueue.java:196)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:287)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.initializeQueues(CapacitySchedulerQueueManager.java:158)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:715)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:360)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:425)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:819)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1224)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:318)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1506)
2021-06-14 03:30:05,287 INFO org.apache.hadoop.service.AbstractService: Service RMActiveServices failed in state INITED
java.lang.IllegalArgumentException: Illegal capacity of 0.4 for children of queue root

问题是 Illegal capacity of 0.4 for children of queue root 说明是配置文件中给定的队列容量出现问题，知道队列中的相关配置位置

  <property>
    <name>yarn.scheduler.capacity.root.default.capacity</name>
    <value>40</value>
    <description>Default queue target capacity.</description>
  </property>
  <property>
    <name>yarn.scheduler.capacity.hive.fault.capacity</name>
    <value>60</value>
    <description>Default queue target capacity.</description>
  </property>

首先倒数第四行错误，出现的主要原因 ，修改错位置，并且insert键打开了导致出错。

  <property>
    <name>yarn.scheduler.capacity.root.default.capacity</name>
    <value>40</value>
    <description>Default queue target capacity.</description>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.hive.capacity</name>
    <value>60</value>
    <description>Default queue target capacity.</description>
  </property>

修改后就OK了

=============== hadoop102 ===============
22448 Jps
21891 DataNode
22377 JobHistoryServer
22190 NodeManager
21727 NameNode
=============== hadoop103 ===============
10181 DataNode
10376 ResourceManager
10700 NodeManager
10975 Jps
=============== hadoop104 ===============
14266 NodeManager
14187 SecondaryNameNode
14411 Jps
14078 DataNode

本文标签： 队列报错配置文件容量Capacity