GKE扩展内存和Kubernetes内存可分配(GKE extended memory and Kubernetes memory allocatable)

GKE扩展内存和Kubernetes内存可分配(GKE extended memory and Kubernetes memory allocatable)

在GKE上创建集群时，可以创建自定义实例类型 。 向n1-standard-1 Kubernetes添加8GB内存时，仅显示可分配6.37GB内存。 为什么是这样？

请求的内存包括kube-system命名空间中的所有pod，那么这个额外的内存在哪里？

When creating a cluster on GKE its possible to create Custom Instance Types. When adding 8GB of memory to an n1-standard-1 Kubernetes only shows memory allocatable of 6.37GB. Why is this?

The requested memory includes all the pods in kube-system namespace so where is this extra memory going?

最满意答案

来自文档的 Quotinig：

节点可分配资源

请注意 ，运行Kubernetes Engine和Kubernetes资源需要一些节点资源，以使该节点作为集群的一部分运行。 因此，您可能会注意到节点的总资源（在机器类型文档中指定）与Kubernetes Engine中节点的可分配资源之间存在差异

注意 ：由于较大的机器类型倾向于运行更多容器（并且通过扩展，Kubernetes pod），因此Kubernetes Engine为集群进程预留的资源量会因较大的机器而向上扩展。

警告 ：在1.7.6之前的Kubernetes Engine节点版本中，保留的资源不计入节点的可分配总资源。 如果您的节点最近已升级到1.7.6版，则它们可能看起来可用资源较少，因为Kubernetes Engine现在显示可分配资源。 这可能会导致群集节点出现过度使用，因此您可能希望调整群集大小。

例如，执行一些测试，您可以双重检查：

Machine type Memory(GB) Allocatable(GB) CPU(cores) Allocatable(cores) g1-small 1.7 1.2 0.5 0.47 n1-standard-1 (default) 3.75 2.7 1 0.94 n1-standard-2 7.5 5.7 2 1.93 n1-standard-4 15 12 4 3.92 n1-standard-8 30 26.6 8 7.91 n1-standard-16 60 54.7 16 15.89

注意 ：为可分配资源列出的值不考虑kube-system pod使用的资源，其数量随每个Kubernetes版本而变化。 这些系统盒通常在每个节点上占用额外的400m CPU和400mi内存（值是近似值）。 如果您需要在每个节点上精确计算可用资源，建议您直接检查群集。

UPDATE

Kubernetes文档中还有关于为何使用此资源的官方解释：

kube-reserved用于捕获kubernetes系统守护进程的资源预留，例如kubelet，容器运行时，节点问题检测器等。 它并不意味着为作为pod运行的系统守护进程保留资源 。 kube-reserved通常是节点上pod密度的函数。 此性能仪表板在多个pod密度级别公开了kubelet和docker引擎的cpu和内存使用情况。 此博客文章解释了如何解释仪表板以提出合适的kube保留预订。

如果您有兴趣了解更多，我建议您继续阅读本页。

Quotinig from documentation:

Node allocatable resources

Note that some of a node's resources are required to run the Kubernetes Engine and Kubernetes resources necessary to make that node function as part of your cluster. As such, you may notice a disparity between your node's total resources (as specified in the machine type documentation) and the node's allocatable resources in Kubernetes Engine

Note: As larger machine types tend to run more containers (and by extension, Kubernetes pods), the amount of resources that Kubernetes Engine reserves for cluster processes scales upward for larger machines.

Caution: In Kubernetes Engine node versions prior to 1.7.6, reserved resources were not counted against a node's total allocatable resources. If your nodes have recently upgraded to version 1.7.6, they might appear to have fewer resources available, as Kubernetes Engine now displays allocatable resources. This can potentially lead to your cluster's nodes appearing overcommitted, and you might want to resize your cluster as a result.

For example performing some tests you can doublecheck:

Machine type Memory(GB) Allocatable(GB) CPU(cores) Allocatable(cores) g1-small 1.7 1.2 0.5 0.47 n1-standard-1 (default) 3.75 2.7 1 0.94 n1-standard-2 7.5 5.7 2 1.93 n1-standard-4 15 12 4 3.92 n1-standard-8 30 26.6 8 7.91 n1-standard-16 60 54.7 16 15.89

Note: The values listed for allocatable resources do not account for the resources used by kube-system pods, the amount of which varies with each Kubernetes release. These system pods generally occupy an additional 400m CPU and 400mi memory on each node (values are approximate). It is recommended that you directly inspect your cluster if you require an exact accounting of usable resources on each node.

UPDATE

There is also the official explanation from the Kubernetes Documentation regarding why this resources are used:

kube-reserved is meant to capture resource reservation for kubernetes system daemons like the kubelet, container runtime, node problem detector, etc. It is not meant to reserve resources for system daemons that are run as pods. kube-reserved is typically a function of pod density on the nodes. This performance dashboard exposes cpu and memory usage profiles of kubelet and docker engine at multiple levels of pod density. This blog post explains how the dashboard can be interpreted to come up with a suitable kube-reserved reservation.

I would suggest you to go thorugh this page if you are interested to learn more.