Golang：计算地图的内存占用（或字节长度）(Golang: computing the memory footprint (or byte length) of a map)

我想将地图限制为最大X字节。似乎没有直接计算地图字节长度的方法。

"encoding/binary"包具有很好的Size函数，但它只适用于片或“固定值”，而不适用于映射。

我可以尝试从地图中获取所有的键/值对，推断它们的类型（如果它是一个map[string]interface{} ）并计算长度 - 但这样做既麻烦也可能不正确（因为这会排除“内部“地图本身的golang成本 - 管理指向元素的指针等）。

任何建议的方式这样做？最好是一个代码示例。

I want to limit a map to be maximum X bytes. It seems there is no straightforward way of computing the byte length of a map though.

"encoding/binary" package has a nice Size function, but it only works for slices or "fixed values", not for map.

I could try to get all key/value pairs from the map, infer their type (if it's a map[string]interface{}) and compute the length - but that would be both cumbersome and probably incorrect (because that would exclude the "internal" golang cost of the map itself - managing pointers to elements etc.).

Any suggested way of doing this? preferably a code example.

最满意答案

这是地图标题的定义：

// A header for a Go map. type hmap struct { // Note: the format of the Hmap is encoded in ../../cmd/gc/reflect.c and // ../reflect/type.go. Don't change this structure without also changing that code! count int // # live cells == size of map. Must be first (used by len() builtin) flags uint32 hash0 uint32 // hash seed B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items) buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0. oldbuckets unsafe.Pointer // previous bucket array of half the size, non-nil only when growing nevacuate uintptr // progress counter for evacuation (buckets less than this have been evacuated) }

计算其大小非常简单（unsafe.Sizeof）。

这是地图指向的每个桶的定义：

// A bucket for a Go map. type bmap struct { tophash [bucketCnt]uint8 // Followed by bucketCnt keys and then bucketCnt values. // NOTE: packing all the keys together and then all the values together makes the // code a bit more complicated than alternating key/value/key/value/... but it allows // us to eliminate padding which would be needed for, e.g., map[int64]int8. // Followed by an overflow pointer. }

bucketCnt是一个常量，定义如下：

bucketCnt = 1 << bucketCntBits // equals decimal 8 bucketCntBits = 3

最终的计算是：

unsafe.Sizeof(hmap) + (len(theMap) * 8) + (len(theMap) * 8 * unsafe.Sizeof(x)) + (len(theMap) * 8 * unsafe.Sizeof(y))

其中，地图是您的地图值， x是地图的关键字类型的值， y是地图的值类型的值。

您必须通过程序集与您的程序包共享hmap结构，类似于运行时的thunk.s 。

This is the definition for a map header:

// A header for a Go map. type hmap struct { // Note: the format of the Hmap is encoded in ../../cmd/gc/reflect.c and // ../reflect/type.go. Don't change this structure without also changing that code! count int // # live cells == size of map. Must be first (used by len() builtin) flags uint32 hash0 uint32 // hash seed B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items) buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0. oldbuckets unsafe.Pointer // previous bucket array of half the size, non-nil only when growing nevacuate uintptr // progress counter for evacuation (buckets less than this have been evacuated) }

Calculating its size is pretty straightforward (unsafe.Sizeof).

This is the definition for each individual bucket the map points to:

// A bucket for a Go map. type bmap struct { tophash [bucketCnt]uint8 // Followed by bucketCnt keys and then bucketCnt values. // NOTE: packing all the keys together and then all the values together makes the // code a bit more complicated than alternating key/value/key/value/... but it allows // us to eliminate padding which would be needed for, e.g., map[int64]int8. // Followed by an overflow pointer. }

bucketCnt is a constant defined as:

bucketCnt = 1 << bucketCntBits // equals decimal 8 bucketCntBits = 3

The final calculation would be:

unsafe.Sizeof(hmap) + (len(theMap) * 8) + (len(theMap) * 8 * unsafe.Sizeof(x)) + (len(theMap) * 8 * unsafe.Sizeof(y))

Where theMap is your map value, x is a value of the map's key type and y a value of the map's value type.

You'll have to share the hmap structure with your package via assembly, analogously to thunk.s in the runtime.

更多推荐