大约80000个项目的列表在python中会消耗多少内存?

本文介绍了大约80000个项目的列表在python中会消耗多少内存?的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我有一个python列表，其中包含80000个列表.这些内部列表中的每一个或多或少都具有以下格式:

I have a python list, which consists of 80000 lists. Each of these inner lists more or less have this format:

["012345", "MYNAME" "Mon", "A", 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20]

您能告诉我们这个由80000个列表组成的列表消耗多少内存吗?

Could you tell approximately how much memory would this list consisting of 80000 lists consume?

在python中很大的列表上使用和操作是否常见/确定?我要做的大多数操作是使用列表理解方法从此列表中提取数据.

And is it common/OK to use and operate on lists that big in python? Most of the operations I do is to extract data from this list with list comprehension method.

实际上，我想学习的是:python足够快，可以使用列表理解方法从大列表中提取数据.我希望我的脚本很快

Actually, what I would like to learn is: is python fast enough to extract data from that big lists using list comprehension methods. I want my script to be fast

推荐答案

In [39]: lis=["012345", "MYNAME" "Mon", "A", 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20] In [40]: k=[lis[:] for _ in xrange(80000)] In [41]: k.__sizeof__() Out[41]: 325664 In [42]: sys.getsizeof(k) #after gc_head Out[42]: 325676

按照 sysmodule.c 中的代码看起来它调用了__sizeof__方法来获取对象的大小.

As per the code in sysmodule.c it looks like it calls __sizeof__ method to get the size of an object.

837 method = _PyObject_LookupSpecial(o, &PyId___sizeof__); 838 if (method == NULL) { 839 if (!PyErr_Occurred()) 840 PyErr_Format(PyExc_TypeError, 841 "Type %.100s doesn't define __sizeof__", 842 Py_TYPE(o)->tp_name); 843 } 844 else { 845 res = PyObject_CallFunctionObjArgs(method, NULL); 846 Py_DECREF(method); 847 }

，然后增加一些gc开销:

860 /* add gc_head size */ 861 if (PyObject_IS_GC(o)) { 862 PyObject *tmp = res; 863 res = PyNumber_Add(tmp, gc_head_size); 864 Py_DECREF(tmp); 865 } 866 return res; 867 }

我们还可以按照文档以递归方式计算每个容器的大小:

We can also use the recursive sizeof recipe as suggested in docs to recursively calculate the size of each container:

In [17]: total_size(k) #from recursive sizeof recipe Out[17]: 13125767 In [18]: sum(y.__sizeof__() for x in k for y in x) Out[18]: 34160000