这是针对Python 3.6的.
This is for Python 3.6.
编辑并删除了很多无关紧要的内容.
Edited and removed a lot of stuff that turned out to be irrelevant.
我曾经以为json比pickle快,而且关于Stack Overflow的其他答案和评论似乎也使很多其他人也相信这一点.
I had thought json was faster than pickle and other answers and comments on Stack Overflow make it seem like a lot of other people believe this as well.
我的测试是犹太洁食吗?差距比我预期的要大得多.在非常大的对象上进行测试,我得到相同的结果.
Is my test kosher? The disparity is much larger than I expected. I get the same results testing on very large objects.
import json import pickle import timeit file_name = 'foo' num_tests = 100000 obj = {1: 1} command = 'pickle.dumps(obj)' setup = 'from __main__ import pickle, obj' result = timeit.timeit(command, setup=setup, number=num_tests) print("pickle: %f seconds" % result) command = 'json.dumps(obj)' setup = 'from __main__ import json, obj' result = timeit.timeit(command, setup=setup, number=num_tests) print("json: %f seconds" % result)和输出:
pickle: 0.054130 seconds json: 0.467168 seconds推荐答案
我根据您的代码段尝试了几种方法,发现使用cPickle并将dumps方法的协议参数设置为:cPickle.dumps(obj, protocol=cPickle.HIGHEST_PROTOCOL)是最快的转储方法.
I have tried several methods based on your code snippet and found out that using cPickle with setting the protocol argument of the dumps method as: cPickle.dumps(obj, protocol=cPickle.HIGHEST_PROTOCOL) is the fastest dump method.
import msgpack import json import pickle import timeit import cPickle import numpy as np num_tests = 10 obj = np.random.normal(0.5, 1, [240, 320, 3]) command = 'pickle.dumps(obj)' setup = 'from __main__ import pickle, obj' result = timeit.timeit(command, setup=setup, number=num_tests) print("pickle: %f seconds" % result) command = 'cPickle.dumps(obj)' setup = 'from __main__ import cPickle, obj' result = timeit.timeit(command, setup=setup, number=num_tests) print("cPickle: %f seconds" % result) command = 'cPickle.dumps(obj, protocol=cPickle.HIGHEST_PROTOCOL)' setup = 'from __main__ import cPickle, obj' result = timeit.timeit(command, setup=setup, number=num_tests) print("cPickle highest: %f seconds" % result) command = 'json.dumps(obj.tolist())' setup = 'from __main__ import json, obj' result = timeit.timeit(command, setup=setup, number=num_tests) print("json: %f seconds" % result) command = 'msgpack.packb(obj.tolist())' setup = 'from __main__ import msgpack, obj' result = timeit.timeit(command, setup=setup, number=num_tests) print("msgpack: %f seconds" % result)输出:
pickle : 0.847938 seconds cPickle : 0.810384 seconds cPickle highest: 0.004283 seconds json : 1.769215 seconds msgpack : 0.270886 seconds更多推荐
为什么用`pickle`转储比`json`快得多?
发布评论