cPickle替代？

编程入门行业动态更新时间:2024-10-27 02:18:54

本文介绍了cPickle替代？的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！问题描述

你好，我把一个非常简单的结构加载到内存中有一个很大的问题它是一个元组列表，它有6MB，由100000个元素组成/>

import cPickle plik = open（" mealy"，" r"） mealy = cPickle.load（plik） plik.close（）

这需要大约30秒！如何加速它？谢谢在adv。

解决方案

Drochom写道：

你好，我将非常简单的结构加载到内存中有一个很大的问题它是一个元组列表，它有6MB，由100000个元素组成

import cPickle

plik = open（mealy，r） mealy = cPickle.load（plik） plik.close（）

<这需要大约30秒！如何我可以加速它吗？谢谢你。

你用什么协议来腌制你的数据？默认（协议0， ASCII文本）是最慢的。我建议你升级到Python 2.3和用新协议2保存你的数据 - 它可能是最快的。 Alex

我不知道！前几天我使用了类似的方案并制作了一些基准测试（我*喜欢*基准测试！）大约6 MB花了4秒钟倾倒以及在800 Mhz P3笔记本电脑上加载。当使用二进制模式时，它下降到大约1.5秒（空间到2 MB）这没关系，因为我通常遇到的问题比我的2英寸/秒更快地超过1 MB /秒驱动器，处理器和Python ;-) Python 2.3似乎甚至有一个更有效的协议模式2。可能是你的结构非常*非常复杂？亲切的 Michael P " Drochom" < PE ****** @ gazeta.pl> schrieb im Newsbeitrag 新闻：bh ********** @ atlantis.news.tpi.pl ...

你好，我将非常简单的结构加载到内存中有一个很大的问题它是一个元组列表，它有6MB，由100000个元素组成

import cPickle

plik = open（mealy，r） mealy = cPickle.load（plik） plik.close（）

<这需要大约30秒！我怎样才能加速它？谢谢你。

Drochom写道：

你用什么协议来腌制你的数据？默认（协议0， ASCII文本）是最慢的。我建议您升级到Python 2.3并使用新协议2保存数据 - 它可能是最快的。 Alex

谢谢:) 我使用的是默认协议，我不确定我是否可以升级如此简单，因为我正在使用Py2.2的许多模块

然后使用协议1 - 这是二进制泡菜协议很长一段时间，并且在Python 2.2.x上完美运行:-) （它比协议0 - 文本协议快得多） --Irmen

Hello, I have a huge problem with loading very simple structure into memory it is a list of tuples, it has 6MB and consists of 100000 elements

import cPickle plik = open("mealy","r")mealy = cPickle.load(plik)plik.close()

this takes about 30 seconds! How can I accelerate it? Thanks in adv.

解决方案

Drochom wrote:

Hello, I have a huge problem with loading very simple structure into memory it is a list of tuples, it has 6MB and consists of 100000 elements

import cPickle

plik = open("mealy","r")mealy = cPickle.load(plik)plik.close()

this takes about 30 seconds! How can I accelerate it? Thanks in adv.

What protocol did you pickle your data with? The default (protocol 0, ASCII text) is the slowest. I suggest you upgrade to Python 2.3 and save your data with the new protocol 2 -- it''s likely to be fastest. Alex

Hi, I have no idea! I used a similar scheme the other day and made some benchmarks (I *like* benchmarks!) About 6 MB took 4 seconds dumping as well as loading on a 800 Mhz P3 Laptop. When using binary mode it went down to about 1.5 seconds (And space to 2 MB) THis is o.k., because I generally have problems beeing faster than 1 MB/sec with my 2" drive, processor and Python ;-) Python 2.3 seems to have even a more effective "protocoll mode 2". May be your structures are *very* complex?? Kindly Michael P "Drochom" <pe******@gazeta.pl> schrieb im Newsbeitrag news:bh**********@atlantis.news.tpi.pl...

Hello, I have a huge problem with loading very simple structure into memory it is a list of tuples, it has 6MB and consists of 100000 elements

import cPickle

plik = open("mealy","r")mealy = cPickle.load(plik)plik.close()

this takes about 30 seconds! How can I accelerate it? Thanks in adv.

Drochom wrote:

What protocol did you pickle your data with? The default (protocol 0,ASCII text) is the slowest. I suggest you upgrade to Python 2.3 andsave your data with the new protocol 2 -- it''s likely to be fastest. Alex

Thanks:) i''m using default protocol, i''m not sure if i can upgrade so simply, because i''m using many modules for Py2.2

Then use protocol 1 instead -- that has been the binary pickle protocol for a long time, and works perfectly on Python 2.2.x :-) (and it''s much faster than protocol 0 -- the text protocol) --Irmen