如何减少Matlab'find'函数的内存需求？(how to reduce memory demands of Matlab 'find' function?

如何减少Matlab'find'函数的内存需求？(how to reduce memory demands of Matlab 'find' function?)

我在Matlab中有一行代码如下：

output = find(input);

其中列向量“output”包含列向量“input”中元素为非零的所有索引。 例如，如果：

input = [1 3 4 0 0 2 0];

那么结果，output = find（输入）; 将会：

output = 1 2 3 6

对应于非零的数组“输入”的第一（“1”），第二（“3”），第三（“4”）和第6（“2”）索引。

由于我的“输入”数组非常大，这行代码消耗了我所有的本地RAM以及大量的虚拟内存，导致系统慢慢爬行。

任何人都知道减少这种操作的内存需求的好方法（或任何方式）？ 我想把“查找”代码放在一个循环中，但由于数组“输出”的大小（以及这个数组的索引）取决于“查找”操作的结果，我看不出它是如何可能的这样做。 想出来了。

提前感谢任何意见/建议。 -gkk

I have a line of code in Matlab that reads:

output = find(input);

where column vector "output" contains all indices in column vector "input" whose elements are nonzero. For example, if:

input = [1 3 4 0 0 2 0];

then the result of, output = find(input); would be:

output = 1 2 3 6

corresponding to the 1st ("1"), 2nd ("3"), 3rd ("4"), and 6th ("2") indices of array "input" that are nonzero.

Since my "input" array is very large, this line of code consumes all of my local RAM plus a huge portion of virtual memory, causing the system to slow to a crawl.

Anyone know of a good way (or any way) to reduce the memory requirements of such an operation? I thought about putting the "find" code in a loop, but since the size of array "output" (and thus indexing of this array) depends on the result of the "find" operation, I don't see how it's possible to do so. Ran out of ideas.

Thanks in advance for any comments/suggestions. -gkk

最满意答案

如果您有足够的RAM来容纳相同大小的input数组，则可以替换要find的调用

output = 1:length(input); output = output(input~=0);

如果input少于2 ^ 32-1个元素，则可以将其初始化为uint32 ，从而进一步节省内存。

更好的方法可能是将input数组转换为稀疏 ，如果input包含大量零，则可以节省内存，然后使用find ，即

input = sparse(input); output = find(input);

---

编辑

要执行find操作，我将执行以下操作：

nIn = length(input); blockLength = 100000; nBlocks = ceil(nIn/blockLength); %# work in chunks of 100k entries out = cell(nBlocks,1); for i=1:nBlocks out{i} = (i-1)*blockLength+1:i*blockLength; %# assign as your favorite integer format here out{i} = out{i}(input(out{i})~=0); end out = cat(1,out{:});

If you have enough RAM to hold an array the same size of input, you can replace the call to find by

output = 1:length(input); output = output(input~=0);

If input has less than 2^32-1 elements, you can initialize it as uint32, and thus further save on memory.

A better way might be to convert your input array to sparse, which saves memory if input contains lots of zeros, and then use find on that, i.e.

input = sparse(input); output = find(input);

---

EDIT

To perform the find operation in pieces, I'd do the following:

nIn = length(input); blockLength = 100000; nBlocks = ceil(nIn/blockLength); %# work in chunks of 100k entries out = cell(nBlocks,1); for i=1:nBlocks out{i} = (i-1)*blockLength+1:i*blockLength; %# assign as your favorite integer format here out{i} = out{i}(input(out{i})~=0); end out = cat(1,out{:});