将大数据写入管道时进程挂起

本文介绍了将大数据写入管道时进程挂起的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我的 Perl 程序遇到挂起进程的问题，我想我已经将其隔离到每当我将大量数据写入管道时.

I have a problem with hung processes with my Perl program, and I think I have isolated it to whenever I write significant amounts of data to a pipe.

以下是我认为与我的程序相关的所有代码.程序挂起时，挂在ResponseConstructor.pm中的一行:print { $self->{writer} } $data;.

Below is all of the code that I think is relevant from my program. When the program hangs, it hangs on the line in ResponseConstructor.pm: print { $self->{writer} } $data;.

我已经用不同的数据大小进行了测试，它似乎没有以确切的大小挂起.不过，随着大小的增加，它可能变得更有可能:大约 32KB 的大小有时有效，有时则无效.每次我尝试 110KB 字符串时，它都失败了.

I've tested with different data sizes, and it doesn't appear to hang at an exact size. It may become more likely with size, though: sizes around 32KB sometimes work, sometimes don't. Every time I've tried a 110KB string it has failed.

我相信我也排除了数据的内容作为原因，因为相同的数据有时会导致挂起，有时不会.

I believe I've also ruled out the contents of the data as a cause, because the same data sometimes causes a hang, and othertimes doesn't.

这可能是我第一次在程序中使用管道，所以我不确定接下来要尝试什么.有什么想法吗?

This is probably the first time I have used pipes in a program before, so I'm not sure what to try next. Any ideas?

use POSIX ":sys_wait_h"; STDOUT->autoflush(1); pipe(my $pipe_reader, my $pipe_writer); $pipe_writer->autoflush(1); my $pid = fork; if ($pid) { #I am the parent close $pipe_writer; while (waitpid(-1, WNOHANG) <= 0){ #do some stuff while waiting for child to send data on pipe } #process the data it got open(my $fh, '>', "myoutfile.txt"); while ( my $line = <$pipe_reader>){ print $fh $line; } close $pipe_reader; close $fh; else { #I am the child die "cannot fork: $!" unless defined $pid; close $pipe_reader; my $response = ResponseConstructor->new($pipe_writer); if ([a condition where we want to return small data]){ $response->respond('small data'); exit; } elsif ([a condition where we want to return big data]){ $response->respond('imagine this is a really big string'); } }

ResponseConstructor.pm:

ResponseConstructor.pm:

package ResponseConstructor; use strict; use warnings; sub new { my $class = shift; my $writer = shift; my $self = { writer => $writer }; bless($self, $class); return($self); } #Writes the response then closes the writer (pipe) sub respond { my $self = shift; my $data = shift; print { $self->{writer} } $data; close $self->{writer}; } 1;

推荐答案

您可能不应该在管道返回数据时忽略它:您可以在管道上使用 select(而不是waitpid) 以查看在等待循环期间是否有任何数据要读取，但是如果您真的想要更大的管道缓冲区以便可以一次读取所有数据，则可以使用 socketpair 而不是管道，然后您可以使用 setsockopt 使缓冲区尽可能大.

You probably shouldn't be ignoring your pipe while it is returning data: you can use a select on the pipe (instead of waitpid) to see if there's any data to read during your waiting loop, but if you really want a larger pipe buffer so you can read it all at once, you can use a socketpair instead of a pipe and then you can use setsockopt to make the buffer as large as you need it.