VB文件阅读器中的内存泄漏(Memory leak in VB file reader)

我目前正在为VB中的固定宽度表编写一个文件读取器，编译好的应用程序似乎正在吸取记忆，就像没有明天一样。我正在处理一系列〜50兆字节的文件，但是经过几次后，这个过程开始占用大约200+兆字节的RAM，这远远超过了应有的程度。

我已经做了一些尝试，我认为这个问题是对NewRow（）的调用，但不要听我的话。

有没有人有一些优化这个技巧？如果问题出现在NewRow（）调用中，是否有解决此问题的方法？

代码如下：

Function LoadFixedWidthFileToDataTable(ByVal filepath As String, ByRef Colnames() As String, ByRef colwidth() As Integer) As DataTable Dim filetable As New DataTable For Each name As String In Colnames filetable.Columns.Add(name) Next Dim loadedfile As StreamReader Try loadedfile = New StreamReader(filepath) Catch io As IOException MsgBox(io.Message) Return Nothing Exit Function End Try Dim line As String = loadedfile.ReadLine Dim filerow As DataRow = filetable.NewRow Dim i As Integer = 0 While Not loadedfile.EndOfStream line = loadedfile.ReadLine filerow = filetable.NewRow i = 0 For Each colsize As Integer In colwidth Try filerow(i) = line.Substring(0, colsize) line = line.Remove(0, colsize) Catch ex As ArgumentOutOfRangeException ''If the line doesn't match array params Exit For End Try i = i + 1 Next filetable.Rows.Add(filerow) End While loadedfile.Close() Return filetable End Function

I'm currently coding a file reader for fixed-width tables in VB, and the compiled application seems to be sucking down memory like there's no tomorrow. I'm working with a series of ~50 megabyte files, but after running through several, the process starts taking up about 200+ megabytes of RAM, which is way more than it should.

I've done some poking around, and I think the issue is the call to NewRow(), but don't take my word for it.

Does anyone have some tips for optimizing this? If the problem's with the NewRow() call, is there a way of clearing this out?

Code follows below:

Function LoadFixedWidthFileToDataTable(ByVal filepath As String, ByRef Colnames() As String, ByRef colwidth() As Integer) As DataTable Dim filetable As New DataTable For Each name As String In Colnames filetable.Columns.Add(name) Next Dim loadedfile As StreamReader Try loadedfile = New StreamReader(filepath) Catch io As IOException MsgBox(io.Message) Return Nothing Exit Function End Try Dim line As String = loadedfile.ReadLine Dim filerow As DataRow = filetable.NewRow Dim i As Integer = 0 While Not loadedfile.EndOfStream line = loadedfile.ReadLine filerow = filetable.NewRow i = 0 For Each colsize As Integer In colwidth Try filerow(i) = line.Substring(0, colsize) line = line.Remove(0, colsize) Catch ex As ArgumentOutOfRangeException ''If the line doesn't match array params Exit For End Try i = i + 1 Next filetable.Rows.Add(filerow) End While loadedfile.Close() Return filetable End Function

最满意答案

Mikurski，

我立即发现一个问题。您正在昏迷Stream阅读器。你应该把它放在一个使用块中，或者确保你在每个代码路径的末尾都做一个.Dispose 。任何实现IDisposable接口的应该在使用块中处理或使用，如下所示：

Using fs As New FileStream("C:\testing.txt", FileMode.Open) Using sr As StreamReader = New StreamReader(fs) Dim message As String = sr.ReadLine() MessageBox.Show(message) End Using End Using

我希望这有帮助，

谢谢！

编辑：

以下是实现IDisposable的项目，代码不会处理：

数据表 StreamReader的

希望这可以帮助，

再次感谢！

这是一个内存密集程度较低的功能：

Function LoadFixedWidthFileToDataTable(ByVal filepath As String, ByRef Colnames() As String, ByRef colwidth() As Integer) As DataTable Dim filetable As New DataTable Try For Each name As String In Colnames filetable.Columns.Add(name) Next Try Using loadedfile As StreamReader = New StreamReader(filepath) Dim line As String = loadedfile.ReadLine Dim filerow As DataRow = filetable.NewRow Dim i As Integer = 0 While Not loadedfile.EndOfStream line = loadedfile.ReadLine filerow = filetable.NewRow i = 0 For Each colsize As Integer In colwidth Try filerow(i) = line.Substring(0, colsize) line = line.Remove(0, colsize) Catch ex As ArgumentOutOfRangeException Exit For End Try i = i + 1 Next filetable.Rows.Add(filerow) End While loadedfile.Close() End Using Catch io As IOException MsgBox(io.Message) Return Nothing End Try Return filetable Catch ex As Exception filetable.Dispose() End Try Return Nothing End Function

我注意到你正在返回你应该处置的东西。如果你这样做，你不能在这里处理它。相反，这个函数被调用的地方必须处理它。如果你围绕这个使用块进行包装，你将会遇到问题，因为在你有机会从调用代码中对其进行处理之前，将返回的对象将被丢弃。

希望这可以帮助，

再次感谢！

Mikurski,

I see one problem right off the bat. You are Dim'ing the Stream reader. You should enclose this in a using block, or be sure you do a .Dispose at the end of every code path. Anything that implements IDisposable interface should be disposed or used in a using block as follows:

Using fs As New FileStream("C:\testing.txt", FileMode.Open) Using sr As StreamReader = New StreamReader(fs) Dim message As String = sr.ReadLine() MessageBox.Show(message) End Using End Using

I hope this helps,

Thanks!

EDIT:

Here are the items that implement IDisposable that your code isn't disposing:

DataTable StreamReader

Hope this helps,

Thanks again!

Here is a function that is less memory intensive:

Function LoadFixedWidthFileToDataTable(ByVal filepath As String, ByRef Colnames() As String, ByRef colwidth() As Integer) As DataTable Dim filetable As New DataTable Try For Each name As String In Colnames filetable.Columns.Add(name) Next Try Using loadedfile As StreamReader = New StreamReader(filepath) Dim line As String = loadedfile.ReadLine Dim filerow As DataRow = filetable.NewRow Dim i As Integer = 0 While Not loadedfile.EndOfStream line = loadedfile.ReadLine filerow = filetable.NewRow i = 0 For Each colsize As Integer In colwidth Try filerow(i) = line.Substring(0, colsize) line = line.Remove(0, colsize) Catch ex As ArgumentOutOfRangeException Exit For End Try i = i + 1 Next filetable.Rows.Add(filerow) End While loadedfile.Close() End Using Catch io As IOException MsgBox(io.Message) Return Nothing End Try Return filetable Catch ex As Exception filetable.Dispose() End Try Return Nothing End Function

I notice that you're returning the thing that you should be disposing. If you're doing this, you cannot dispose of it here. Instead, the place this function is called from must dispose of it. If you wrap a using block around this you will have issues as the returned object will be disposed before you have a chance to act upon it from the calling code.

Hope this helps,