OutOfMemory exception

[JuggaloBrotha]

I have a sub routine that will loop through a List (Of String), what this routine does is checks the item in the list with another item in the same list and if it's not the same item, it then checks the same two spots in another list to see if those values are the same. If they are, the item gets added to a delete list.

The items in the first list is a list of filenames, the second is the file's CRC-32's that were calculated and stored in a different sub.

The problem is that I'm getting an OutOfMemory exception if there's more than 1 thousand file names in the list, I'm not sure how to do this without getting any errors.

Here's the code:

    Private m_FileNames As New List(Of String)
    Private m_FileCrc As New List(Of String)
    Private m_FilesToDelete As New List(Of String)
    Private m_SourceFilesToDelete As New List(Of String)

    Private Sub MatchFiles()
        Try
            For Counter1 As Integer = 0 To m_FileNames.Count - 1
                For Counter2 As Integer = 0 To m_FileNames.Count - 1
                    If m_CancelProcessingBoolean <> True Then
                        processBackgroundWorker.ReportProgress(100I, "Matching: " & Path.GetFileName(m_FileNames(Counter1)) & ", " & Path.GetFileName(m_FileNames(Counter2)))
                        If m_FileNames(Counter1) <> m_FileNames(Counter2) Then
                            If m_FileCrc(Counter1) = m_FileCrc(Counter2) Then
                                If m_FilesToDelete.Contains(m_RootFolderString & m_FileNames(Counter2)) = False AndAlso m_FilesToDelete.Contains(m_RootFolderString & m_FileNames(Counter1)) = False Then 'AndAlso m_SourceFilesToDelete.Contains(m_FileNames(Counter1)) = False AndAlso m_SourceFilesToDelete.Contains(m_FileNames(Counter2)) = False Then
                                    m_FilesToDelete.Add(m_RootFolderString & m_FileNames(Counter2))
                                    m_SourceFilesToDelete.Add(m_RootFolderString & m_FileNames(Counter1))
                                End If
                            End If
                        End If
                    End If
                Next Counter2
            Next Counter1
        Catch ex As Exception
            MessageBox.Show("Error in Stage 2:" & Environment.NewLine & ex.Message, "Duplicate Files " & m_AppVersionString, MessageBoxButtons.OK, MessageBoxIcon.Error)
        End Try
    End Sub

 

[JohnH]

Give this a thought:

Dim dupIndexes As New List(Of Integer)
For i As Integer = 0 To FileCrc.Count - 1
    For ii As Integer = i + 1 To FileCrc.Count - 1
        If i <> ii Then
            If FileCrc(i) = FileCrc(ii) Then
                If Not dupIndexes.Contains(i) Then
                    dupIndexes.Add(i)
                End If
                If Not dupIndexes.Contains(ii) Then
                    dupIndexes.Add(ii)
                End If
            End If
        End If
    Next
Next

 

[JuggaloBrotha]

Give this a thought:

Dim dupIndexes As New List(Of Integer)
For i As Integer = 0 To FileCrc.Count - 1
    For ii As Integer = i + 1 To FileCrc.Count - 1
        If i <> ii Then
            If FileCrc(i) = FileCrc(ii) Then
                If Not dupIndexes.Contains(i) Then
                    dupIndexes.Add(i)
                End If
                If Not dupIndexes.Contains(ii) Then
                    dupIndexes.Add(ii)
                End If
            End If
        End If
    Next
Next

I will give this a go, I hadn't thought of only checking from i + 1 for the inner loop.

 

[JohnH]

If there can be more than two identical CRCs I would use a Dictionary(Of crcString, List(Of indexInteger)).
The filenames array can also be stored to a temp file while processing the CRCs.
Or perhaps store matches in UI, a Treeview with CRCs as root nodes and matching filenames as childs.