load 1 million files into listbox with file locations

[Mythodikal]

Is there an easier way to do this than this way? I have tried using a backgroundworker and simple recursion, but I am trying to load my listbox with file locations from my main drive.

Sub scanSubfolders(ByVal FolderLocation As String)
Try


Dim di As New IO.DirectoryInfo(FolderLocation)
Dim aryFiles() As IO.FileInfo = di.GetFiles("*.*")
Dim aryDirs() As IO.DirectoryInfo = di.GetDirectories()

For Each fi As IO.FileInfo In aryFiles
Me.ListBox3.Items.Add(fi.FullName)
Next

For Each d As IO.DirectoryInfo In aryDirs
scanSubfolders(d.FullName)
Next

Catch ex As Exception
'Stop 'the catch should be more specific
End Try
End Sub

 

[jmcilhinney]

Don't even try to load that many items into a control in one go. How could the user possibly manage them anyway?

 

[JohnH]

Alternative to recursion is using a stack and a while loop, and an Iterator function (a variation of recursive in this case). Either way it would be beneficial to load the items in a background thread and when finished add items to listbox with a single call to AddRange method. If there are many items they could be kept in memory and only presented in UI as filtered views, for example a search filter or paged.
Here are examples of the different methods, including recursive as a function:

Private Iterator Function ScanSubfolders_Iterator(FolderLocation As String) As IEnumerable(Of String)
    Try
        Dim di As New DirectoryInfo(FolderLocation)

        For Each fi As FileInfo In di.EnumerateFiles("*.*")
            Yield fi.FullName
        Next

        For Each d As DirectoryInfo In di.EnumerateDirectories
            For Each file In ScanSubfolders_Iterator(d.FullName)
                Yield file
            Next
        Next

    Catch ex As Exception
        'Stop 'the catch should be more specific
    End Try
End Function

Private Function ScanSubfolders_Stack(FolderLocation As String) As String()
    Dim results As New List(Of String)
    Dim folders As New Stack(Of String)({FolderLocation})
    Try
        While folders.Count > 0
            Dim di As New DirectoryInfo(folders.Pop)

            For Each fi As FileInfo In di.EnumerateFiles("*.*")
                results.Add(fi.FullName)
            Next

            For Each d As DirectoryInfo In di.EnumerateDirectories
                folders.Push(d.FullName)
            Next
        End While
    Catch ex As Exception
        'Stop 'the catch should be more specific
    End Try
    Return results.ToArray
End Function

Private Function ScanSubfolders_Recursive(FolderLocation As String) As String()
    Dim results As New List(Of String)
    Try
        Dim di As New DirectoryInfo(FolderLocation)

        For Each fi As FileInfo In di.EnumerateFiles("*.*")
            results.Add(fi.FullName)
        Next

        For Each d As DirectoryInfo In di.EnumerateDirectories
            results.AddRange(ScanSubfolders_Recursive(d.FullName))
        Next

    Catch ex As Exception
        'Stop 'the catch should be more specific
    End Try
    Return results.ToArray
End Function

Using a background thread to get result in an Async method:

Dim folder = "C:\sample"
Dim result = Await Task.Run(Function() ScanSubfolders_Iterator(folder).ToArray)

and some quick examples of adding to listbox:

'add all items
ListBoxFiles.Items.AddRange(result)

'add items by search filter
Dim filter = From f In result Where f.Contains("something")
ListBoxFiles.Items.AddRange(filter.ToArray)

'add "paged" 100 items
Dim page = result.Skip(100).Take(100)
ListBoxFiles.Items.AddRange(page.ToArray)

 

[jdelano]

Very nice!