Fast file copy?

[dustintinsley]

Ok, I am new to VB.NET. Have been using VB6 for some time now. I had created some backup software in VB6 that used a faster file copy function then the standard VB file copy. I need help trying to convert this to VB.NET using the .NET file functions. I tthink it could be done using FileStrem and BinaryReader, and ReadChars(), but I just can't figure out how to do it. Any help is appreciated. Here's the VB6 code (sorry for the no comments, I didnt use to comment code):

Public Sub CopyFile(fSource As String, fDest As String, BufferSize As Integer)
    'buffer size is in MB
    Dim fLen As Long, fFileSource As Integer, fFileDest As Integer, PackagSize As Long
    Dim LastData As Boolean, sTemp As String

    fLen = FileLen(fSource)
    fFileSource = FreeFile
    fFileDest = fFileSource + 1
    
    PackageSize = BufferSize * 1048576
    
    LastData = False
    
    
    frmLog.pBarFile.Max = fLen
    frmLog.pBarFile.Position = 0
    
    Open fSource For Binary Access Read As fFileSource
    Open fDest For Binary Access Write As fFileDest
    sTemp = ""
    
    Do Until EOF(fFileSource)
        If fLen - Loc(fFileSource) <= PackageSize Then
            PackageSize = fLen - Loc(fFileSource) + 1
            LastData = True
        End If
        
        sTemp = Space$(PackageSize)
        
        Get #fFileSource, , sTemp
        If LastData = True Then
            sTemp = Mid(sTemp, 1, Len(sTemp) - 1)
        End If
        
        frmLog.pBarFile.Position = frmLog.pBarFile.Position + Len(sTemp)
        
        Put #fFileDest, , sTemp
        DoEvents
    Loop
    Close #fFileSource, #fFileDest
        
        
    
End Sub

 

[JohnH]

My.Computer.FileSystem.CopyFile(sourcefile, destinationfile) 'or other method overload
'or
IO.File.Copy(...)

 

[jmcilhinney]

If you don't want to use the in-built methods that JohnH recommends then you would use two FileStreams, alternately calling Read on the source and Write on the destination.

Using source As New IO.FileStream("source path here", IO.FileMode.Open)
    Using target As New IO.FileStream("destination path here", IO.FileMode.Create)
        Const BUFFER_SIZE As Integer = 1024
        Dim buffer(BUFFER_SIZE - 1) As Byte
        Dim bytesRead As Integer = source.Read(buffer, 0, BUFFER_SIZE)

        While bytesRead > 0
            target.Write(buffer, 0, bytesRead)
            bytesRead = source.Read(buffer, 0, BUFFER_SIZE)
        End While

        target.Close()
    End Using

    source.Close()
End Using

You could experiment with the buffer size to see where the optimum between space and speed lies. You may find that it'snot significantly faster than the in-built methods though, but only testing would tell you that. Note that the methods of My.Computer.FileSystem also have the added advantage of having in-built progress dialogues like Windows Explorer, which may be an advantage in some circumstances.

 

[dustintinsley]

Thanks guys. Here is the code I have came up with:

    Public Sub BackupFile(ByVal OriginalFile As String, ByVal NewFile As String)
        'check to see if the original file exists; if not, do nothing
        If IO.File.Exists(OriginalFile) Then
            'create filestreams for input and output files
            Dim FileIn As New IO.FileStream(OriginalFile, IO.FileMode.Open, IO.FileAccess.Read)
            Dim FileOut As New IO.FileStream(NewFile, IO.FileMode.Create, IO.FileAccess.Write)

            'create buffer size (in mb(s))
            Dim BufferSize As Integer = (1024 * 1024) * 5
            Dim b(BufferSize - 1) As Byte

            'copy the buffered amount until the buffer size is to large
            Do Until FileIn.Position + BufferSize >= FileIn.Length
                'reads bytes (size of buffersize) into the byte array b
                FileIn.Read(b, 0, BufferSize)

                'writes bytes (size of buffersize) to the new file
                FileOut.Write(b, 0, BufferSize)

                'amount of file copied (to show progress, if desired; comment out line below)
                'Val(FileIn.Position * 100 \ filein.Length)
            Loop

            'copy the remaining bytes at the end of file that were to small for the buffersize
            If FileIn.Length - FileIn.Position > 0 Then
                'get length of bytes remaining
                Dim RemainingBytes As Integer = FileIn.Length - FileIn.Position
                ReDim b(RemainingBytes - 1)

                'reads remaining bytes into the byte array b
                FileIn.Read(b, 0, RemainingBytes)

                'writes the remaining bytes to the new file
                FileOut.Write(b, 0, RemainingBytes)

                'final progress of the file copied(to show progress, if desired; comment out line below)
                'Val(FileIn.Position * 100 \ FileIn.Length)
            End If

            'close the input file
            FileIn.Close()
            FileIn.Dispose()

            'close the output file
            FileOut.Close()
            FileOut.Dispose()
        End If
    End Sub

 

[jmcilhinney]

Excuse me for saying so but isn't mine somewhat cleaner?

 

[dustintinsley]

Looking it over, it is. I was allowing to not over read bytes, but I see with your code that was just extra coding that in the end was unuseful. The performance is about the same. I tested a 250mb file and both methods took around 7-8 seconds. Your code is just cleaner. Thank you! Would it be possible to add GZip compression to this? Like I said, I am new to .NET and am trying to figure out a few things. Would like to see how to add the built in GZip compression to this, if possible.

 

[jmcilhinney]

Compression's a piece of cake. It's almost exactly the same as before but you pass the target stream to a GZipStream constructor and then Write to the GZipStream. It will compress the data it receives and then write to the underlying FileStream.

Using source As New IO.FileStream("source path here", IO.FileMode.Open)
    Using target As New IO.FileStream("destination path here", IO.FileMode.Create)
        Using compressor As New IO.Compression.GZipStream(target, IO.Compression.CompressionMode.Compress)
            Const BUFFER_SIZE As Integer = 1024
            Dim buffer(BUFFER_SIZE - 1) As Byte
            Dim bytesRead As Integer = source.Read(buffer, 0, BUFFER_SIZE)

            While bytesRead > 0
                compressor.Write(buffer, 0, bytesRead)
                bytesRead = source.Read(buffer, 0, BUFFER_SIZE)
            End While

            compressor.Close()
        End Using
    End Using

    source.Close()
End Using

 

[dustintinsley]

Ok, I must be something wrong. The file size increased. Here is what I did:

    Public Sub BackupFile2(ByVal OriginalFile As String, ByVal NewFile As String)
        'create filestream for source file
        Using SourceFile As New IO.FileStream(OriginalFile, IO.FileMode.Open)
            'create filestream for target file
            Using TargetFile As New IO.FileStream(NewFile, IO.FileMode.Create)
                Using Compressor As New IO.Compression.GZipStream(TargetFile, IO.Compression.CompressionMode.Compress)
                    'specify the buffer size
                    Const BUFFER_SIZE As Integer = (1024 * 1024) * 8
                    Dim Buffer(BUFFER_SIZE - 1) As Byte
                    'read bytes in from source file
                    Dim BytesRead As Integer = SourceFile.Read(Buffer, 0, BUFFER_SIZE)

                    'write bytes out to source file and read next set of bytes
                    While BytesRead > 0
                        Compressor.Write(Buffer, 0, BytesRead)
                        'percentage of file copied (to show progress, comment out line below)
                        ProgressBar1.Value = Val(SourceFile.Position * 100 \ SourceFile.Length)
                        BytesRead = SourceFile.Read(Buffer, 0, BUFFER_SIZE)
                    End While
                    'close target file when complete
                    Compressor.Close()
                    Compressor.Dispose()
                End Using
                'close source file when complete
                SourceFile.Close()
                SourceFile.Dispose()
            End Using
        End Using
    End Sub

 

[JohnH]

Your 250mb test file wouldn't happen to be a video file? They usually won't compress any smaller. Compare with Winzip or something.

Also, you didn't see that Using block already dispose of the resource it's using. Or more precise, resource is disposed if it is an unmanaged object, but just goes out of scope when 'End Using' and is handled as normal by Garbage Collector (GC) if resource is a managed object.

 

[dustintinsley]

JohnH: since the Using block disposes of it's resource, I don't need the SourceFile.Dipose()?

 

[jmcilhinney]

The sole purpose of a Using block is to Dispose of the object created on the opening line. This:

Using obj As New DisposableType
    '...
End Using

is equivalent to this:

Dim obj As New DisposableType

Try
    '...
Finally
    obj.Dispose()
End Try

Did you think I just omitted the Dispose calls? Shame!

As JohnH says, a file that is already in a compressed format is unlikely to shrink with application of another compression algorithm. It may well grow, as you saw, because of the addition of the header for the new format.