Optimize Code

tg93

Member
Joined
Jul 26, 2014
Messages
7
Programming Experience
Beginner
Hello, I have written a code which opens, edits, and saves text files by taking a start date and an end date from the user (the text files are named and located according to their date). The code is functional (I've tried it on small spans and small files) but the actual text files I want to implement this on are about 24,500 lines long (and there is one for every day of the year from 1999 till the present). I am looking to optimize the code as much as possible, I have heard that not updating the textbox for every line read would make things better but I do not know how to implement it other than this way. Any suggestion would be helpful, please keep in mind I am a beginner at VB.

VB.NET:
Imports System.IO
Public Class Form1


    Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
        Dim watch As Stopwatch = Stopwatch.StartNew()   'Starts stopwatch
        Dim Reader As System.IO.StreamReader        'Declares the Reader
        Dim Writer As System.IO.StreamWriter        'Declares the Writer
        Dim i As Date                               'Loop counter (by date)
        Dim j As Integer                            'Loop counter
        Dim k As Integer                            'Files notFound counter
        Dim StartDate As Date                       'Date to begin compiling text files
        Dim EndDate As Date                         'Date to stop compiling text files
        Dim SaveTo As String                        'Name of the file that will be saved
        Dim Yearv As String                         'Year value
        Dim Monthv1 As Integer                      'Month value
        Dim Monthv As String                        'Month value
        Dim Dayv1 As Integer                        'Day value
        Dim Dayv As String                          'Day value
        Dim DayofWeek As String                     'Weekday value
        Dim YearMonthv As String                    'Year and Month value
        Dim CurrentDate As String                   'Date of the file being worked on
        Dim line As String                          'Holds individual lines from the Reader
        Dim found As Boolean                        'Marks instances when a file does not exist
        Dim array() As String = {"All files present."}   'Holds text to be printed


        Try
            StartDate = "#" & txtStartDate.Text & "#"   'User enters start & end date in textboxes
            EndDate = "#" & txtEndDate.Text & "#"
        Catch                                           'Catches invalid dates and exits with message
            MsgBox("Date(s) not valid.")
            Exit Sub
        End Try


        If StartDate > EndDate Then                     'Catches unordered dates and exits with message
            MsgBox("Start date cannot be after end date.")
            Exit Sub
        End If


        i = StartDate                               'Hold StartDate for counting
        j = 0                                       'Intializes loop counter
        k = 0                                       'Intialize number of times a file hasn't been found
        found = True                                'File is assumed to exist unless not found
        TextBox1.Clear()                            'Clears textboxes
        notFound.Clear()


        While (i <= EndDate)                        'Loop runs from start date to end date


            j = j + 1                               'Count loops
            Yearv = Str(i.Year)                     'Assigning Year, Month, Day, and Day of Week to different elements
            Monthv1 = i.Month
            Dayv1 = i.Day
            DayofWeek = WeekdayName(Weekday(i))


            If Monthv1 < 10 Then                                'Converts 1 to 01 for Month no. (due to naming convention of the files)
                Monthv = "0" + Trim(Str(Monthv1))
                YearMonthv = Yearv + Monthv                     'Corrects YearMonthv accordingly
            Else
                Monthv = Trim(Str(Monthv1))
                YearMonthv = Trim(Trim(Yearv) + Trim(Monthv))
            End If


            If Dayv1 < 10 Then                                      'Converts 1 to 01 for Day no. (due to naming convention of the files)
                Dayv = "0" + Trim(Str(Dayv1))
            Else
                Dayv = Str(Dayv1)
            End If


            CurrentDate = Trim(Yearv) + Trim(Monthv) + Trim(Dayv)   'Sets current date as string (due to naming convention of the                                                                                              'files)


            SaveTo = "C:\Documents and Settings\Walid\Desktop\September\" _
                & Trim(YearMonthv) & "\a" & Trim(CurrentDate) & ".txt"                          'Name of file to be saved


            Try                                                                                 'Checking if file to be opened exists
                Reader = File.OpenText("C:\Documents and Settings\Walid\Desktop\September\" _
                            & Trim(YearMonthv) & "\c" & Trim(CurrentDate) & ".txt")             'Name of file to be opened
            Catch
                ReDim Preserve array(k)
                array(k) = SaveTo
                k = k + 1                                                                       'Counts number of files not found
                found = False                                                                   'Current date has no associated file
            End Try


            If found = True Then                                        'Reads only if file exists
                Do Until Reader.EndOfStream                             'Reads through the open file line by line to last line
                    line = Reader.ReadLine                              'Holds individual line for the iteration


                    If line.Contains("VOL") Then                        'Looks for the header
                        line = "Time,DET#,VOL,OCC,Date of:,Day,Type,"   'Modifies the header
                    Else
                        line = line + Trim(Yearv) + " " + Trim(Monthv) + " " _
                            + Trim(Dayv) + "," + Trim(DayofWeek) + ",c"             'Adds info to all other lines
                    End If


                    TextBox1.Text = TextBox1.Text & line & vbCrLf                   'Assigns the current line to the textbox
                Loop                                                                'Reiterate
                Reader.Close()                                                      'Closes the reader
            End If


            If found = True Then                        'Only saves files whose read files exists
                Writer = My.Computer.FileSystem. _
                OpenTextFileWriter(SaveTo, True)        'Opens text document
                Writer.WriteLine(TextBox1.Text)         'Writes the text in the text box
                Writer.Close()                          'Closes the Writer
            End If
            TextBox1.Clear()                            'Clears textbox
            found = True                                'Reintialized as true


            i = i.AddDays(1)                            'Increments date by one day


        End While


        watch.Stop()                                    'Stops stopwatch


        TextBox1.Text = "Program Complete. A total of " & (j - k) & _
            " text files compiled." & vbCrLf & k & _
            " files were not found or written." & vbCrLf & _
            "Program took " & Str(watch.Elapsed.TotalSeconds) & _
            " second to run."                                           'Output text


        For Each item In array
            notFound.Text += item & vbCrLf
        Next item


    End Sub
 

tg93

Member
Joined
Jul 26, 2014
Messages
7
Programming Experience
Beginner
I have managed to greatly improve performance by using StringBuilder. I would still like to hear any suggestions you may have to further improve it.

VB.NET:
Imports System.IO
Imports System.Text
Public Class Form1


    Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
        Dim watch As Stopwatch = Stopwatch.StartNew()   'Starts stopwatch
        Dim Reader As System.IO.StreamReader            'Declares the Reader
        Dim Writer As System.IO.StreamWriter            'Declares the Writer
        Dim i As Date                                   'Loop counter (by date)
        Dim j As Integer                                'Loop counter
        Dim k As Integer                                'Files notFound counter
        Dim StartDate As Date                           'Date to begin compiling text files
        Dim EndDate As Date                             'Date to stop compiling text files
        Dim SaveTo As String                            'Name of the file that will be saved
        Dim Yearv As String                             'Year value
        Dim Monthv1 As Integer                          'Month value (integer)
        Dim Monthv As String                            'Month value (string)
        Dim Dayv1 As Integer                            'Day value (integer)
        Dim Dayv As String                              'Day value (string)
        Dim DayofWeek As String                         'Weekday value
        Dim YearMonthv As String                        'Year and Month value (string)
        Dim CurrentDate As String                       'Date of the file being worked on (string)
        Dim line As String                              'Holds individual lines from the Reader
        Dim found As Boolean                            'Marks instances when a file does not exist
        Dim arrayf As New StringBuilder                 'Holds text to be printed in Files Not Found
        Dim arrayt As New StringBuilder                 'Holds text from text files


        TextBox1.Clear()                                'Clears textboxes
        notFound.Clear()


        Try
            StartDate = "#" & txtStartDate.Text & "#"   'User enters start & end date in textboxes
            EndDate = "#" & txtEndDate.Text & "#"
        Catch                                           'Catches invalid dates and exits with message
            MsgBox("Date(s) not valid.")
            Exit Sub
        End Try


        If StartDate > EndDate Then                     'Catches unordered dates and exits with message
            MsgBox("Start date cannot be after end date.")
            Exit Sub
        End If


        i = StartDate                               'Hold StartDate for counting
        j = 0                                       'Intializes loop counter
        k = 0                                       'Intialize number of times a file hasn't been found
        found = True                                'File is assumed to exist unless not found


        While (i <= EndDate)                        'Loop runs from start date to end date


            j = j + 1                               'Count loops
            Yearv = Str(i.Year)                     'Assigning Year, Month, Day, and Day of Week to different elements
            Monthv1 = i.Month
            Dayv1 = i.Day
            DayofWeek = WeekdayName(Weekday(i))


            If Monthv1 < 10 Then                                'Converts 1 to 01 for Month no. (due to naming convention of the files)
                Monthv = "0" + Trim(Str(Monthv1))
                YearMonthv = Yearv + Monthv                     'Corrects YearMonthv accordingly
            Else
                Monthv = Trim(Str(Monthv1))
                YearMonthv = Trim(Trim(Yearv) + Trim(Monthv))
            End If


            If Dayv1 < 10 Then                                      'Converts 1 to 01 for Day no. (due to naming convention of the files)
                Dayv = "0" + Trim(Str(Dayv1))
            Else
                Dayv = Str(Dayv1)
            End If


            CurrentDate = Trim(Yearv) + Trim(Monthv) + Trim(Dayv)   'Sets current date as string (due to naming convention of the files)


            SaveTo = "C:\Documents and Settings\Walid\Desktop\September\" _
                & Trim(YearMonthv) & "\a" & Trim(CurrentDate) & ".txt"                          'Name of file to be saved


            Try                                                                                 'Checking if file to be opened exists
                Reader = File.OpenText("C:\Documents and Settings\Walid\Desktop\September\" _
                            & Trim(YearMonthv) & "\c" & Trim(CurrentDate) & ".txt")             'Name of file to be opened
            Catch
                arrayf.AppendLine(SaveTo)                               'Saves the name of the file not found
                k = k + 1                                               'Counts number of files not found
                found = False                                           'Current date has no associated file
            End Try


            If found = True Then                                        'Reads only if file exists
                Do Until Reader.EndOfStream                             'Reads through the open file line by line to last line
                    line = Reader.ReadLine                              'Holds individual line for the iteration


                    If line.Contains("VOL") Then                        'Looks for the header
                        line = "Time,DET#,VOL,OCC,Date of:,Day,Type,"   'Modifies the header
                    Else
                        line = line + Trim(Yearv) + " " + Trim(Monthv) + " " _
                            + Trim(Dayv) + "," + Trim(DayofWeek) + ",c"             'Adds info to all other lines
                    End If
                    arrayt.AppendLine(line)                                         'Assigns the current line to the textbox
                Loop                                                                'Reiterate
                Reader.Close()                                                      'Closes the reader
            End If


            If found = True Then                        'Only saves files whose read files exists
                Writer = My.Computer.FileSystem. _
                OpenTextFileWriter(SaveTo, True)        'Opens text document
                Writer.WriteLine(arrayt)                'Writes the text saved to arrayt
                Writer.Close()                          'Closes the Writer
            End If
            arrayt.Clear()                              'Clears arrayt
            found = True                                'Reintialized as true


            i = i.AddDays(1)                            'Increments date by one day


        End While


        If k = 0 Then
            arrayf.AppendLine("All files present.")     'If all files are present
        End If
        notFound.Text = arrayf.ToString


        watch.Stop()                                                    'Stops stopwatch


        TextBox1.Text = "Program Complete. A total of " & (j - k) & _
            " text files compiled." & vbCrLf & k & _
            " files were not found or written." & vbCrLf & _
            "Program took " & Str(watch.Elapsed.TotalSeconds) & _
            " seconds to run."                                           'Output text

    End Sub
 

jmcilhinney

VB.NET Forum Moderator
Staff member
Joined
Aug 17, 2004
Messages
14,350
Location
Sydney, Australia
Programming Experience
10+
Firstly, you're date handling is poor. If the user is entering start and end dates then they should be using a DateTimePicker for that, not a TextBox. The control will then handle all the validation and conversion for you, giving you a Date via its Value property. You can then access all the files in the specified date range something like this:
Dim filePath = String.Format("C:\MyFolder\{0:yyyyMM}\{0:yyyyMMdd}.txt", myDateTimePicker.Value)
You should visit MSDN and read up on standard and custom date and time format strings, which can be used when converting Dates to Strings and vice versa.
 

tg93

Member
Joined
Jul 26, 2014
Messages
7
Programming Experience
Beginner
Thank you, I was not aware of the datepicker type. My revised code with some other updates is this:

VB.NET:
'Edits coma delimited files (not raw data) to include type, date, and day for each entry
'Works in Visual Basic .NET


Imports System.IO
Imports System.Text


Public Class Form1


    Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
        Dim watch As Stopwatch = Stopwatch.StartNew()   'Starts stopwatch
        Dim Reader As System.IO.StreamReader            'Declares the Reader
        Dim Writer As System.IO.StreamWriter            'Declares the Writer
        Dim i As Date                                   'Loop counter (by date)
        Dim j As Integer                                'Loop counter
        Dim k As Integer                                'Files notFound counter
        Dim dettype As String                           'Detector type
        Dim z As Integer                                'Type counter
        Dim StartDate As Date                           'Date to begin compiling text files
        Dim EndDate As Date                             'Date to stop compiling text files
        Dim filepath As String                          'Name of the file to open, edit, save
        Dim DayofWeek As String                         'Weekday value
        Dim Line As String                              'Holds individual lines from the Reader
        Dim ArrayF As New StringBuilder                 'Holds text to be printed in Files Not Found
        Dim ArrayT As New StringBuilder                 'Holds text from text files
        Dim GoAhead As Boolean                          'Indicates whether or not to produce a file
        Dim lastline As Boolean                         'To distinguish a second header at the end of the text file


        TextBox1.Clear()                                'Clears textboxes
        notFound.Clear()


        StartDate = myStartDate.Value                   'Gets dates
        EndDate = myEndDate.Value




        If StartDate > EndDate Then                     'Catches unordered dates and exits with message
            MsgBox("Start date cannot be after end date.")
            Exit Sub
        End If


        j = 0                                       'Intializes loop counter
        k = 0                                       'Intialize number of times a file hasn't been found
        z = 0                                       'Intializes large loop


        While z <= 1


            i = StartDate                           'Hold StartDate for counting


            If z = 0 Then                           'Loops detector type
                dettype = "c"
            Else
                dettype = "r"
            End If


            While (i <= EndDate)                    'Loop runs from start date to end date


                j = j + 1                           'Count loops


                DayofWeek = WeekdayName(Weekday(i))


                filepath = String.Format("C:\Documents and Settings\Walid\Desktop\Project\September\{0:yyyyMM}\" & dettype & "{0:yyyyMMdd}.txt", i)


                If File.Exists(filepath) Then                               'Checks if file exists
                    Reader = File.OpenText(filepath)                        'Name of file to be opened
                Else
                    ArrayF.AppendLine(filepath)                             'Saves the name of the file not found
                    k = k + 1                                               'Counts number of files not found
                End If


                If File.Exists(filepath) Then                               'Reads only if file exists
                    lastline = False
                    Do Until Reader.EndOfStream                             'Reads through the open file line by line
                        Line = Reader.ReadLine                              'Holds individual line for the iteration
                        If Line.Contains("VOL") And lastline = False Then   'Looks for the (real) header
                            Line = "Time,DET#,VOL,OCC,Date of:,Day,Type"    'Modifies the header
                        ElseIf Line.Contains("VOL") And lastline = True Then 'Looks for (fake) header
                            Line = "00:00:00,0,0,0,1999/12/31,Null,Null"
                        ElseIf Line.Substring(Line.Length - 1) = "," And Line.Contains("VOL") = False Then
                            Line = Line + Trim(Str(i.Year)) + "/" + Trim(Str(i.Month)) _
                                + "/" + Trim(Str(i.Day)) + "," + Trim(DayofWeek) + "," + dettype
                        Else
                            Line = Line + "," + Trim(Str(i.Year)) + "/" + Trim(Str(i.Month)) _
                                + "/" + Trim(Str(i.Day)) + "," + Trim(DayofWeek) + "," + dettype
                        End If
                        ArrayT.AppendLine(Line)                             'Assigns the current line to the textbox
                        lastline = True
                    Loop                                                    'Reiterate
                    Reader.Close()                                          'Closes the reader


                End If


                If File.Exists(filepath) Then
                    File.Delete(filepath)
                    GoAhead = True
                Else
                    GoAhead = False
                End If


                If GoAhead = True Then                      'Only saves files whose read files exists
                    Writer = My.Computer.FileSystem. _
                    OpenTextFileWriter(filepath, True)      'Opens text document
                    Writer.WriteLine(ArrayT)                'Writes the text saved to arrayt
                    Writer.Close()                          'Closes the Writer
                End If


                ArrayT.Clear()                              'Clears arrayt
                i = i.AddDays(1)                            'Increments date by one day


            End While
            z = z + 1                                       'Next detector type
        End While


        If k = 0 Then
            ArrayF.AppendLine("All files present.")                     'If all files are present
        End If
        notFound.Text = ArrayF.ToString                                 'Report files missing


        watch.Stop()                                                    'Stops stopwatch


        TextBox1.Text = "Program Complete. A total of " & (j - k) & _
                " text files compiled." & vbCrLf & k & _
                " files were not found or written." & vbCrLf & _
                "Program took " & Str(watch.Elapsed.TotalSeconds) & _
                " seconds to run."                                      'Output text


    End Sub
 

Pyth007

Member
Joined
Nov 17, 2008
Messages
18
Programming Experience
5-10
I noticed that you keep having conditional statements which never change after the first... Essentially inside of your While loop, you want to check to see if a file exists and then do all of the other operations with that file; if the file doesn't exist, then you just want to increment your count of non-existant files and add the file's name to the list to be displayed at the end. Instead of making all of these boolean variables and flipping them depending on if the file exists, try just having one if statement at the beginning catching if the file doesn't exit, do your code for non-existant files, and then Continue While to loop to the next file. With that Continue statement, the rest of the body will only be for those files that exist, so you don't even need to encapsulate that code inside the Else block. The only other code you want to occur regardless of whether the file exists is located at the end of your While loop which is to clear arrayt and increase i by 1 day; these lines could occur at the very beginning of the While loop before your If statement to catch non-existant files instead of at the end so that they'd still be run even if the loop Continued after catching a non-existant file. If you were to do this, just set "i=StartDate.AddDays(-1)" before entering into the While loop so that when the loop begins with "i=i.AddDays(1)" it will start at the correct date.

Also, since you're now using a DatePicker as well as defining i as a date, there should be no whitespace around the strings that you create from i. Thus the Trim() is unnecessary to use. In fact you may consider using the string formatting similar to how filepath gets assigned...
 
Top Bottom