Detecting changes in files

[tyremt]

Hi all, been a VB fan for sometime now but had little time to get really involved.

Does anyone know what would be the most feasible method to achieve the following...

I have a text file, about 30mb's in size which gets modified regularly (lines are appended - logfile), but what I would like to do is only read the contents that have been added since the file was last modified.

Is this possible?

Cheers all

 

[rcombs4]

My initial thought would be to create a DB Table or just a simple CSV file that your app has access to. Then just store the file location and the line number you last read.

My New Log File Contents

c:\some folder\another folder\some_file.txt, 1281
c:\some folder\another folder\another_file.txt, 1495

Then when you access some_file.txt and another_file.txt all you have to do is read from line numbers 2181 & 1495 on till the new end of the file.

 

[tyremt]

hi mate,

ya thats what i thought aswell, problem being I need this process to run frequently as in every 10 mins and the file size is circa 30mb's, so i'd be reading into memory a 30mb file which is roughly 800,000 lines, to loop through and catch new lines from saved line num

now... that wouldnt be so bad if there was a way to loop through a file without reading it all into memory, i gather the StreamReader would do this at a guess (or I could be totally wrong?) only problem with that is I tried using StreamReader to parse lines on the file in question & lets just say after I watched the console loop through only a few thousand lines in about 3 mins I quickly gave up... had no idea StreamReader was so slow by line?

 

[rcombs4]

Hmmm I wouldn't think it would be that slow. I've used ReadAllLines to read in files about 14MB. It reads the entire file in like a second or so.

Now i'm second guessing if i'm confusing two projects, but I'm pretty sure we can read in a 14MB file in like a second.

 

[tyremt]

ya you will be able too, i think i timed the readalllines at about 5 secs for 32mb's, shouldbe faster, but maybe its my local env thats cluttering the request up

soo, the StreamReader line by line, does this read not read the file into memory? Or does it?

i remember a good while ago in VB6 i had a class somewhere which allowed me to count the lines in files quickly and specifically pull out specific line numbers without looping through the whole file, is there any thing in .NET which achieves something similar?

im just curious