Hi there.
I need to write an application that will copy files from one NAS to another. This isn't a sync, more of an additive copy, if file dates are different, the file sizes are different or the A attribute is set the file will be copied. Of course a file will also be copied if the destination doesn't exist.
However, there are two things I need to avoid, one relates to the other.
The first issue is a lot of these files are large and I don't want data being written to the system disk. Generally speaking in this scenario the file being read is written to the system drive in chunks while they wait to be shipped out to the destination. But this is going to be a high volume copy process and I don't want to destroy the system SSD's erase block count. I need to be able to avoid this at all costs.
I had two ideas. Either somehow I tell Windows to use a second hard drive as the temp storage for this application, but I don't know if that's possible, I don't want to tell Windows to use a mechanical drive for virtual memory as I don't want to slow this machine down more than I have to. Or I read the files in small chunks myself, holding them in memory then writing the chunks to the destination one by one. I've done this before with large file copies, and while it's a bit slower I had it working really well, it also allows you to show meaningful progress. However, I'm scared silly of file corruption, if anything interrupts the writing process you end up with a totalled file, then your backup is void.
I just wonder if anyone might have another idea how to achieve this. But before anyone gives me stats on how long an SSD should last, I should explain I totalled two last year using software what continually used the system drive for small temp files.
Thanks folks.
I need to write an application that will copy files from one NAS to another. This isn't a sync, more of an additive copy, if file dates are different, the file sizes are different or the A attribute is set the file will be copied. Of course a file will also be copied if the destination doesn't exist.
However, there are two things I need to avoid, one relates to the other.
The first issue is a lot of these files are large and I don't want data being written to the system disk. Generally speaking in this scenario the file being read is written to the system drive in chunks while they wait to be shipped out to the destination. But this is going to be a high volume copy process and I don't want to destroy the system SSD's erase block count. I need to be able to avoid this at all costs.
I had two ideas. Either somehow I tell Windows to use a second hard drive as the temp storage for this application, but I don't know if that's possible, I don't want to tell Windows to use a mechanical drive for virtual memory as I don't want to slow this machine down more than I have to. Or I read the files in small chunks myself, holding them in memory then writing the chunks to the destination one by one. I've done this before with large file copies, and while it's a bit slower I had it working really well, it also allows you to show meaningful progress. However, I'm scared silly of file corruption, if anything interrupts the writing process you end up with a totalled file, then your backup is void.
I just wonder if anyone might have another idea how to achieve this. But before anyone gives me stats on how long an SSD should last, I should explain I totalled two last year using software what continually used the system drive for small temp files.
Thanks folks.
