I've been trying for about 3 months now to write a multithreaded app to collect data from multiple machines in a machine shop and insert the data into a database. There are 87 machines, 48 of which dump their data into text files on a network drives, and 39 of which that can be polled for data with an API. I've written a class to use the API and handle all the data collection from those machines and a second class to read the text files and parse the contents, 1 line of comma separated values (4kb file size). The data from both sources is identical so each of the two classes inherit a parent class which serves as a container for all the machine properties. I want to create a separate instance of the appropriate class for each machine and have them loop collection and updating of the database at a set interval potentially indefinitely. The main form has a DataGridView bound to the database containing the machine data, and start, stop buttons.
The main problem encountered would be for a network drive or a machine to be offline which causes a dramatic increase in collection time so it's necessary that each machine is updated independent to the rest.
I've tried so many different approaches that it would be impossible to post them all here but in summary, I've tried the following techniques:
- Spawning 87 threads, 1 for each machine (I've since read that this is a poor approach)
- Threadpools
- BackgroundWorkers
- Tasks & Async/Await
However it's very possible that I failed to implement the techniques I tried correctly
Unfortunately, nothing I have tried has offered an acceptable user experience. Despite my best efforts the UI locks up and/or the app goes into a 'Not Responding' state until it eventually crashes.
I'm sure there has to be a way to achieve this as it's not a hugely complicated app.
I'd appreciate any advice, help or guidance anyone can offer as I'm at the end of my tether with this project
If code samples for any of my efforts would be of benefit, please let me know and I'll post them.
Thanks in advance!!
The main problem encountered would be for a network drive or a machine to be offline which causes a dramatic increase in collection time so it's necessary that each machine is updated independent to the rest.
I've tried so many different approaches that it would be impossible to post them all here but in summary, I've tried the following techniques:
- Spawning 87 threads, 1 for each machine (I've since read that this is a poor approach)
- Threadpools
- BackgroundWorkers
- Tasks & Async/Await
However it's very possible that I failed to implement the techniques I tried correctly
Unfortunately, nothing I have tried has offered an acceptable user experience. Despite my best efforts the UI locks up and/or the app goes into a 'Not Responding' state until it eventually crashes.
I'm sure there has to be a way to achieve this as it's not a hugely complicated app.
I'd appreciate any advice, help or guidance anyone can offer as I'm at the end of my tether with this project
If code samples for any of my efforts would be of benefit, please let me know and I'll post them.
Thanks in advance!!