I have about 6 applications which are constantly generating data, on average 100 characters of text each second which will go in about 8 different fields of a database. I am still working out the design. Sometimes I will need to add like 50 rows into the database almost at once, sometimes the programs will go for 30 seconds with no updates.
Meanwhile, I would like to create a webpage using ajax which will dynamically update itself to show the newest content as often as possible. From what I have learned it's not possible to trigger post-backs (right term?) via the server, so I am thinking of looping some javascript code that will send a post back every few seconds asking it for any new data.
I would like the new data to be added to the existing table with the existing rows dropped down. I would set limits on how many rows will show at a time, among many other settings.
I also want to create filters so I can show/hide columns and rows in real time without a total page refresh. If the colums with the data already exist, it will simply hide the <td></td> elements, if I want to add the data then it will query the server for the updated data, add the <td></td> tags and fill them with the updated data.
Some filters effect collumns, some will effect the rows.
Does anyone have any tips on what kind of database technologies will be best for this? I would like to keep as much data for as long as possible, but I am sure that the amount of data generated will easily equal many, many megabytes, in a very short amount of time.
The older data would not be needed nearly as often as the old data, so I am wondering if I could compress it once it reaches a certain age (1 week? 2 weeks?), then after say a month, start removing it permently to make room for the new data.
I am writing the application in Visual Basic which is getting the data from the applications with DDE. I am working out the kinks right now, it requires a ton of string parsing. I am pretty much to the point where instead of writing lines of text to a text file to troubleshoot and create my app, I now need to start inputting it into a database so I can start working on the asp.net end of it.
I find working with databases much trickier in asp.net then old asp, I just don't understand the .net database classes very well at all.
Thanks much in advance for any advice or tips.
Meanwhile, I would like to create a webpage using ajax which will dynamically update itself to show the newest content as often as possible. From what I have learned it's not possible to trigger post-backs (right term?) via the server, so I am thinking of looping some javascript code that will send a post back every few seconds asking it for any new data.
I would like the new data to be added to the existing table with the existing rows dropped down. I would set limits on how many rows will show at a time, among many other settings.
I also want to create filters so I can show/hide columns and rows in real time without a total page refresh. If the colums with the data already exist, it will simply hide the <td></td> elements, if I want to add the data then it will query the server for the updated data, add the <td></td> tags and fill them with the updated data.
Some filters effect collumns, some will effect the rows.
Does anyone have any tips on what kind of database technologies will be best for this? I would like to keep as much data for as long as possible, but I am sure that the amount of data generated will easily equal many, many megabytes, in a very short amount of time.
The older data would not be needed nearly as often as the old data, so I am wondering if I could compress it once it reaches a certain age (1 week? 2 weeks?), then after say a month, start removing it permently to make room for the new data.
I am writing the application in Visual Basic which is getting the data from the applications with DDE. I am working out the kinks right now, it requires a ton of string parsing. I am pretty much to the point where instead of writing lines of text to a text file to troubleshoot and create my app, I now need to start inputting it into a database so I can start working on the asp.net end of it.
I find working with databases much trickier in asp.net then old asp, I just don't understand the .net database classes very well at all.
Thanks much in advance for any advice or tips.
