Do collections slow down the app performance?

[danyeungw]

I am using .NET 2003. If an application uses many collections to store data from datasets (looping through the datasets to read data), does it slow down the application performance? If multiple users use the application, does it slow down the SQL server?

Thanks.
DanYeung

 

[jmcilhinney]

The fact that you're using collections will not in itself be an issue. If you have lots of collections then presumably you have lots of data. Handling lots of data will take time, whether you're using collections or not.

Obviously the more clients are accessing SQL Server the more work it has to do, so it may not be able to service all requests as quickly as it would if they were coming from a sole client. That said, SQL Server is a robust piece of software that is designed for heavy multi-user environments. If it's installed on a reasonably fast machine it should be able to handle multiple clients without breaking a sweat.

 

[cjard]

I am using .NET 2003. If an application uses many collections to store data from datasets (looping through the datasets to read data), does it slow down the application performance? If multiple users use the application, does it slow down the SQL server?

Thanks.
DanYeung

Might I ask what you logic is in looping through a collection, which is a store of data (dataset of datatables) and transferring the data into another collection?

i.e. When you make a cup of tea, do you get 57 cups out, brew the tea in the first one, then pour it into the second,m drink a bit, then pour the remaining into the third, drink a bit, then pour...

 

[danyeungw]

A very good question. The application was created by one of my co-workers. The server is hammered so bad that I was suspicious it might be the collections caused it. Before I raise the question, I want to do some research. The reason I thought it might be the collections issues is that he created a web page for my web app and it was slow. There were three levels drilldown on the page – Region, Distributor, and Request. There were about 1000 records and it took about 20 to 30 seconds to bring up the screen, because the three levels data was stored in three collections when the page was load. Even though, it shouldn’t take 20 to 30 seconds. So I assumed that the collections slow down the app performance. Do the collections affect the server performance? According to jmcilhinney, it wouldn’t. Any ideas?

Thanks.
DanYeung

 

[cjard]

Youre askign too generically, i'm afraid, but I suspect that your database design is somewhere between bad, and incorrect (or the usage of it is bad)

Database driven apps should use the database for searching and filtering. That is all a database does, all it is designed for.

Ive often seen people download an entire database table into a datatable, and then use datatable.Select rather than writing a proper query, because they didnt know how to do it.. They transfer 5 million records off a high powered quad cpu 10 gigs ram server, into a 256meg 1 cpu laptop, and then have it search.. Lame.. But it happens!


Your app should be set up so that the table is properly indexed with the index branches growing in the right order. Aggregate queries can provide a distinct list and count of all regions, sub regions etc, whatever.. and the same index will be used in query. Dont query thosands of rows from the db, as the user wont be able to take them all in anyway.

Certainly you shouldnt be loading lots of data into the client (in this case the app is ASP, so the web server is the client of the db).

I couldnt say for sure what was bad without reviewing the code, and I'm so busy right now, that I'd have to charge you for doing that. Perhaps another forum member can help, but really you need to go away and think very hard about the volumes of data flowing in your app.. data should be left ina database as long as possible, and the database tuned to rip through it.

 

[danyeungw]

First of all my original question was in vb.net not asp.net. I used the asp app for example. Second he didn't load the whole table(s) in the client. We are professional. My question was if collections slow down the performance. If you don't know or you are busy, you don't have to answer the question. Please keep it professional.

DanYeung

 

[ScottyB]

You may find this link informative.

Cutting Edge: DataSets vs. Collections -- MSDN Magazine, August 2005
http://msdn.microsoft.com/msdnmag/issues/05/08/CuttingEdge/default.aspx

Hope this helps

Regards

ScottyB

 

[cjard]

Second he didn't load the whole table(s) in the client.

I'm not saying he did, I'm jsut giving you an example of the bad coding I've seen. You've provided no code, nor any detailed descriptions of what you do beyond vague talk of "collections" and loading data into them. I've seen a thousand ways to write bad code; should I list them all to you so you can say 1000 times "no, he didnt do that"?
- if that's the way you think (and hence code) then it might actually be a good clue why your programs are slow: you pick inappropriate ways to do things. Tell us what he does do, not wait for us to suggest N possibilities of what he doesnt do..
(As a coding analogy: If you want to check a number is greater than 0, less than 99, and equal to 1, do you do one check, or do you check 98 times than it is not equal to number 0, 2, 3, 4, ... 99 ?)

We are professional.

I'd point out that your problem descriptions and bug reports arent.

My question was if collections slow down the performance.

That's like asking if a hammer affects the performance of a computer. You dont say whether youre repeatedly smashing the the keyboard with it, or whether it's just on the desk. Having "a collection" lying around in memory isnt going to "slow the performance". Having a million collections lying aroudn with 100 threads dedicated to reading and writing to them randomly and meaninglessly, is going to slow the performance of other threads.

Are you starting to see that your question is nearly completely nonsensical?

If you don't know or you are busy, you don't have to answer the question.

I guess I should say, put more effort into the question, describing your modus operandi etc, and someone will put more effort into an answer. As it stands, what can I day; i wrote a program that uses Collections to de-dupe data. It can write files faster than the RAID array can keep up. Clearly Collections dont "slow the performance" - should I just have said "No" in answer to your original question?