Fastest Data Retrieval. VB.net

[qwazimoto]

Hi people

I am in my first year of programming, Im learning VB.net and I am building a media player using the WMP component. I would like to know which is the best way to store media info given the options of a MS Access data base, or a random access file, or create a structure for each song and add to a collection and serialize the collection, well I guess deserializing is the key, and I guess a .txt file is an option too. I have built some simple media players before but, having 5000 songs it takes a signifigant amount of time to load, and as I understand an MS Access database isnt realy all that reliable for anymore than 4000 rows of data (am I correct with this?).
The Database will need 8 or so columns, structure 8 variables and well if im using a text file I dont really need any advice on that, other than if it is better suited to my application. I have always used txt files for my media players and the results not being all that impresive. I also would like to know how to go about the fastest way of putting this data into a listview box currently I declare a listview item create an array for the data in the listview row and set the Items contents to the array and then add to listview in a for loop...
NOTE:The media information only needs to be read once(when loading)
(any options i miss or am unaware of please tell)

Thanks in advance...

 

[jmcilhinney]

Use a database for all but the simplest of scenarios.

 

[qwazimoto]

So Is this considered a simplest of situations?
if I use a textfile there would be upto maybe 500 chars on the biggest line
probably more like 100 or 200 on avereage, with a few tabs inbetween, the db would have about 8 or 9 feilds.
Well I guess I could try all options in a hack and time them I spose but getting an awnser would be easier.This by itself is not all that important but there is allot of other stuff to do at form load so im trying to speed anything up.

1: There will be upto 5000 lines. is an mdb the speediest option ?

2: Am i limited to users needing access database drivers ?

3: If access turnes out to be ok, will it handle 5000 rows ?

 

[jmcilhinney]

A text file could handle your situation OK. A single table of data in a single text file of 5000 lines wouldn't really be an issue. Access would also handle it without breaking a sweat. The ability to interact with Access databases is built into Windows. Access would offer more flexibility in future if there is a likelihood that things may change.

 

[Anti-Rich]

hey all,

i was thinking that if you want speed, couldnt you just use an XML file rather than an mdb? xml is extremely flexible and can be searched through quite easily (and quickly) using the methods in .net.

hope it helps mate

have a good one
regards
adam

 

[jmcilhinney]

That's quite true. In fact, you can also load and save a DataSet directly to XML with its ReadXml and WriteXml methods.

 

[qwazimoto]

Indexing XML

I dont know buggerall about XML.
Can I add my library info to XML and index by alphabetical order, for example, searching for a song, get the first letter(e.g "s") and then find the index where "s" starts without having to search through a-r?
I could come up with many ways to do this but an inbuilt way is always best.
So here is some psuedo code

strText = textbox1.text
intIndex = goto xml files first index of strText
return intIndex

for i = intIndex to what ever
if ADF(intIndex).text = textbox1.text then

end if
next

Im sure you all get what im trying to say, cause I spose another big question is, which I am sure the awnser is yes, Can VB.net search an array faster than a DB or XML,
because I could do the indexing on a background process or thread at form load and save to file at form close if no changes media was added...
Actually I think thats a pretty good Idea and i might go with that, unless any better suggestions

 

[Anti-Rich]

hmm... yeah probably it would search an array faster than a db or xml, but if you want to search even faster, load all values into a List (Of Type), and this will search much faster, because the values are stored in a hash table. and hash tables are extremely efficient at searching.


regards
adam

 

[qwazimoto]

Ok.

so...

Heres my conclusion.

Use a type(structure) for each media item...
serialisie and deserialize this array of structures...
simple

Thats what I was just doing for another project with a collection rather than array.

 

[jmcilhinney]

The generic List is NOT implemented using a hash table. Just like the ArrayList, of which the generic List is basically a strongly-typed version, it is implemented using an array.

 

[Anti-Rich]

ok, am i mixing it up with a Dictionary?

 

[cjard]

yesh.. dictionary is lookup of values from the hash of the key

quaz - use a database if you plan on searching anything.. to write your own code for searching lists, text files etc. is reinventing a pretty-good-already wheel

 

[Anti-Rich]

hah, whoops my bad.

is there any performance hit when using a list compared to a dictionary?

 

[jmcilhinney]

hah, whoops my bad.

is there any performance hit when using a list compared to a dictionary?

They are two different things used for two different purposes. A List is for a simple list of items where each is retrieved by index. A Dictionary is for a keyed list of items, where each item is identified by a unique key. You might have a Dictionary of American citizens where their social security number is the key. Dictionaries are fast to retrieve items because they are stored in what's known as a "hash bucket" acoording to their key. When you add an item its key is hashed and the item stored in a hash bucket corresponding to that value. When you retrieve an item the key you specify is hashed and the corresponding hash bucket immeidately retrieved. If it only contains one item then you have your item immediately. Multiple keys can correspond to the same hash value though, so as the collection gets bigger there is a greater chance of hash buckets containing more than one item. It's still faster to search the contents of a single hash bucket than it is to search the whole collection though. A crowded collection may contain hundreds of items but each hash bucket wouldn't have more than a few items. I don't know the exact ratio but you get the idea.

 

[Anti-Rich]

so what youre saying is even though a dictionary item is stored by a key, it is possible for several items to be referenced by that key (or a collection of items?)? im a little confused.

a side note: is it possible to store arrays in dictionaries?

regards
adam