store memory to disk

[warnockm]

I using a DataSet object to get data from a MySql database into a vb.net program. It transfers about 40000 rows in a few seconds which is great. Now i'm trying to use the application w/o the DB since all it is is a list of pairs of numbers. The DataSet object stores to XML very nicely, but takes forever to load. Is there a way to store the DataSet as a binary memory "dump", or use another datatype, like an ArrayList that will store the contents directly to the disk for easy loading?

 

[JohnH]

My equal test DataSet (40000 rows, pair of numbers) takes only 3 seconds to load from xml and load into DataGridView. (Also it's only one line of code each with ReadXml and WriteXml methods).

Dataset is serializable and it is possible serialize it as binary stream, this creates a much smaller human-unreadable (and in other Xml contexts unusable) file and is only slightly faster - in this very large test case same procedure took about 2 seconds.

Here are examples for writing and reading Dataset binary:

    Sub write_ser(ByVal ds As DataSet)
        ds.RemotingFormat = SerializationFormat.Binary
        Dim ser As New System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
        Dim fn As String = Application.StartupPath & "\test.bin"
        Using fs As New IO.FileStream(fn, FileMode.Create, FileAccess.Write)
            ser.Serialize(fs, ds)
        End Using
    End Sub
 
    Sub read_ser()
        Dim ds As DataSet
        Dim ser As New System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
        Dim fn As String = Application.StartupPath & "\test.bin"
        Using fs As New IO.FileStream(fn, FileMode.Open, FileAccess.Read)
            ds = DirectCast(ser.Deserialize(fs), DataSet)
        End Using
        DataGridView1.DataSource = ds.Tables(0)
    End Sub