NetworkStream and large amount of data

[Robert_Zenz]

Hello people.

A while ago I've started to write my own network layer based upon the TCPClient and the NetworkStream.

I'm using a while loop to receive data:

            Dim buffer(Me.prv_client.ReceiveBufferSize - 1) As Byte
            Dim read As Integer = Int32.MaxValue

            Do
                Try
                    read = Me.prv_strm.Read(buffer, 0, buffer.Length)
                Catch ex As IOException
                    Me.writeError(ex.Message)
                    read = 0
                End Try

                'Do something with it
            Loop While Me.prv_strm.DataAvailable

Unfortunately, this loop is faster then the underlying connection and DataAvalaible is false at some point. This happens especially with a big amount of data (> 500 Kb), 'cause at some point the loop is just faster.
If I know the size of the received data, there's no problem 'cause I read until I have everything. But what I don't have that information?

Of course, using Threading.Thread.Sleep() seems to be a solution, but that one feels just plain wrong.

Anyone some ideas on that?
Bobby

 

[JohnH]

What is it you're reading, and why don't you know the size?

If the reading is indefinite and you're just reading from the open stream you can do-loop on some other condition, f.ex until app or stream close or message known to be ended, and only read if data is available.

 

[Robert_Zenz]

I'm able to request the size for my purposes (f.e. FTP, POP3) but was wondering if it is really necessary.

I also tried other conditions, f.e. until the amount of read data is zero...but that does not only throw an exception, but does also kill my whole connection with the server. Which is not desirable if you're within a protocol session.

Loop While read > 0

I've also though of checking if read is smaller then the buffer, but there's the same problem as with DataAvalaible, that I'm faster with reading then the connection is with filling the buffer.

Thanks for replying and in advance,
Bobby

 

[anthony.selby]

I have the same issue I'm actually seriallizing a structure on the server side then sending that to the client ... I ended up just putting the .sleep(10) in there and in my network that seems to do the trick ... but your right that does seem totally wrong ... I can't think of a better way. .. if you come up with one I would love to hear it

 

[Robert_Zenz]

Hello anthony.

No I didn't come up with a better solution then this by now. Though, I've got some basic ideas how to bypass this...but none seems to be a really good idea to me. One was to build somekind of Timout-Loop into the function:

            Dim buffer(Me.prv_client.ReceiveBufferSize - 1) As Byte
            Dim read As Integer = Int32.MaxValue
            Dim timeOut As Integer = 0

            Do
                Try
                    read = Me.prv_strm.Read(buffer, 0, buffer.Length)
                Catch ex As IOException
                    read = 0
                End Try

                'Do something with it

                While timeOut < 256 And Not Me.prv.strm.DataAvailable
                      timeout += 1
                End While
                timeOut = 0
            Loop While Me.prv_strm.DataAvailable

This seems to be some way, but it is also a waste like Sleep() in my eyes...also I can't shake the feeling that the problem is not ours, but that of the Framework. The MSDN is full of examples which runs exactly into this issue, which tells me that even the MS people didn't expect this problem.

But you brought me on the idea to have a look at these classes in the Reflector...maybe I can find a solution down there. Though, I ask myself if the same issue exists within Mono, I'll try that one out tomorrow.

Bobby