How to receive bytes through tcpclient or sockets?

[PatM]

Under VB6 with winsock I spend half a day writing an app that receives a plain old byte array that was mostly text but also had control characters. Under VB.net I've been trying for four days to do the same thing without success.

I stole a client server example from MS Windows Simple Multi-User TCP/IP Client/Server Example sample in VB.NET for Visual Studio 2010 and used the client portion (the server is actually a piece of hardware, nothing I can change). This almost works but it appears the encoding is mangling the stream and I can't figure out how to stop it. I triend encoding.default (example uses acii I think) with no luck.

I also thought why bother with passing strings between the connection class and main app but I really can't figure out how. Why something so simple in VB6 has to be so mind destroyingly difficult in .net is beyond me. I was finally starting to get to like vb.net too.

Can anyone point to an example that uses simple byte arrays or give me a clue as to how to modify this one?

Thanks!

Edit: I should clarify.. The data coming in is 8 bit ascii and looks something like this

<first byte values as a command sequence>
200
15
8
9
<then text>
serial: # 12356
version: 2.83.1
blah blah blah
<final byte value meaning end of message>
5

EOLs are chr(10)

When I get these through vb.net the EOLs are messed up, I end up with data after the chr(5) etc. In VB6 with winsock the message comes in perfect.y

 

[JohnH]

With a connected TcpClient you call GetStream to get the underlying NetworkStream. You can then call the Read method to request reading a set of bytes from the stream.
The parts where you need to convert bytes to an encoded ascii string you can use Encoding.Ascii and GetString method.
If all data is line terminated ascii strings you can use a StreamReader with encoding set to Encoding.Ascii to read the NetworkStream, you would then just call ReadLine to read each.

 

[PatM]

Thanks for the tips. I'm going to stay away from the encoding methods because they garble the text. I have to check bytes for commands so I'll construct the strings using chr() after checking for commands.

I also just stumbled across a description of asnychronous tcpclient reads that finally clued me in as to what it is. I'm going to persue that because it gets me away from using seperate classes (which I've never wanted to try and figure out) and with the callbacks it's about the same as winsock - I just get to name the sub instead of using the pre-named winsock sub. If I'm not mistaken (and there's NO guarantee I'm not!) I receive a byte array by default in the async result object so I've no difficulties with .net mangling the data.

 

[JohnH]

I'm going to stay away from the encoding methods because they garble the text

Then you either use the encoding incorrectly with the provided data, or use the wrong encoding.

so I've no difficulties with .net mangling the data

.Net will not mangle anything, if things get mangled up it's because you did something wrong in your code.

I also just stumbled across a description of asnychronous tcpclient reads that finally clued me in as to what it is.

TcpClient is a wrapper for a NetworkStream over a Socket. The NetworkStream can be read with the synchronous Read method or the asynchronous BeginRead/EndRead methods, both does the same.

 

[PatM]

I figured out the problem an you're right. It's the demo code I was working from that's the problem. It was written as a chat server and never expected more than 63 bytes to be in a packet. Not only was the buffer only 63 bytes but it never checked if there was more data, it just re-started it's networkstream read so lots of bytes were lost. the ASCII encoding was an issue AFAICT but changing that to default works fine. I just increased the buffer size to the size of the receive buffer and along with the encoding change I got all the data.

I've dumped that and I'm experimenting with BeginReads. I'm trying to figure out how to get the data out of the IAsyncResult and how to dimension the BeginRead buffer appropriately. My biggest question at the moment though is the callback receiving the result. All the examples have ByVal xxx as IAsyncResult but if I have an 8k buffer (default receive buffer size) won't that be a big hit? Should I use ByRef and a class level buffer variable instead of a local in the BeginRead procedure? Nothing gets written to the buffer until I issue another BeginRead so there's no issue of data getting overwritten (Or at least it seems that way).

So, local and ByVal for the buffer or class level and ByRef?

Thanks!

 

[JohnH]

I'm trying to figure out how to get the data out of the IAsyncResult and how to dimension the BeginRead buffer appropriately.

What you do is to allocate a byte array the size needed for the requested bytes, then you call (networkstream).BeginRead. When the callback is invoked you must call (networkstream).EndRead, which returns the number of bytes received and written to your byte array.

Should I use ByRef

Don't worry about ByRef/ByVal, those are passing mechanisms, ByVal is default and should be used in almost all cases. Usually (including arrays if that applies) you are passing objects that are references anyway.

So, local and ByVal for the buffer or class level

BeginRead method allows for passing a state object, where you can put any value/object. That state can be retrieved from the IAsyncResult in callback.

 

[PatM]

Ah, so if the callback procedure used byref it'd be receiving a pointer to a pointer while byval results in a simple pointer. I was envisioning multiple copying of arrays being tossed around and chewing up CPU time.

As for the rest I've got that functioning (using netstream.Read in the callback to fill my byte array) and I'm just trying to figure out why the callback doesn't seem to get called until I've received two <1k communications from the machine. I set myTCPClient.NoDelay=true before doing BeginRead but it seems like thats not enough data to trigger the callback. I have to check but perhaps I've set NoDelay after getting the stream from myTCPClient and it needs to be done first? Still googling that.

AT the moment I have the buffers so large they can contain waaay more than the messages contain (~8k) and I don't loop to check for more data. Later I'll be getting trend logs in excess of 500k so I'll try getting that working before I try downloading actual logs. Once I get things (apparently) working I'll post some code and see if I've made any deadly mistakes.

Thanks!

 

[JohnH]

I have to check but perhaps I've set NoDelay after getting the stream from myTCPClient and it needs to be done first?

That does not matter, but you must set it before requesting reads. NetworkStream object is only initialized first time you call GetStream and associated with the Socket object, you can call GetStream method any number of times after that and the existing stream object is returned.

 

[PatM]

Ok I've been experimenting and experimenting and I'm just not quite getting there. I'm back to using the class from the chat example listed in the first post and it's sort of working for what I'm currently doing.

The big issue is that I have to receive data twice before the callback happens. I have nodelay set and I'm sure it's working because when I do a 4 byte write to the stream I see the machine's LCD display light up (which it only does when receiving a valid command). I don't use Flush after either so nodelay is working.

The first time the command is sent nothing happens at the client end. When I sent the command again I receive one copy of the data I know I should receive instead of two. So I figure something is short circuiting and the first batch of data has disappeared into digital purgatory. Here's the relevant parts of the code, I'm hoping someone can see the problem.

Main Form Declarations

Private _Connection As ConnectionInfo
Private _ServerAddress As IPAddress

Connection sub - only called for a new connection

    Private Sub DoConnection(IP As String, port As Integer)

        SetConnecting()
        strServerPort = Str(port)
        strServerIP = IP
        intConnectionStatus = conConnecting
        lblConnStatus.Text = "Connecting"
        Me.Refresh()
        _ServerAddress = IPAddress.Parse(IP)
        Try
            _Connection = New ConnectionInfo(_ServerAddress, port, AddressOf InvokeAppendOutput)
            _Connection.AwaitData()
            intConnectionStatus = conConnected
            SetConnected()
        Catch ex As Exception
            MessageBox.Show(ex.Message, "Error Connecting to Server", MessageBoxButtons.OK, MessageBoxIcon.Exclamation)
        End Try
    End Sub

the main class's way of receiving data from the connectioninfo class

    'The InvokeAppendOutput method could easily be replaced with a lambda method passed
    'to the ConnectionInfo contstructor in the ConnectButton_CheckChanged event handler
    Private Sub InvokeAppendOutput(message As String)
        Dim doAppendOutput As New Action(Of String)(AddressOf AppendOutput)
        Me.Invoke(doAppendOutput, message)
    End Sub

    Private Sub AppendOutput(message As String)

        MsgBox("Got Data")
        Dim intCount As Integer
        Dim theByte As Byte
        Static Dim intCommandStep, intLineCount As Integer
        Static Dim boExtraSpace As Boolean = False

        For intCount = 1 To message.Length
            theByte = Asc(Mid(message, intCount, 1))
            Select Case intCommandStep
                Case 0
                <bunch of code removed>

And finally the connectioninfo class

    'Encapuslates the client connection and provides a state object for async read operations
    Public Class ConnectionInfo
        Private _AppendMethod As Action(Of String)
        Public ReadOnly Property AppendMethod As Action(Of String)
            Get
                Return _AppendMethod
            End Get
        End Property

        Private _Client As TcpClient
        Public ReadOnly Property Client As TcpClient
            Get
                Return _Client
            End Get
        End Property

        Private _Stream As NetworkStream
        Public ReadOnly Property Stream As NetworkStream
            Get
                Return _Stream
            End Get
        End Property

        Private _LastReadLength As Integer
        Public ReadOnly Property LastReadLength As Integer
            Get
                Return _LastReadLength
            End Get
        End Property

        Private _Buffer(100) As Byte

        Public Sub New(address As IPAddress, port As Integer, append As Action(Of String))
            _AppendMethod = append
            _Client = New TcpClient
            _Client.NoDelay = True
            _Client.Connect(address, port)
            _Stream = _Client.GetStream
        End Sub

        Public Sub AwaitData()
            ReDim _Buffer(_Client.ReceiveBufferSize)
            _Stream.BeginRead(_Buffer, 0, _Buffer.Length, AddressOf DoReadData, Me)
        End Sub

        Public Sub Close()
            If _Client IsNot Nothing Then _Client.Close()
            _Client = Nothing
            _Stream = Nothing
        End Sub

        Private Sub DoReadData(result As IAsyncResult)
            Dim info As ConnectionInfo = CType(result.AsyncState, ConnectionInfo)
            Try
                If info._Stream IsNot Nothing AndAlso info._Stream.CanRead Then
                    info._LastReadLength = info._Stream.EndRead(result)
                    If info._LastReadLength > 0 Then
                        Dim message As String = System.Text.Encoding.Default.GetString(info._Buffer)
                        info._AppendMethod(message)
                    End If
                    info.AwaitData()
                End If
            Catch ex As Exception
                info._LastReadLength = -1
                info._AppendMethod(ex.Message)
            End Try
        End Sub
    End Class

 

[JohnH]

It seems to me your method of handling the received bytes is fragile and depends on each read being one complete 'package'. I'm also not sure about how well defined that 'package' of data is.

 

[PatM]

It seems to me your method of handling the received bytes is fragile and depends on each read being one complete 'package'. I'm also not sure about how well defined that 'package' of data is.

I know what you mean - but I have to deal with what the machine puts out. Basically a package starts with a 4 byte command code which is just an echo of the 4 byte command sent to the machine. After the four bytes is an unknown amount of text and the end of the package is a single byte value of 5. That part I have working, I can complete a package regardless of how many parts it's sent in. It's the non-firing callback that has me flummoxed.

As an experiment I started a completely new project and used a different tcp wrapper class then made nothing more than three buttons (connect, disconnect, get data). The callback for receiving data does nothing other than a message box saying "Got Data!".

When I hit the get data button the machine's LCD lights up, showing it received a valid command. No messagebox appears so the callback simply isn't executing. If I send the command again I receive data twice. I haven't tried diplaying what data I was receiving yet, I just wanted to see if it was just the original project.

I did some searching and found there's a second NoDelay setting - I thought the TCPClient.NoDelay was for both send and receive (thats what the docs allude to) but there's also TCPClient.Client.NoDelay - so I set both to true and tried again, with the same result.

I stopped by virus checker (someone reported that could cause delays - but mine isn't a delay, it just doesn't fire).

I've been trying to figure out where the threshold value is kept for when nodelay is false. Perhaps nodelay is being ignored and there's a threshold of 1k bytes or something (like you can set in the old winsock). Perhaps I need to set that to 0 or 1 so it fires regardless.

 

[JohnH]

I did some searching and found there's a second NoDelay setting - I thought the TCPClient.NoDelay was for both send and receive (thats what the docs allude to) but there's also TCPClient.Client.NoDelay - so I set both to true and tried

They are the same. You may recall I said TcpClient is a wrapper for a Networkstream over a Socket, Client property expose that Socket. TcpClient.NoDelay property just sets that option for its Socket.

Maybe the sending socket uses the default Tcp Nagle algorithm and buffers output? I don't see anything in the NetworkStream that would cause buffering in any direction, but you could try just to make sure creating just a Socket, Connect it, Send the command bytes, and Receive bytes.

 

[PatM]

Actually, it's just me. As is often the case I ran off and did something completely unrelated to programming and while thinking of completely programming unrelated things it suddenly popped into my head.

I had gotten so wrapped up in tcp stuff that I'd somehow forgotten the command header is six bytes, not four. The particular command I was using was just to send over the machine setup information so the last two byte values didn't matter. When I sent the second command it filled in the last two bytes (which are ignored in this case) and initiated the tranfer. THe extra two bytes aren't the correct values to start a new command so they are ignored.

The first three bytes of the command are like a lock combination and denote the start of a new command. Once the fourth byte (the actual command type) is received the LCD lights up. I was thinking it took an entire command header to light the LCD but a bit of experimenting shows it only takes four.

That means that my prior experimentation with BeginRead was probably working just fine. I'll go back to that instead of using other's classes. The classes work fine now but I'd rather keep experimenting until I feel I understand the tcpclient and maybe even direct socket methods. Never know when I might have to do something fancier at a later date.

My VB6 project was sending the proper six byte commands. I should have gone back an examined it again when I started running into this trouble.

Thanks very much for attempting to help! Not much you can really do when it's the dofus on the other end doing silly things 8)

 

[JohnH]

Great you figured that out!

 

[PatM]

Great you figured that out!

Almost... Call me dense but I just can't figure out how to retrieve the byte array from AsyncState. I see lots of examples of how to retrieve a string like:


Dim message As String = System.Text.Encoding.Default.GetString(info._Buffer)

But I need the raw bytes. When I look in the AsyncState members and methods I see nothing remotely useful. I've tried casting asyncstate to a network stream but none of it's members seem useful either...

myNetStream.BeginRead(bBuffer, 0, bBuffer.Length, New AsyncCallback(AddressOf invDataRead), myNetStream)

then the callback

Private Sub invDataRead(ns As System.IAsyncResult)

From there I've had no luck at all. I've been googling for hours (last night and started again at 3:00 AM... The MS docs, as usual, are full of non workable examples (none of which are what I want to do anyway).