Server stops responding

[elektrobank]

I have a UDP server that I developed. The majority of the time it works great, however in some cases people are telling me that when they quit the client program, then open it back up again, the server stops responding. They say that if they restart the server app, it starts working again. I have not been able to reproduce this on my end so I have no idea why this is happening. My guess is that it somehow stops receiving and that if I called the StartReceive() when this happens, it might work again. If that's the case, how should I handle that, should I have a loop running that keeps calling it? I have posted my code below, can anyone see any reason why this may be happening and what I can do to fix it? Thanks!!

Private Function startUDPServer() As Boolean

    listener = New Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp)

    listener.Bind(New IPEndPoint(IPAddress.Any, localPort))

    StartReceive()

End Sub

Private Sub DataReceived(ByVal ar As IAsyncResult)

        Dim buff As Byte()
        Dim content As String = ""

        Dim sender As New IPEndPoint(IPAddress.Any, 0)
        'Dim tempRemoteEP As EndPoint = CType(sender, EndPoint)
        clientEndpoint = CType(sender, EndPoint)

        Try

            Dim read As Integer = listener.EndReceiveFrom(ar, clientEndpoint)

            buff = CType(ar.AsyncState, Byte())

        Catch ex As Exception

        End Try

        StartReceive()

       doStuff(buff)

End Sub

Private Sub StartReceive()

    Dim buff(1024) As Byte

    Dim sender As New IPEndPoint(IPAddress.Any, 0)
    Dim tempRemoteEP As EndPoint = CType(sender, EndPoint)

    listener.BeginReceiveFrom(buff, 0, buff.Length, 0, tempRemoteEP, _
                                    New AsyncCallback(AddressOf DataReceived), buff)

End Sub