Abnormal HttpWebRequest.GetResponse() Activity

[BlackIce]

Hi
The program I'm writing gets urls fed in from a database then searches for my companies url on the page returned from the database url. I'm using Dim response As HttpWebResponse = CType(request.GetResponse, HttpWebResponse) to retrieve the pages.

My Problem:
The HttpWebRequest.GetResponse just falls apart for one particular address and if the url is malformed. Instead of giving me a 404/Page not found status
I get a run-time error. It never gets to a point to check the status it just breaks on this line:
Dim response As HttpWebResponse = CType(request.GetResponse, HttpWebResponse) and then goes into my error handling the rest of my records don't get processed

For the broken link(www.stay-warsaw.com/other_resources.asp?group=36&groupname=travel) I get this error:
System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A socket operation was attempted to an unreachable host 89.171.73.120:80 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) at System.Net.Sockets.Socket.InternalConnect(EndPoint remoteEP) at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception) --- End of inner exception stack trace --- at System.Net.HttpWebRequest.GetResponse() at LinkValidator.frmMain.OpenUrl(String URL) in C:\Documents and Settings\User\My Documents\Visual Studio 2008\Projects\LinkValidator\LinkValidator\Form1.vb:line 126 Unable to connect to the remote server

For a malformed link("http://test.asp") I get this error:
System.Net.WebException: The remote name could not be resolved: 'test.asp' at System.Net.HttpWebRequest.GetResponse() at LinkValidator.frmMain.OpenUrl(String URL) in C:\Documents and Settings\User\My Documents\Visual Studio 2008\Projects\LinkValidator\LinkValidator\Form1.vb:line 126 The remote name could not be resolved: 'test.asp'

Here is my code can someone pls tell me where I've gone wrong the code works fine for working/properly formed links.

Private Function OpenUrl(ByVal URL As String) As String
Try
Dim newUrl As String
Dim returnHtml As String

Dim request As HttpWebRequest = DirectCast(Net.WebRequest.Create(URL), Net.HttpWebRequest)

request.Credentials = CredentialCache.DefaultNetworkCredentials

' Get the response.
Dim response As HttpWebResponse = CType(request.GetResponse, HttpWebResponse)

If response.StatusCode = HttpStatusCode.NotFound Then
Return Nothing
Else
' Get the stream containing content returned by the server.
Dim dataStream As Stream = response.GetResponseStream()
' Open the stream using a StreamReader for easy access.
Dim reader As New StreamReader(dataStream)
' Read the content.
Dim responseFromServer As String = reader.ReadToEnd()

returnHtml = responseFromServer
Return returnHtml
End If
Catch ex As Exception
'Send email with error message and close application
SendEMail(replyTo, errorMail, "HTML", "Link Validator Email Editor", ex.ToString() + vbCrLf + ex.Message)
Return Nothing
End Try
End Function

 

[JohnH]

HttpWebRequest.GetResponse Method (System.Net)
GetResponse method is documented to throw exceptions in given situations, such as those you outlined. Use Try-Catch blocks to catch exceptions, see also Exception and Error Handling in Visual Basic
To elaborate, in both cases you are not getting any response from server (either server don't exist, or server don't respond in time), and not a 404 response. The same error messages can be seen in a regular browser.

 

[BlackIce]

If that's the case I suppose
I'll just set the function to send a empty string on exception.

Thank you very much

 

[JohnH]

There is something missing in your function, you have to Close either the (Stream)reader or the response object. HttpWebResponse.Close Method (System.Net)

Also, there exist a simpler approach you perhaps can use, the WebClient class and its DownloadString method instead. WebClient.DownloadString Method (String) (System.Net)