I have been working on a small application that can crawl Google to show some of the search engine results pages and the WebClient normally connects fine, but after trying to connect to Google thru the WebClient proxy property, it just isn't connecting properly. Here is the code that I'm working with:
I know that I have found proxies that should work with Google as I've clearly tested them in my Google Chrome browser to see if they're dead or alive, but when I try to connect in my own application through the WebClient proxy setup, I keep getting the following error messages:
"Unable to connect to the remote server"
OR
"(503) Server Unavailable"
What am I missing? Could it be the headers are not fully configured or some other properties? Would appreciate any ideas.
VB.NET:
Try
Dim wc As New System.Net.WebClient()
'This line below will bring in the "User Agent" to make the HTTP request:
wc.Headers.Add("user-agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36")
Dim wp As New System.Net.WebProxy(TextBox1.Text)
wc.Proxy = wp
Dim HTML As String = wc.DownloadString("https://www.google.com/search?q=dog+training")
Catch ex As Exception
MsgBox(ex.Message)
End Try
I know that I have found proxies that should work with Google as I've clearly tested them in my Google Chrome browser to see if they're dead or alive, but when I try to connect in my own application through the WebClient proxy setup, I keep getting the following error messages:
"Unable to connect to the remote server"
OR
"(503) Server Unavailable"
What am I missing? Could it be the headers are not fully configured or some other properties? Would appreciate any ideas.