Question Proxies - Optimizing/Fixing problems

[xGhost]

Hi

I'm at a next step in my learning phase with webrequests. Basicly I want to learn how to use proxies most efficiently with a single webrequest.

I learned how to use them with one webrequest, so for my test I've set up a simple loop like this with a bigger number of proxies in order to see difficulties, speed etc:

For x as integer = 0 to proxies.count-1 'proxies could be an arraylist with proxy strings
      'do a webrequest with the current proxy - call method x
next

In method x:

- I do a webrequest
- if the response is nothing, I check the exception message and re-do the request with another proxy
- if the response was successfull I check wherether the result (html) is the result I want* and close the response.
* -> sometimes proxies return a server page, you think you received a valid response then but actually you need to check the response html

Methods for the proxy:

theProx = proxy.Split(":")
request.Proxy = New WebProxy(theProx(0), Convert.ToInt32(theProx(1)))
request.Timeout = 10000
request.KeepAlive = False

Questions:

Is it possible to speed up 1 webrequest with proxies, I do not mean using multiple webrequests at thesame time.
It seems that it takes ages to get some valid responses. Next to that most of the time the response isn't valid. 
Even if I checked all the proxies with a good proxy checker before I've put them into an arraylist

Some exception messages I've encountered:

The operation has timed out, this is the most given message
The server committed a protocol violation. Section=ResponseStatusLine, this is the second most given message
The underlying connection was closed: An unexpected error occurred on a send. (When I where trying with a post to my server.
The underlying connection was closed: An unexpected error occurred on a Receive.
The remote server returned an error: (404) Not Found.
The remote server returned an error: (403) Forbidden.
Invalid URI: Invalid port specified.
The remote server returned an error: (500) Internal Server Error.
The underlying connection was closed: The connection was closed unexpectedly.

Like I said before I started my test application I tested the proxies with a proxy checker called AccessDiver:
The fake 301 message doesn't mean the proxy isn't valid, this message is occured because I'm behind a router.

And at last some statistics:

Valid proxy amount from AccessDiver was 60 proxies.
I could get 3 proxies giving a response in my test application, 57 proxies didn't work. It took ages to do the 60 requests, which is not really acceptable,

Where does the big delay occurs, making the proxy webrequests really slow:

At this line: response = request.GetResponse() After this line either way a exception is thrown (if the timeout occurs or any other problem), or it continues to the streamreader to check the result.
streamReader = New StreamReader(response.GetResponseStream(), Encoding.ASCII)
result = streamReader.ReadToEnd().Trim()

I do not expect miracles, its obvious that working with a single request + a proxy is slow, But I do hope if I can optimize a bit, solve for example those protocol messages or try to speed it a little bit up. Thanks for the time.

 

[xGhost]

These are a bunch of exceptions whom won't be solved in 123, but any help is appreciated. I will give example code where needed.