Question httpWebRequest.GetResponse redirection problem

cgautier

New member
Joined
Feb 9, 2012
Messages
2
Programming Experience
5-10
Hi, I've written a podcast retriever app that uses HttpWebRequest.Create (URL), httpWebRequest.GetResponse() and httpWebResponse.GetResponseStream() calls to retrieve podcast (.mp3) files from various web sites. As long as there is no URL redirection, the app works fine. When there is redirection, GetResponse goes astray (whether AllowAutoRedirect is True or False) and ends up pointing to an html file rather than to the podcast. The problem seems to be in the headers information returned by GetResponse. If I set AllowAutoRedirect to False and follow the redirection manually (by examining the httpWebResponse.Headers), I find that the header info doesn't match what I get when I use CURL to follow the redirection on the same URL.

For example, here's the CURL -I -L output for the URL: http://traffic.libsyn.com/pointofinquiry/POI_2012_02_06_Lawrence_Krauss.mp3

HTTP/1.0 302 Found
Date: Thu, 09 Feb 2012 14:30:41 GMT
Server: Apache/2.2.14 (Ubuntu)
X-Powered-By: PHP/5.3.8-ZS5.5.0 ZendServer/5.0
Location: http://hw.libsyn.com/p/d/d/0/dd0b3a533786a68b/POI_2012_02_06_Lawrence_
Krauss.mp3?sid=754a6272b039a82de7ae5d76b6b9108a&l_ sid=18988&l_eid=&l_mid=2893533
& expiration=1328800427&hwt=0d5ab5ea055869ea39825ed cd5b2e338
X-Libsyn-Host: traffic3.libsyn.com
Vary: Accept-Encoding
Connection: close
Content-Type: text/html

HTTP/1.0 200 OK
Date: Thu, 09 Feb 2012 14:28:37 GMT
Connection: close
Accept-Ranges: bytes
ETag: "1328572856"
Last-Modified: Tue, 07 Feb 2012 00:00:56 GMT
Expires: Fri, 10 Feb 2012 14:28:37 GMT
Content-Length: 15579891
Content-Type: audio/mpeg
X-HW: 1328797717.ce004s1


Note that there is one redirection here and that CURL correctly follows the redirection to the location where the mp3 file is located.

In my program, when I run this URL through the code, the header output returned is:

X-Libsyn-Host:traffic3.libsyn.com
Vary:Accept-Encoding
Connection:close
Content-Length:912
Content-Type:text/html
Date:Thu, 09 Feb 2012 14:51:10 GMT
Server: Apache/2.2.14 (Ubuntu)
X-Powered-By:pHP/5.3.8-ZS5.50 ZendServer/5.0


Note that this is the same as the first portion of the CURL output, but there is no Location. Without a Location value, the redirection ends (because there is nowhere to go). As a result the GetResponse fails to get to the URL that actually points to the mp3 file.

Anybody know what's going on here? Is there a HttpWebRequest or HttpWebResponse property that I need to set to some nondefault value to get GetResponse to return Location values? Is there possibly some incompatibility between Apache servers and the .NET code that's causing GetResponse to return bad (or incomplete) header data? Any suggestions for how to fix this would be much appreciated. Thanks.
 

JohnH

VB.NET Forum Moderator
Staff member
Joined
Dec 17, 2005
Messages
15,540
Location
Norway
Programming Experience
10+
You need to set UserAgent header for server to give proper response. AllowAutoRedirect to True gets you directly to the redirected url.
WebClient class will also get the job done with less code, setting UserAgent header applies to that as well.
 

cgautier

New member
Joined
Feb 9, 2012
Messages
2
Programming Experience
5-10
That did the trick. Setting the UserAgent string so my app mimics a browser (e.g. IE9) made redirection work correctly. Many thanks JohnH.
 
Top Bottom