Accessing web sites (not displaying)

[JimmyFo]

Hi, I am wondering how a VB.Net application can access web sites. I'm not concerned with displaying the site within the application, but I would like to be able to go through a site automatically. Basically, I would point out a page (example, cnn.com) and the application would be able to do a GET or it's VB equivalent and get the page (source, images, etc) as long as it had security access. I'd like to be able to then redirect the app to visit one level lower on a selected link. Is this possible in VB.Net? Do they have equivalents of Redirect, URLDecode, Title, etc?

Thanks,
James

 

[Raven65]

I use the following code in a little pet project to load a web pages source into a text string. From here I do all sorts of regex parsing to get all the images, links, etc. I'd wager you can start here, and expand your app into what it is you desire.

(Url is a full "http://www.website.com/" string that I pass in)

Dim wrq As WebRequest = WebRequest.Create(Url)
Dim wrp As HttpWebResponse = DirectCast(wrq.GetResponse(), HttpWebResponse)
Dim sr As StreamReader = New StreamReader(wrp.GetResponseStream)
Dim Text As String = sr.ReadToEnd()
sr.Close()
wrp.Close()

Let me know if you need more information.

 

[JimmyFo]

Thanks a lot Raven, that's exactly what I was looking for!