Talking to Web Sites

[infidel]

I have a VB6 Windows app that parses references in a journal article, searches a web site for them, and inserts corrected information into the file. The VB6 app uses the old Inet control to send a smart URL to the site. What I get back is XML. I'm looking for the best way to get this functionality in VB2005. I do not want to display the web site in any way. The user need not even know it's there. This could be done with the Inet control, which was invisible at run time.

Now I've found the WebBrowser control, which will go to the site, but two things.

1. I don't need the visual part of the control. I guess I could set it invisible but that seems like a waste. Is there anything in VB2005 akin to the Inet control which would do web communications without display?

2. Assuming I use an invisible WebBrowser control, I'm having trouble getting the text. If I do

WebBrowser.Navigate(url)
text = WebBrowser.

it gets to the second line before the page is loaded and text = "". Looked for a setting to make it synchronous but could not find one.

I tried setting up a handler to detect the page loaded event and change a boolean variable when it occurs. I created a form-level boolean. Set to false before navigating, and have the handler call a method that sets it to true. Look likes this:

' Add an event handler that prints the document after it loads.
' docComplete() just sets documentCompleted to True
AddHandler WebBrowser1.DocumentCompleted, New WebBrowserDocumentCompletedEventHandler(AddressOf docComplete)

documentCompleted = False
WebBrowser1.Navigate(uri)
While Not documentCompleted
  '* Wait for document to finish
  System.Windows.Forms.Application.DoEvents()
End While
text = WebBrowser1.DocumentText()

Works, but is there a simpler way I'm missing?

 

[Paszt]

The System.Net.WebClient Class will allow you to download the web page with no GUI.
Here's an example: Webclient Class on MSDN

 

[infidel]

Thanks Paszt!! I think that's exactly what I want. I am trying to get it to work, but I can't figure out how to get my proxy username and password in. But I'm working on it. Thanks for the tip!

 

[Paszt]

I've never used a proxy before but the syntax would be something like:

Dim wClient As New System.Net.WebClient()
Dim prxy As New System.Net.WebProxy("host", portNumber)
Dim creds As New System.Net.NetworkCredential("username", "password")
prxy.Credentials = creds
wClient.proxy = prxy

All the above classes' constructors are overloaded so you may need to choose the appropriate overload for your situation.