Question How to parse through website hit?

[lolarennt]

Hi community,

My first post here. Please be kind to me...

I am fairly new to vb.net, but have quite some experience with Delphi, and did some VBA. Oh, I'm old school. I like DOS.

So, I need to write a program, and chose vb.net (visual studio 2008) as my language to do this in.

What the program is supposed to do is generate a webpage hit (but not necessarily render the html in a browser) and then parge through the returned HTML code, finding links, certain key words etc.

How should I go about this? I.e., how do I generate a hit to a webpage, but I don't want to display this webpage, but rather parse through the code of the page itself.

Thanks for your insights.

 

[ProtekNickz]

Hi their a few bits on here about this allready mentioned, have looksee

http://www.vbdotnetforums.com/vb-net-general-discussion/37395-search-website-return-text-another-webpage.html#post109194

 

[TheDeathly]

Just do

Dim Webpage as string = getpage("http://www.google.com")

 Function GetPage(ByVal pageUrl As String) As String
        Dim s As String = ""
        Try
            Dim request As Net.HttpWebRequest = Net.WebRequest.Create(pageUrl)
            Dim response As Net.HttpWebResponse = request.GetResponse()
            Using reader As IO.StreamReader = New IO.StreamReader(response.GetResponseStream())
                s = reader.ReadToEnd()
            End Using
        Catch ex As Exception
        End Try
        Return s
    End Function

then just start splitting it or using it for whatever you need. The thing is, im not sure if this generates a hit. im pretty positive it should since its grabbing the code. another way to go about this is making a web browser, making it view that page on load up. making the web browser invisible and im pretty sure its like

Dim WebPage as string = WebBrowser1.DocumentText()

So what youd need to do is

' on form load
webbrowser1.visible = false
webbrowser1.navigate("http://www.google.com")
'On web browser document completed
dim Webpage as string = webbrowser1.documenttext()
or what ever you need to do with the html code.

Hoped this helped! good luck...