Question Adding large list of web urls to VS from a text file ?

[blakeandsons]

I am currently looking to code my first web browser/form filler application. So far I have the browser and a few form fields that are working.

At the moment I am only able to have it load 1 url but I need it to cycle through a .txt file with about 1200 different url's while form filling.

My question is how would I load from a .txt file instead of just the one designated url as I have working now?

 Private Sub btnSubmit_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnSubmit.Click
        WebBrowser1.Navigate("https://login.yahoo.com/config/login_verify2?.intl=us&.src=ym")
    End Sub

Any help is appreciated.

 

[jmcilhinney]

E.g.

Imports System.IO

Public Class Form1

    Private urls As Queue(Of String)

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        Me.urls = New Queue(Of String)(File.ReadAllLines("file path here"))
        Me.LoadNextPage()
    End Sub

    Private Sub WebBrowser1_DocumentCompleted(sender As Object, e As WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
        If Me.WebBrowser1.ReadyState = WebBrowserReadyState.Complete Then
            Me.ProcessPage()
            Me.LoadNextPage()
        End If
    End Sub

    Private Sub LoadNextPage()
        If Me.urls.Count > 0 Then
            Me.WebBrowser1.Navigate(urls.Dequeue())
        End If
    End Sub

    Private Sub ProcessPage()
        '...
    End Sub

End Class

 

[blakeandsons]

That worked perfectly, thank you so much. I am having an issue getting the form filler to work with the browser now though (Keep getting errors when trying to execute both at the same time). Is there anyway you could show me how to add the form fill side of it to the code you already provided me. Sorry I'm just getting back into VB since about 2000 and its rougher then I remember.

This was what I came up with before I came here, was entering 4 fields and having to to manually submit. (Had not got around to trying to submit through VB)

Public Class frmMain

    Private Sub btnSubmit_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnSubmit.Click
        WebBrowser1.Navigate("https://login.yahoo.com/config/login_verify2?.intl=us&.src=ym")
    End Sub

    Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
        Dim PageInput As HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName("input")
        For Each elem As HtmlElement In PageInput
            If elem.GetAttribute("name") = "firstname" Then
                elem.SetAttribute("value", txtFName.Text)
            End If
            If elem.GetAttribute("name") = "lastname" Then
                elem.SetAttribute("value", txtLName.Text)
            End If
            If elem.GetAttribute("name") = "email_address" Then
                elem.SetAttribute("value", txtSite.Text)
            End If
        Next

        Dim Comments As HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName("textarea")
        For Each elem As HtmlElement In Comments
            If elem.GetAttribute("name") = "comments" Then
                elem.SetAttribute("value", txtComments.Text)
            End If
        Next
    End Sub
End Class

 

[jmcilhinney]

What errors? I notice that you're not testing the ReadyState so you might be processing the page before the whole thing is loaded.

 

[blakeandsons]

What errors? I notice that you're not testing the ReadyState so you might be processing the page before the whole thing is loaded.

Im getting errors on lines 2 and 5: Statement is not valid in a namespace (Both Column 1)

Imports System.IO
Private Sub btnSubmit_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnSubmit.Click
End Sub

Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
    Dim PageInput As HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName("input")
    For Each elem As HtmlElement In PageInput
        If elem.GetAttribute("name") = "firstname" Then
            elem.SetAttribute("value", txtFName.Text)
        End If
        If elem.GetAttribute("name") = "lastname" Then
            elem.SetAttribute("value", txtLName.Text)
        End If
        If elem.GetAttribute("name") = "email_address" Then
            elem.SetAttribute("value", txtSite.Text)
        End If
    Next

    Dim Comments As HtmlElementCollection = WebBrowser1.Document.GetElementsByTagName("textarea")
    For Each elem As HtmlElement In Comments
        If elem.GetAttribute("name") = "comments" Then
            elem.SetAttribute("value", txtComments.Text)
        End If
    Next
End Sub

Public Class Form1

    Private urls As Queue(Of String)

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        Me.urls = New Queue(Of String)(File.ReadAllLines("c:\test2.txt"))
        Me.LoadNextPage()
    End Sub

    Private Sub WebBrowser1_DocumentCompleted(sender As Object, e As WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
        If Me.WebBrowser1.ReadyState = WebBrowserReadyState.Complete Then
            Me.ProcessPage()
            Me.LoadNextPage()
        End If
    End Sub

    Private Sub LoadNextPage()
        If Me.urls.Count > 0 Then
            Me.WebBrowser1.Navigate(urls.Dequeue())
        End If
    End Sub

    Private Sub ProcessPage()
        '...
    End Sub

End Class

 

[jmcilhinney]

You have two event handlers (btnSubmit_Click & WebBrowser1_DocumentCompleted) outside the form class. All the form code has to be inside the form, i.e.

Public Class Form1

    'All form code goes in here.

End Class

 

[blakeandsons]

Ah I See, last one for the night. Now I get:
Error 1 'Private Sub WebBrowser1_DocumentCompleted(sender As Object, e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs)' has multiple definitions with identical signatures.

Private Sub WebBrowser1_DocumentCompleted is referenced in the form filling and in your read from text both inside the form. I can change Private Sub WebBrowser1_DocumentCompleted to Private Sub WebBrowser1_DocumentCompleted2 on one of but it kills the form filling on compile. How can I use both instances at the same time?

 

[jmcilhinney]

This is a perfect example of why I don't like posting code: because it gets copied and pasted without understanding. The code I posted was an example only and not intended to be copied and pasted it was in its entirety. My code included a full class definition. If you already had a class definition, which you would have to have had, then you would copy mine. If you already had an event handler for your WebBrowser's DocumentCompleted event then you wouldn't copy mine. Take from my example only what you need. Do not just copy and paste it because it was never meant to be used that way. Take a bit of time to understand what I posted and then the bits you need will become clear, e.g. the body of a method rather than the entire method definition.

 

[Herman]

The first hint should have been "form-filler" here John. :encouragement:

It's never a legitimate learning experience when it involves automated online form filling before knowing what a class is.