Lend
New member
- Joined
- Oct 4, 2014
- Messages
- 1
- Programming Experience
- 1-3
I am attempting to scrap text from multiple URL's but in quick succession. I currently have it set up in a queue system but it isn't fairing so well as it only gains a small amount of values.
Start Button
Scrap Method
Any ideas how I would make this faster?
Thanks so much!
Start Button
VB.NET:
Try For i As UInt64 = startID To endID
intQueue.Enqueue(i)
Next
For i As Integer = startID To endID
Dim t As New Thread(AddressOf Scrap)
t.IsBackground = True
t.Start()
Next
Catch ex As Exception
MessageBox.Show(ex.ToString)
End Try
Scrap Method
VB.NET:
Private Sub Scrap() Try
If lCount < endID Then
Do While intQueue.Count > 0
Dim source As String = WebReuqest(URL & intQueue.Dequeue)
If source <> Nothing Then
Dim r As New Regex("About (.*)")
Dim m As Match = r.Match(source)
If m.Success Then
UpdateListbox(m.Value.Split(" ")(1))
testLbl.Text = endID & ":" & lCount
Call rCount()
RichTextBox1.Lines = (From o In ListBox1.Items
Let ostr = o.ToString
Select ostr).ToArray
End If
End If
Loop
ElseIf lCount = endID Then
MessageBox.Show("Finished!")
End If
Catch ex As Exception
Dim sb As New System.Text.StringBuilder()
Dim Str = sb.Append("Error:").Append(" ").Append(ex.ToString).ToString()
MessageBox.Show(Str, _
"Critical Warning", MessageBoxButtons.OKCancel, MessageBoxIcon.Warning, MessageBoxDefaultButton.Button1, MessageBoxOptions.RightAlign, True)
End Try
Thanks so much!