rexcarnation
Member
Hello,
I search all day and I cannot find the answer how to multi threading with queue. So like this.
I have 100 URL in datagridview on col 1, an then I want to scrape all url on the datagrid by 10 thread, so the process only 1 -10 on the datagridview list. And when one of all process (scrape) finish it will do next scrape on the 11 and next until all of 100 URLs processed. It like queue but running 10 queue at once.
Here is my code.
my code just run only 1 -10 not the other.
plz help me
i found some article here Create Multiple Background workers with WebClient-VBForums
but any thread which process the same url.
I search all day and I cannot find the answer how to multi threading with queue. So like this.
I have 100 URL in datagridview on col 1, an then I want to scrape all url on the datagrid by 10 thread, so the process only 1 -10 on the datagridview list. And when one of all process (scrape) finish it will do next scrape on the 11 and next until all of 100 URLs processed. It like queue but running 10 queue at once.
Here is my code.
Private Sub BtGrab_Click(sender As Object, e As EventArgs) Handles BtGrab.Click ToolProgress.Maximum = DataURL.Rows.Count ToolProgress.Value = 0 Dim maxthread As Integer = 10 Dim upperBound = maxthread Dim waitHandles(upperBound) As ManualResetEvent For i = 0 To upperBound waitHandles(i) = New ManualResetEvent(False) ThreadPool.QueueUserWorkItem(AddressOf ProcessItem, i) Next End Sub Private Sub ProcessItem(ItemIndex As Object) DataURL(4, ItemIndex).Value = "Process" Try DataURL(4, ItemIndex).Value = "Scrapping" Dim request As HttpWebRequest = HttpWebRequest.Create(DataURL(1, ItemIndex).Value) request.Timeout = 10000 Dim response As HttpWebResponse response = CType(request.GetResponse, HttpWebResponse) Dim webstream As Stream = response.GetResponseStream Dim streamreader As New StreamReader(webstream, Encoding.UTF8) Dim html As String = streamreader.ReadToEnd response.Close() DataURL(4, ItemIndex).Value = "Great" Catch ex As Exception DataURL(4, ItemIndex).Value = "Fail" End Try End Sub
my code just run only 1 -10 not the other.
plz help me
i found some article here Create Multiple Background workers with WebClient-VBForums
but any thread which process the same url.