How To test If Multiple Backgroundworkers have completed

[curlydog]

Hi,
I'm trying to develop an application which searches a sql server database. One particular search entered by the user actually sends seven queries to the database. I've designed it so each of the seven queries is conducted in a seperate backgroundworker, so that they run and return their hits concurrently. When the user enters a search term, a routine (runSearch) is called , which in turn causes each seperate backgroundworker to run.

This work fine when the user enters a single search term.

I'm now trying to allow the user to enter multiple search terms in a multiline textbox. I iterate through the lines in the textbox and call runsearch for each line. I assumed that the progam would wait for each call to runsearch to complete, before running the next.

I now find out this is not the case. When I call runSearch, the backgroundworkers are set in motion. The next line in the textbox then calls runSearch again, but the backgroundworkers from the previous calling of runSearch are still running. This throws up an error stating that the backgroundworker is still busy and cannot be called.

I need a way to test that all of the backgroundworkers have completed before running runsearch again. If one or more of the backgroundworkers are still running, I need to wait before calling runSearch.

I'd appreciate any pointers as to how to acheive this. I'm fairly new to VB so sample code/examples would be great.

thanks
Jason

 

[jmcilhinney]

That's exactly what the RunWorkerCompleted event is for: to tell you that the background operation is complete.

 

[ignyte87]

There is a backgroundworker.isBusy property that you can use to check if the background worker is currently doing something. You'd probably have to check it for each backgroundworker though. Your other option would be to handle the RunWorkerCompleted event and set some flag to indicate that everything is finished running.

My *suggestion* which allows queries to be done even more multi-threadedly without waiting for limited resources that you have to manage is to use the ThreadPool. Unless you only want to be able to run one search at a time, and don't want the user to be able to start the next search without being finished the previous. This would allow flexibility if you ever needed to send 8 queries to the database, you wouldn't have add an extra backgroundworker reference everywhere it is needed (and then forget one and wonder why everything just broke).

My approach to this would be:

Imports System.Threading

Public Class Test

    Dim runningThreads As Integer = 0
    Dim waitHandle As AutoResetEvent = New AutoResetEvent(False)

Public Sub Sample
        ' Create 10 threads to do whatever. These threads will run the RunThread method, so same thing as BackgroundWorker's DoWork method.
        For i As Integer = 1 To 10
            ThreadPool.QueueUserWorkItem(AddressOf Me.RunThread)
            Threading.Interlocked.Increment(runningThreads)
        Next


        ' Wait for all threads to finish executing.
        waitHandle.WaitOne()

        ' Do whatever.

End Sub

    Private Sub RunThread()

        ' Do something

        Threading.Interlocked.Decrement(runningThreads)
        If runningThreads = 0 Then
            ' Finished all threads. Set a WaitHandle to indicate we are finished.
            waitHandle.Set()
        End If
End Sub
End Class

This is a quick example, I'm sure there are some things that could be done to make it better, but it should get you a start.

The AutoResetEvent waits for a signal that something has happened without you having to do "While (condition) {Thread.Sleep(1000)}" or something similar that wastes CPU cycles.

The ThreadPool has other properties, like MaxThreads, so you can set the number of threads (like backgroundworkers) that are allowed to execute concurrently. I'm not sure...anyone know if the main thread is included in the ThreadPool count?

 

[curlydog]

That's exactly what the RunWorkerCompleted event is for: to tell you that the background operation is complete.

Thanks jmcilhinney for your response. Unfortunately I don't think I was specific enough. I'm aware that the RunWorkerCompleted event fires when a BackGroundWorker is complete. I currently use it to increment a progressbar.

My question was;

I need a way to test that all of the backgroundworkers have completed before running runsearch again. If one or more of the backgroundworkers are still running, I need to wait before calling runSearch.

To be more specific, I perhaps should have asked, how do I check if several BackGroundWorkers have all completed before I run the next search and if they haven't all finished, how do I make my application wait before running the next search.

I thought of using .IsBusy to test each BackGroundWorker prior to running each search. I also considered using a HasCompleted boolean for each BackGroundWorker which I could test before running the next search. Both of these approaches would involve using a very long If statement. I didn't see this as a very elegant way of acheiving what I need.

I was also hoping for some guidance on how to make the next search wait until all BackGroundWorkers have completed.

 

[curlydog]

ignyte87, thanks for your suggestion. Although a number of the some of the concepts you suggest are new to me, this looks like the kind of thing I'm after. I'll see if I can use your example as a basis from which to start.

Thanks again

 

[jmcilhinney]

If you're using multiple BackgroundWorkers then testing multiple IsBusy properties is the correct way to go. The simplest way to implement that would be to put all the BGWs in an array and then call its Any method, e.g.

If Not myBGWArray.Any(Function(bgw) bgw.IsBusy) Then
    'No background tasks are in progress.
End If

 

[curlydog]

I'm now trying to use a combination of the two answers given so far.
I'm the isbusy function to test of any of my backgroundworkers are still running. If so, I'm using an autoresetevent to wait for them all to finish. I'm still having problems with BGWs running concurrently. Here's my code

    Dim waitHandle As System.Threading.AutoResetEvent = New System.Threading.AutoResetEvent(False) 'Create Autosetevent to wait for multiple BGWorkers to finish

.............................................

Dim arrayBGW() = New System.ComponentModel.BackgroundWorker() {BGW1, BGW2, BGW3, BGW4, BGW5, BGW6, BGW7}

For Each stringSearch In arraySearchString
     Debug.Print("frmSearch.butMultipleNumberSearch_Click: searchTerm = " & stringSearch)
     If Not arrayBGW.Any(Function(bgw) bgw.IsBusy) Then
          'No background tasks are in progress.
          Debug.Print("frmSearch.butMultipleNumberSearch_Click: all BGW finished")
           waitHandle.Set()
     Else
           Debug.Print("frmSearch.butMultipleNumberSearch_Click: at least one BGW still busy")
           waitHandle.WaitOne()
     End If
            searchNumber(stringSearch)
     Next

SearchNumber is the routine that calls the RunWorkerAsync method for each of my seven BackGroundWorker.

I have also tried putting the IsBusy test (

 If Not arrayBGW.Any(Function(bgw) bgw.IsBusy) Then

) within the searchNumber routine. but I get the same error.

Any suggestions how I can improve my code?

thanks

 

[jmcilhinney]

You might also like to consider my own BackgroundMultiWorker class:

BackgroundMultiWorker

 

[ignyte87]

I would skip the AutoResetEvent and just stick with IsBusy(). Doing both adds some unnecessary complexity.

So putting this in your SearchNumber doesn't work?

If Not myBGWArray.Any(Function(bgw) bgw.IsBusy) Then
    ' Start all background workers
    For Each bg as BackgroundWorker in myBGWArray
        ' Start each background worker.
        bg.RunWorkerAsync(...)
    Next
End If

If absolutely necessary, reinstantiate the BackgroundWorker (bg = new backgroundworker) which will for sure clear the busy field (and do who knows what to any tasks that are still running...)

Also...you say you are having problems with your background workers *running* concurrently. Does this mean they are running one at a time, or that you are having troubles restarting them (since this post only looks at the latter)?

 

[jmcilhinney]

BackgroundMultiWorker; nice addition, but I would add the DefaultEvent attribute to the class.

Done. I also added a test/demo application, which happens to demonstrate the use of the IsBusy method to determine whether all tasks have completed.

 

[curlydog]

I would skip the AutoResetEvent and just stick with IsBusy(). Doing both adds some unnecessary complexity.

So putting this in your SearchNumber doesn't work?

If Not myBGWArray.Any(Function(bgw) bgw.IsBusy) Then
    ' Start all background workers
    For Each bg as BackgroundWorker in myBGWArray
        ' Start each background worker.
        bg.RunWorkerAsync(...)
    Next
End If

If absolutely necessary, reinstantiate the BackgroundWorker (bg = new backgroundworker) which will for sure clear the busy field (and do who knows what to any tasks that are still running...)

Also...you say you are having problems with your background workers *running* concurrently. Does this mean they are running one at a time, or that you are having troubles restarting them (since this post only looks at the latter)?

Yes, it doesn't seem to matter if I place this code within SearchNumber, or in the method that calls SearchNumber, the results tend to be the same.

Essentially the first search runs and before it has finished, the second search starts to run. This means that a backgroundworker is called before the same backgroundworker from the previous search has finished. I then get the error.

I also tried placing the IsBusy test with a Do While loop to try and make my calling routine loop until all of the BackGroundWorkers have finished. This didn't work either. The first search started, before it finished the second search started and then loop just continued running.

My latest approach incoporates an integer value (numWorkers_Completed) I use to monitor how many of the background workers have completed. My theory being that each time I call SearchNumber, 7 BackGroundWorkers are set running. If the user enters multiple searches, then when numWorkers_Completed is a multiple of 7, a search has finished and hence all of the backgroundworker should be finished. To this end I used the following code in the routine which calle SearchNumber

 For Each stringSearch In arraySearchString
      Debug.Print("frmSearch.butMultipleNumberSearch_Click: searchTerm = " & stringSearch)
      searchNumber(stringSearch)
      Do While Not numWorkers_Completed Mod 7 = 0
           Debug.Print("frmSearch.butMultipleNumberSearch_Click: mod<>0")
      Loop
           Debug.Print("frmSearch.butMultipleNumberSearch_Click: mod=0")
      Next

I get a similar result to when I use the IsBusy test in a loop. Essentially it appears that first search runs and before it has finished, the second is called.

    You might also like to consider my own BackgroundMultiWorker class:

    BackgroundMultiWorker

Thanks for your suggestion. I've taken a look at you post. Unfortunately my inexperience doesn't allow me to fully understand what it does, so at the momment I'm not sure a) if it would help me and b) how I would use it.

I may have to revisit it however if I can't make it work in my basic manner.

Thanks again

 

[jmcilhinney]

Thanks for your suggestion. I've taken a look at you post. Unfortunately my inexperience doesn't allow me to fully understand what it does, so at the momment I'm not sure a) if it would help me and b) how I would use it.

It does what the BackgroundWorker does except it lets you execute multiple background tasks with a single instance, instead of one instance per task. In your case, you would create a single BackgroundMultiWorker object, call RunWorkerAsync multiple times and then just call the IsBusy method. It will return True if one or more tasks are running and False otherwise.

You would basically use it exactly like a regular BackgroundWorker with the exception that you would have to pass in a token to identify the task when calling RunWorkerAsync. Unless you have some object already that uniquely identifies a task, I suggest using a Guid. You can check out the demo/test app I included in the same solution. Your need the 2010 version of VS or VB Express to run it. If you already have 2008, you can install both side by side.

 

[samuelg78]

I had tried using your example. But how do i pass a class into the worker? I tried but failed,
the value is always 15.

Dim rc as new raceclass

For i = 1 To 15
rc.rcNo = i
'Generate a random token to identify each task.
Me.worker.RunWorkerAsync(Guid.NewGuid(), rc)
Next

Private Sub worker_DoWork(sender As Object, e As DoWorkEventArgs) Handles worker.DoWork
Dim rc As New raceclass
rc = DirectCast(e.Argument, raceclass)

'Get the token that identifies this task.
Dim token = DirectCast(e.Token, Guid)

'Always print "15"
console.writeline(rc.rcNo)
End Sub

 

[jmcilhinney]

That "issue" is nothing to do with the BackgroundMultiWorker. You are passing the data into the background task correctly. The reason you see 15 every time is that you only have one 'raceclass' object and your For loop has completed by the time any of the DoWork event handlers are executed. Think of it this way. Let's say that I put a red shirt on you and tell you to go visit Peter, then I put a green shirt on you and tell you to go visit Paul, then I put a blue shirt on you and tell you to go visit Mary. What colour shirt will Peter, Paul and Mary see you wearing? All blue, right? Same thing.

 

[samuelg78]

That "issue" is nothing to do with the BackgroundMultiWorker. You are passing the data into the background task correctly. The reason you see 15 every time is that you only have one 'raceclass' object and your For loop has completed by the time any of the DoWork event handlers are executed. Think of it this way. Let's say that I put a red shirt on you and tell you to go visit Peter, then I put a green shirt on you and tell you to go visit Paul, then I put a blue shirt on you and tell you to go visit Mary. What colour shirt will Peter, Paul and Mary see you wearing? All blue, right? Same thing.

Thks for the reply, so i think i need a 'raceclass' array to store the values. Pls kindly advise.