Question Background worker slow.

[mbb]

Hello,

I have written some code to generate about 100,000 amount of objects on to a list collection.

When run in the application's UI thread this was a very fast process - at most 2 seconds.

I moved the generation code into a background worker and now it is running considerably slower. About 1 minute.

I also have save and load mechanisms using serialisation which are even slower when run in the background worker.

So. Is this a problem with thread priorities, which I cannot affect using the backgroundworker class?

Or is there something more complicated with threading e.g. overhead for operating on objects that aren't owned by the thread etc

Thanks.

 

[JohnH]

The only thing I can think of from the information you have provided is that you're using Control.Invoke, using Control.BeginInvoke instead will not wait and block the worker thread for the completion of that call, or you can use the BW.ReportProgress method. I can't sense that you're using other synchronization features that could slow things.

 

[mbb]

The only thing I can think of from the information you have provided is that you're using Control.Invoke, using Control.BeginInvoke instead will not wait and block the worker thread for the completion of that call, or you can use the BW.ReportProgress method. I can't sense that you're using other synchronization features that could slow things.

I don't know if your answer is relevant since I am using the RunWorkerAsync() method to start the worker. Here is the code:

Imports System.ComponentModel
Imports Common
Imports CommonDataLayer
Imports CommonUserInterface

Public Class GenerateGameTask
    Implements ITask

    Private _BackgroundWorker As BackgroundWorker = New BackgroundWorker

    ' Some stats for the generation stats
    Private _TotalRecords As Integer = 0
    Private _UpdateFactor As Integer = 0
    Private _GeneratedRecords As Integer = 0

    Public Sub New()

        _BackgroundWorker.WorkerReportsProgress = True

        ' Hook up the background worker to the work method
        AddHandler _BackgroundWorker.DoWork, AddressOf DoWork
        AddHandler _BackgroundWorker.RunWorkerCompleted, AddressOf WorkDone
        AddHandler _BackgroundWorker.ProgressChanged, AddressOf ProgressChanged
    End Sub

#Region "ITask Interface"
    Public Event Progress(ByVal pMessage As String, ByVal pProgress As Integer) Implements ITask.Progress
    Public Event Done() Implements ITask.Done

    Public Function StartWork() As Boolean Implements ITask.StartWork

        If Not _BackgroundWorker.IsBusy Then
            _BackgroundWorker.RunWorkerAsync()
        End If
    End Function
#End Region

    Private Function DoWork() As Boolean

        ' Hook up to the table index events
        AddHandler TableIndex.Instance.Progress, AddressOf HandleTableIndexProgress

        ' Make sure the game tables are initialised
        TableIndex.Instance.InitialiseGame()

        ' The work goes here
        CreateGame()

        ProgressChanged(Me, New ProgressChangedEventArgs(100, "Game generation complete"))

        Return True
    End Function

    Private Function HandleTableIndexProgress(ByVal sender As Object, ByVal e As TableIndexEventArgs) As Boolean
        RaiseEvent Progress(e.Message, e.Progress)
    End Function

    Private Function ProgressChanged(ByVal sender As Object, ByVal e As ProgressChangedEventArgs) As Boolean
        RaiseEvent Progress(e.UserState, e.ProgressPercentage)
    End Function

    Private Function WorkDone(ByVal sender As Object, ByVal e As RunWorkerCompletedEventArgs) As Boolean
        RaiseEvent Done()
    End Function

#Region "Generation Routines"
    Private Function CreateGame() As Boolean

        ' Set up some geneation parameters
        Dim generationParameters As GenerationParameters = New GenerationParameters()

        ' Calculate the predicted totals for the generation
        _TotalRecords = TableIndex.Instance.OrganisationTable.TotalRanksPerWeightClass * TableIndex.Instance.WeightClassTable.Count

        ' Generate the regional and continental organisations
        GenerateOrganisations()

        ' Generate the boxers in the game
        GenerateBoxerPool()

        Return True
    End Function

    ''' <summary>
    ''' This will count the number of records as they are generated.
    ''' For every 100th of the overall progress, the progress bar will be updated.
    ''' </summary>
    ''' <param name="pMessage">The given message</param>
    Private Function UpdateProgress(ByVal pMessage As String) As Boolean
        _GeneratedRecords += 1
        _UpdateFactor += 1

        If _UpdateFactor >= (_TotalRecords / 100) Then
            _UpdateFactor = 0
            ProgressChanged(Me, New ProgressChangedEventArgs((_GeneratedRecords / _TotalRecords * 100), pMessage))
        End If
    End Function

    Private Function GenerateOrganisations() As Boolean

        ' Generate an organisation for each region
        For Each country As Country In TableIndex.Instance.CountryTable

            ' Id of zero is being used for an unknown country
            If country.Id = 0 Then
                Continue For
            End If

            UpdateProgress("Generating organisations")
            TableIndex.Instance.OrganisationTable.Add(New Organisation(country.Name, OrganisationTypeEnum.Professional, OrganisationLevelEnum.National, country.Id, 30))
        Next

        For Each continent As Continent In TableIndex.Instance.ContinentTable

            ' Id of zero is being used for an unknown continent
            If continent.Id = 0 Then
                Continue For
            End If

            UpdateProgress("Generating organisations")
            TableIndex.Instance.OrganisationTable.Add(New Organisation(continent.Name, OrganisationTypeEnum.Professional, OrganisationLevelEnum.Continental, continent.Id, 60))
        Next
    End Function

    Private Function GenerateBoxerPool() As Boolean
        Dim generationParameters As GenerationParameters = New GenerationParameters()

        UpdateProgress("Generating characters")

        ' Generate boxers for each national organisation
        For Each organisation As Organisation In TableIndex.Instance.OrganisationTable

            ' Ignore continental and international organisations
            If organisation.OrganisationLevelId <> OrganisationLevelEnum.National Then
                Continue For
            End If

            ' Generate boxers for each weight class
            For Each weightClass As WeightClass In TableIndex.Instance.WeightClassTable

                ' Generate one boxer per rank
                For count = 1 To organisation.RanksPerWeightClass

                    ' Preset the generation parameters
                    generationParameters.PresetBoxer(RandomRoll.Roll(0, (GenerationBoxerPresetEnum.Count - 1)))

                    ' Make sure boxer attributes are generated
                    generationParameters.SetParameter(New GenerationParameter(Of Boolean)(GenerationParameterNameEnum.GenerateAttributes, True))

                    ' Assign the country and weight class
                    generationParameters.SetParameter(New GenerationParameter(Of Country)(GenerationParameterNameEnum.AssignCountry, TableIndex.Instance.CountryTable.Row(organisation.ParentId)))
                    generationParameters.SetParameter(New GenerationParameter(Of WeightClass)(GenerationParameterNameEnum.AssignWeightClass, weightClass))

                    Dim newCharacter As BoxerCharacter = New BoxerCharacter()
                    TableIndex.Instance.CharacterTable.Add(newCharacter)
                    newCharacter.Generate(generationParameters)
                Next
            Next
        Next
    End Function
#End Region
End Class

Here are some metrics:

2 International organisations with 100 boxers
5 Continental organisations with 60 boxers
191 National organisations with 30 boxers
17 weight classes

That makes:

2 * 100 = 200
5 * 60 = 300 +
191 * 30 = 5730 = 6230 boxers per weight class.

A total of 105910 boxers.

I've tried disabling progress reports, without any real benefit. Hopefully I'll get to tinker with this some more later.

Thanks.

 

[JohnH]

Use BW.ReportProgress method, the BW events are raised in the thread that created the BW component. Your current code raises events in worker thread.

Apart from that I can't see any code that would behave differently run in one thread or another.

 

[mbb]

Use BW.ReportProgress method, the BW events are raised in the thread that created the BW component. Your current code raises events in worker thread.

Apart from that I can't see any code that would behave differently run in one thread or another.

Thanks for your replies. Turns out the performance issues were nothing to do with the use of a background worker, but rather my code that was running in the body. It took me a a while winkle out the bottlenecks ... cut a long story short: using the standard .NET routines to serialise and deserialise objects within objects can be quite expensive.

Is there anything wrong in raising events in the worker thread?

 

[JohnH]

Is there anything wrong in raising events in the worker thread?

If the handler of these events are to interact with controls in UI thread this means consumer have to invoke to UI thread when receiving the notification. Such events are usually called "asynchronous event" and must be documented for consumers to be able to handle them appropriately. When it is natural that the class/component is to be used in a UI environment it is common to provide some kind of synchronization feature and raise the event in UI thread when specified, look at the Timers.Timer class and its SynchronizingObject property for example. The thing with the BackgroundWorker is that is raises its events in the thread that created the BW instance, this makes its synchronization behaviour easy to handle both for consumer and designer of class.

 

[mbb]

If the handler of these events are to interact with controls in UI thread this means consumer have to invoke to UI thread when receiving the notification.

Yay, have already come across this problem in a couple of guises. I'm tending to cut to the chase in my UI's and just code a delgate/invoke regardless of whether an invoke is required or not.

Such events are usually called "asynchronous event" and must be documented for consumers to be able to handle them appropriately. When it is natural that the class/component is to be used in a UI environment it is common to provide some kind of synchronization feature and raise the event in UI thread when specified, look at the Timers.Timer class and its SynchronizingObject property for example. The thing with the BackgroundWorker is that is raises its events in the thread that created the BW instance, this makes its synchronization behaviour easy to handle both for consumer and designer of class.

This is tricky to grasp when the MSDN descriptions are so limited, but I finally got the idea that I can use the SynchronizingObject to "pull" the events onto my UI thread by setting the property to my form. Right?

Problem is that because the way my application is structured the form is not known to my controller - where the background job is created - so I can't marshall onto the form's UI in quite that way.

Hmm ... thanks. I'm going to have to have a think about this to sort out the mess. I've a feeling my implementation of MVC is not correct.