Insert query in multithreaded application

[Rockstar]

The application im working on is meant for processing large text files and posting them into a mysql database. I have it posting at a rate of about 2000per minute on a single thread but what im wondering is how to get it to do this:

thread1 = 1
thread2 = 2
thread3 = 3
thread1 = 4
thread2 = 5
thread3 = 6

rinse repeat til end of id count(*)

what its currently inserting. its in the threading not the mysql.

thread1 = 1
thread2 = 1
thread3 = 1
thread1 = 2
thread2 = 2
thread3 = 2


text file is delimited field1;field2.

 

[jmcilhinney]

We have no idea where those numbers are coming from in the first place so how can we tell you how to make it work differently? You're going to have to provide a full and complete explanation of what you're doing and what you want to do. Numbers mean nothing if we have no idea how they are generated.

 

[Rockstar]

edit: sorry if this sounds really jumpy ive been workin on this tool for about 4 days now and slept about 4 hours since, so please bear with me for lack of information above.

---

what i was doing was calling 3 threads with threadx.start() from 1 button and they were all processing the file "accounts.txt" seperately. The reason i wanted them sync'd is because i couldve done about 6k a minute with 3 threads and from there scaled it to a higher amount for faster processing.

however, since im working under mysql, and i was unaware that mysql connector for .net included the documentation/help files in the nonexistant start-menu folder(program files/mysql) i had to dig around til i came across horribly documented MySqlBulkLoader class.

so instead of parsing my file myself, and then passing data to the query and looping, and all that good stuff nobody really wants to reinvent the wheel with - i worked til i got that bastard mysqlbulkloader working.

however, this nifty tool DOES dump the text file into an array but im not sure how to output that array as indexed values myself, it does it for you as array(0) & array(1) . so the database.table rows must be setup

array(1) array(2) id

and it will place them correctly.

my method did about 2k rows a minute after the connection was made.
mysqlbulkloader i can insert about 500k-600k rows a minute..

with just (grab your pants whoadies the spaceship is landing)

    Public Shared Sub loadaccts(ByVal dbconn As MySqlConnection)
        Dim conn As MySqlConnection
        Dim loader As MySqlBulkLoader
        Dim connstr As String
        connstr = "host=" & Form1.sqlhost.Text & ";user=" & Form1.sqluser.Text & ";password=" & Form1.sqlpass.Text & ";database=mailer"
        conn = New MySqlConnection(connstr)
        loader = New MySqlBulkLoader(conn)
        loader.TableName = "accounts"
        loader.FieldTerminator = ";"
        loader.FileName = "C:\users\clam spammin\desktop\prereqs\members.sql"
        loader.NumberOfLinesToSkip = 0
        loader.Load()
    End Sub