Oh ConcurrentQueue, where have you been all my life! - Why have I only just found out that you exist?
I know, I know… It seems I’m late to the party.
But it turns out that the new System.Collections.Concurrent namespace was only introduced in .NET Version 4.0
It’s no secret that when it comes to data processing, adding more threads mean you can process data faster. But adding multiple threads means data races. It means you need to make sure you have a mutex/lock setup around the shared data structure. This ensures that all access to this central data is tightly controlled. Essentially, it means only one thread at any one time should be able to access the data.
Now, I have previously rolled my own thread-safe locking queue which is always a scary concept (due to data-race conditions). I have used it in multiple projects and have many variations.
I have written a Blocking Queue which doesn’t spin on a thread. Essentially all threads will wait on a queue instead of having to Thread.Sleep(). I have also written a thread-safe database writer, specific to the database schema needs, where worker threads will pick up work of a certain type (WorkType), then write it to the database in the order it appears in the queue (important). If another thread wakes up and starts taking work from the master work queue it ignores that WorkType as it is being dealt with by the previous worker thread. This means there needs to be intercommunication between threads. It’s tricky code to get right.
Why do I mention all this?
Because these new thread-safe collection classes could simplify a lot of my current code!
I work on many producer/consumer problems where you can have many multiple producers of work and need many consumers trying to keep up! I have a set of utility classes that I bring along with me to every new project.
But the next time I am working on a project, perhaps it’s time to upgrade to .NET 4.0, then refactor my utility classes to use the classes from the new System.Collections.Concurrent namespace.
