Implementing an efficient worker thread pool in Windows can be achieved through the use of IOCP, which is the barbaric acronym for (just as barbaric) Input/Output Completion Port API introduced by Microsoft a while ago.
To make use of IOCP, one has to deal with further barbaric API functions, but the principles are (somewhat) more civilized, and the implementation is quite straightforward.
What are worker threads? They’re threads that don’t have any specific task to perform, but are designed to execute arbitrary “work units” in a threaded, asynchronous fashion.
Usually they come in pools, and you just give work units to the pool. The work units are executed in order by the workers that have nothing else to do.
This is a very simple model in which works threading aspects are abstracted and become just asynchronous work units. It also has very similar real-world analogies, and can often be easier to understand than other forms of multi-threading parallelization.
Implementation-wise, it revolves around a FIFO queue, that has to be thread-safe, along with thread signaling events. This is a mechanism IOCP wraps with just three API functions, with OS kernel support.
Setting up an IOCP queue
To use IOCP the first thing to do is create an IOCP queue, this is accomplished through the CreateIoCompletionPort function. This function was initially meant for I/O, but it can use any object handle, even INVALID_FILE_HANDLE.
FIOCP := CreateIoCompletionPort(INVALID_FILE_HANDLE, 0, 0, 0);
And that’s all you need to setup an IOCP queue.
Posting work to the queue
To post work you have use the PostQueuedCompletionStatus function, it takes four arguments:
- the first is the IOCP handle, which we obtained above
- lpNumberOfBytesTransferred (showing its I/O roots), an integer
- lpCompletionKey, an integer
- lpOverlapped, a pointer
And that’s all. The good thing is that for a queue, they’re not significant, so in 32 bits, that’s effectively 12 bytes we can use, and double that in 64 bits.
One of the simpler ways to define a work unit would be to use an anonymous method (reference to procedure). In Delphi, those are in practice just hidden interfaces, so they fit in a pointer. Let’s queue one:
type TAnonymousWorkUnit = reference to procedure; PAnonymousWorkUnit = ^TAnonymousWorkUnit; ... procedure TWorkerThreadPool.QueueWork(const workUnit : TAnonymousWorkUnit); var lpOverlapped : Pointer; begin lpOverlapped := nil; PAnonymousWorkUnit(@lpOverlapped)^ := workUnit; PostQueuedCompletionStatus(FIOCP, 1, 0, lpOverlapped); end;
In the code above we copy the workUnit into a pointer, it’s important to use a copy and not a cast so as to increment the reference count of the interface aka anonymous method. The destination pointer also explicitly be nil’ed before the assignment (as the compiler won’t initialize it, so it can contain just about anything).
The we post to the queue, using lpNumberOfBytesTransferred to pass a command (1) which will come in use below.