|
| | HighPerformancePool (size_t num_threads=std::thread::hardware_concurrency()) |
|
| HighPerformancePool (HighPerformancePool const &)=delete |
|
auto | operator= (HighPerformancePool const &) -> HighPerformancePool &=delete |
| template<typename F, typename... Args> |
| auto | submit (F &&f, Args &&... args) -> std::future< std::invoke_result_t< F, Args... > > |
| | High-performance task submission (optimized hot path)
|
| template<typename Iterator> |
| auto | submit_batch (Iterator begin, Iterator end) -> std::vector< std::future< void > > |
| | Batch task submission for maximum throughput.
|
| template<typename Iterator, typename F> |
| void | parallel_for_each (Iterator begin, Iterator end, F &&func) |
| | Optimized parallel for_each with work distribution.
|
| auto | size () const noexcept -> size_t |
| auto | pending_tasks () const -> size_t |
| auto | configure_threads (std::string const &name_prefix, SchedulingPolicy policy=SchedulingPolicy::OTHER, ThreadPriority priority=ThreadPriority::normal()) -> expected< void, std::error_code > |
| | Configure all worker threads.
|
| auto | set_affinity (ThreadAffinity const &affinity) -> expected< void, std::error_code > |
| auto | distribute_across_cpus () -> expected< void, std::error_code > |
| void | wait_for_tasks () |
| void | shutdown () |
| auto | get_statistics () const -> Statistics |
| | Get detailed performance statistics.
|
High-performance thread pool optimized for high-frequency task submission.
Uses a work-stealing architecture: each worker thread owns a private WorkStealingDeque, and idle workers attempt to steal tasks from other workers' queues. A shared overflow queue absorbs bursts when all per-thread queues are full.
Optimizations for 1k+ tasks with 10k+ tasks/second throughput:
- Work-stealing architecture with proper synchronization
- Per-thread queues with efficient load balancing
- Batch processing support for maximum throughput
- Optimized wake-up mechanisms
- Cache-friendly data structures with proper alignment
- Performance monitoring and statistics
- How task execution works
- When you call submit(), the callable is wrapped in a std::packaged_task and placed into one of the per-worker queues (round-robin selection). A condition_variable then wakes one sleeping worker. The worker picks up the task from its own queue. If its own queue is empty, the worker tries to steal tasks from up to 4 other workers' queues (random selection). If no per-worker queue has work, the worker checks the shared overflow queue. If nothing is found at all, the worker sleeps for up to 100 microseconds before retrying.
- Execution guarantees
- Every successfully submitted task (submit() returned without throwing) is guaranteed to eventually execute, as long as the pool is not destroyed while shutdown() is draining.
- submit() throws std::runtime_error if the pool is already shutting down. In that case the task is NOT enqueued and will NOT execute.
- Tasks are executed in approximately FIFO order per queue, but the work-stealing mechanism means that the global execution order across all threads is non-deterministic. There is no ordering guarantee between two tasks submitted from different threads, or even from the same thread if they land in different worker queues.
- The returned std::future becomes ready once the task has completed. You can call future.get() to block until the result is available, or future.wait() to just wait without retrieving the result.
- If a task throws an exception, the exception is stored in the future. Calling future.get() will rethrow it. The worker thread itself continues to run and process further tasks.
- shutdown() sets the stop flag and wakes all workers. Workers finish their current task and then drain all remaining queued tasks before exiting. The destructor calls shutdown() implicitly.
- Thread safety
- submit() and submit_batch() may be called from any thread concurrently. shutdown() is internally guarded and is safe to call more than once.
- Exception handling
- Exceptions thrown by tasks are caught inside the worker loop. They do not propagate to the caller directly, but are stored in the std::future returned by submit(). Call future.get() to observe or rethrow the exception. The worker thread is not affected and continues processing.
- Statistics accuracy
- Counters such as completed_tasks_, stolen_tasks_, and total_task_time_ are updated with std::memory_order_relaxed, so the values returned by get_statistics() are approximate and may lag behind the true counts by a small margin.
- Blocking
- wait_for_tasks() blocks the calling thread until every queued and currently active task has finished.
- Lifetime
- The destructor calls shutdown() and joins all worker threads. It is safe to let the pool go out of scope while tasks are still running; they will be drained first. Note that this means the destructor can block for a long time if tasks are slow.
- Copyability / movability
- Not copyable, not movable.
- Note
- Has overhead for small task counts (< 100 tasks) due to work-stealing complexity. Best for high-throughput scenarios like image processing, batch operations, etc.
Definition at line 242 of file thread_pool.hpp.
1.14.0