High-performance thread pool optimized for high-frequency task submission. More...

#include <work_stealing_pool_backend.hpp>

Classes
struct	statistics

Public Types
using	task_type = std::function< void()>

using	queued_task = detail::move_only_function< void()>

Public Member Functions
	work_stealing_pool_backend (size_t num_threads=default_worker_count(), size_t deque_capacity=work_stealing_deque< queued_task >::default_capacity, bool register_workers=false)

template<typename Bool , std::enable_if_t< std::is_same_v< std::decay_t< Bool >, bool >, int > = 0>
	work_stealing_pool_backend (size_t num_threads, Bool register_workers)

	work_stealing_pool_backend (work_stealing_pool_backend const &)=delete

auto	operator= (work_stealing_pool_backend const &) -> work_stealing_pool_backend &=delete

	~work_stealing_pool_backend ()

void	shutdown (shutdown_policy_backend policy=shutdown_policy_backend::drain)
	Shut the pool down.

auto	shutdown_for (std::chrono::milliseconds timeout) -> bool
	Attempt a timed drain: finish as many tasks as possible within `timeout`, then discard queued work.

template<typename F , typename... Args>
auto	try_submit (F &&f, Args &&... args) -> expected< std::future< bind_result_t< F, Args... > >, std::error_code >
	Submit a task without throwing on shutdown.

template<typename F , typename... Args>
auto	submit (F &&f, Args &&... args) -> std::future< bind_result_t< F, Args... > >
	Submit a task, throwing on shutdown.

template<typename F , typename... Args>
void	post (F &&f, Args &&... args)
	Fire-and-forget task submission (throwing variant).

template<typename F , typename... Args>
auto	try_post (F &&f, Args &&... args) -> expected< void, std::error_code >
	Fire-and-forget task submission (non-throwing variant).

template<typename Iterator >
auto	try_submit_batch (Iterator begin, Iterator end) -> expected< std::vector< std::future< void > >, std::error_code >
	Submit a range of `void()` callables in one go (non-throwing).

template<typename Iterator >
auto	submit_batch (Iterator begin, Iterator end) -> std::vector< std::future< void > >
	Submit a range of `void()` callables in one go (throwing).

template<typename Iterator , typename F >
void	parallel_for_each (Iterator begin, Iterator end, F &&func)
	Apply `func` to every element in [begin, end) in parallel.

Observers
auto	size () const noexcept -> size_t
	Number of worker threads in this pool.

auto	pending_tasks () const -> size_t
	Approximate count of tasks waiting in all queues.

auto	get_statistics () const -> statistics
	Collect approximate performance counters.

Thread configuration
auto	configure_threads (std::string const &name_prefix, native_scheduling_policy policy=native_scheduling_policy::other, native_thread_priority priority=native_thread_priority::normal()) -> expected< void, std::error_code >
	Name, schedule and prioritize all worker threads.

auto	configure_threads (native_thread_config const &config) -> expected< void, std::error_code >

auto	set_affinity (native_thread_affinity const &affinity) -> expected< void, std::error_code >
	Pin all workers to the same CPU set.

auto	distribute_across_cpus () -> expected< void, std::error_code >
	Pin each worker to a distinct CPU core (round-robin).

Synchronisation
void	wait_for_tasks ()
	Block until all pending and active tasks have completed.

auto	is_current_worker () const noexcept -> bool

Tracing hooks
void	set_on_task_start (task_start_callback cb)
	Register a callback invoked just before each task executes.

template<typename Callback , std::enable_if_t<!std::is_same_v< detail::remove_cvref_t< Callback >, task_start_callback >, int > = 0>
void	set_on_task_start (Callback &&cb)

void	set_on_task_end (task_end_callback cb)
	Register a callback invoked just after each task completes.

template<typename Callback , std::enable_if_t<!std::is_same_v< detail::remove_cvref_t< Callback >, task_end_callback >, int > = 0>
void	set_on_task_end (Callback &&cb)

Detailed Description

High-performance thread pool optimized for high-frequency task submission.

Uses a work-stealing architecture: each worker thread owns a private work_stealing_deque, and idle workers attempt to steal tasks from other workers' queues. A shared overflow queue absorbs bursts when all per-thread queues are full.

Optimizations for 1k+ tasks with 10k+ tasks/second throughput:

Work-stealing architecture with proper synchronization
Per-thread queues with efficient load balancing
Batch processing support for maximum throughput
Optimized wake-up mechanisms
Cache-friendly data structures with proper alignment
Performance monitoring and statistics

How task execution works: When you call submit(), the callable is wrapped in a std::packaged_task and placed into one of the per-worker queues (round-robin selection). A condition_variable then wakes one sleeping worker. The worker picks up the task from its own queue. If its own queue is empty, the worker tries to steal tasks from up to 4 other workers' queues (random selection). If no per-worker queue has work, the worker checks the shared overflow queue. If nothing is found at all, the worker sleeps for up to 100 microseconds before retrying.

Execution guarantees

Every successfully submitted task (submit() returned without throwing) is guaranteed to eventually execute, as long as the pool is not destroyed while shutdown() is draining.
submit() throws std::runtime_error if the pool is already shutting down. In that case the task is NOT enqueued and will NOT execute.
Tasks are executed in approximately FIFO order per queue, but the work-stealing mechanism means that the global execution order across all threads is non-deterministic. There is no ordering guarantee between two tasks submitted from different threads, or even from the same thread if they land in different worker queues.
The returned std::future becomes ready once the task has completed. You can call future.get() to block until the result is available, or future.wait() to just wait without retrieving the result.
If a task throws an exception, the exception is stored in the future. Calling future.get() will rethrow it. The worker thread itself continues to run and process further tasks.
shutdown() sets the stop flag and wakes all workers. Workers finish their current task and then drain all remaining queued tasks before exiting. The destructor calls shutdown() implicitly.

Thread safety: submit() and submit_batch() may be called from any thread concurrently. shutdown() is internally guarded and is safe to call more than once.

Exception handling: Exceptions thrown by tasks are caught inside the worker loop. They do not propagate to the caller directly, but are stored in the std::future returned by submit(). Call future.get() to observe or rethrow the exception. The worker thread is not affected and continues processing.

statistics accuracy: Counters such as completed_tasks_, stolen_tasks_, and total_task_time_ are updated with std::memory_order_relaxed, so the values returned by get_statistics() are approximate and may lag behind the true counts by a small margin.

Blocking: wait_for_tasks() blocks the calling thread until every queued and currently active task has finished.

Lifetime: The destructor calls shutdown() and joins all worker threads. It is safe to let the pool go out of scope while tasks are still running; they will be drained first. Note that this means the destructor can block for a long time if tasks are slow.

Copyability / movability: Not copyable, not movable.

Note: Has overhead for small task counts (< 100 tasks) due to work-stealing complexity. Best for high-throughput scenarios like image processing, batch operations, etc.

Definition at line 122 of file work_stealing_pool_backend.hpp.

Member Typedef Documentation

◆ queued_task

using threadschedule::detail::work_stealing_pool_backend::queued_task = detail::move_only_function<void()>

Definition at line 126 of file work_stealing_pool_backend.hpp.

◆ task_type

using threadschedule::detail::work_stealing_pool_backend::task_type = std::function<void()>

Definition at line 125 of file work_stealing_pool_backend.hpp.

Constructor & Destructor Documentation

◆ work_stealing_pool_backend() [1/3]

threadschedule::detail::work_stealing_pool_backend::work_stealing_pool_backend	(	size_t	num_threads = `default_worker_count()`,
		size_t	deque_capacity = `work_stealing_deque<queued_task>::default_capacity`,
		bool	register_workers = `false`
	)

inlineexplicit

Definition at line 139 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::worker_startup_latch::wait().

◆ work_stealing_pool_backend() [2/3]

template<typename Bool , std::enable_if_t< std::is_same_v< std::decay_t< Bool >, bool >, int > = 0>

threadschedule::detail::work_stealing_pool_backend::work_stealing_pool_backend	(	size_t	num_threads,
		Bool	register_workers
	)

inline

Definition at line 172 of file work_stealing_pool_backend.hpp.

◆ work_stealing_pool_backend() [3/3]

threadschedule::detail::work_stealing_pool_backend::work_stealing_pool_backend ( work_stealing_pool_backend const & )

delete

◆ ~work_stealing_pool_backend()

threadschedule::detail::work_stealing_pool_backend::~work_stealing_pool_backend ( )

inline

Definition at line 180 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::drain, and shutdown().

Member Function Documentation

◆ configure_threads() [1/2]

auto threadschedule::detail::work_stealing_pool_backend::configure_threads ( native_thread_config const & config ) -> expected<void, std::error_code>

inline

Definition at line 627 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::configure_worker_threads(), and threadschedule::detail::runtime_registry().

◆ configure_threads() [2/2]

auto threadschedule::detail::work_stealing_pool_backend::configure_threads	(	std::string const &	name_prefix,
		native_scheduling_policy	policy = `native_scheduling_policy::other`,
		native_thread_priority	priority = `native_thread_priority::normal()`
	)		-> expected<void, std::error_code>

inline

Name, schedule and prioritize all worker threads.

Each worker is named name_prefix + "_0", "_1", etc.

Returns: expected<void, std::error_code> - error if the OS rejected any configuration call.

Definition at line 618 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::configure_worker_threads(), and threadschedule::detail::runtime_registry().

◆ distribute_across_cpus()

auto threadschedule::detail::work_stealing_pool_backend::distribute_across_cpus ( ) -> expected<void, std::error_code>

inline

Pin each worker to a distinct CPU core (round-robin).

Definition at line 641 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::distribute_workers_across_cpus().

◆ get_statistics()

auto threadschedule::detail::work_stealing_pool_backend::get_statistics ( ) const -> statistics

inline

Collect approximate performance counters.

Definition at line 570 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::work_stealing_pool_backend::statistics::active_threads, threadschedule::detail::work_stealing_pool_backend::statistics::avg_task_time, threadschedule::detail::work_stealing_pool_backend::statistics::completed_tasks, threadschedule::detail::work_stealing_pool_backend::statistics::pending_tasks, pending_tasks(), threadschedule::detail::work_stealing_pool_backend::statistics::stolen_tasks, threadschedule::detail::work_stealing_pool_backend::statistics::tasks_per_second, and threadschedule::detail::work_stealing_pool_backend::statistics::total_threads.

◆ is_current_worker()

auto threadschedule::detail::work_stealing_pool_backend::is_current_worker ( ) const -> bool

inlinenoexcept

Definition at line 662 of file work_stealing_pool_backend.hpp.

Referenced by parallel_for_each(), shutdown(), shutdown_for(), and wait_for_tasks().

◆ operator=()

auto threadschedule::detail::work_stealing_pool_backend::operator= ( work_stealing_pool_backend const & ) -> work_stealing_pool_backend &=delete

delete

◆ parallel_for_each()

template<typename Iterator , typename F >

void threadschedule::detail::work_stealing_pool_backend::parallel_for_each	(	Iterator	begin,
		Iterator	end,
		F &&	func
	)

inline

Apply func to every element in [begin, end) in parallel.

The range is split into chunks and submitted as tasks. Blocks until all elements have been processed.

Template Parameters

Iterator Forward iterator; task chunks retain iterator pairs.

Definition at line 536 of file work_stealing_pool_backend.hpp.

References is_current_worker(), threadschedule::detail::parallel_for_each_chunked(), and threadschedule::detail::throw_worker_deadlock().

◆ pending_tasks()

auto threadschedule::detail::work_stealing_pool_backend::pending_tasks ( ) const -> size_t

inline

Approximate count of tasks waiting in all queues.

Definition at line 555 of file work_stealing_pool_backend.hpp.

Referenced by get_statistics().

◆ post()

template<typename F , typename... Args>

void threadschedule::detail::work_stealing_pool_backend::post	(	F &&	f,
		Args &&...	args
	)

inline

Fire-and-forget task submission (throwing variant).

Enqueues a callable without creating a std::packaged_task or std::future, giving roughly 3x higher throughput than submit() for tasks whose return value is not needed.

Exceptions

std::runtime_error If the pool is shutting down.

See also: try_post() for the non-throwing variant.

Definition at line 370 of file work_stealing_pool_backend.hpp.

References try_post().

◆ set_affinity()

auto threadschedule::detail::work_stealing_pool_backend::set_affinity ( native_thread_affinity const & affinity ) -> expected<void, std::error_code>

inline

Pin all workers to the same CPU set.

Definition at line 634 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::set_worker_affinity().

◆ set_on_task_end() [1/2]

template<typename Callback , std::enable_if_t<!std::is_same_v< detail::remove_cvref_t< Callback >, task_end_callback >, int > = 0>

void threadschedule::detail::work_stealing_pool_backend::set_on_task_end ( Callback && cb )

inline

Definition at line 710 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::make_copyable_function().

◆ set_on_task_end() [2/2]

void threadschedule::detail::work_stealing_pool_backend::set_on_task_end ( task_end_callback cb )

inline

Register a callback invoked just after each task completes.

Parameters

cb	Receives the end time, the worker's `std::thread::id`, and the wall-clock duration of the task.

Definition at line 701 of file work_stealing_pool_backend.hpp.

◆ set_on_task_start() [1/2]

template<typename Callback , std::enable_if_t<!std::is_same_v< detail::remove_cvref_t< Callback >, task_start_callback >, int > = 0>

void threadschedule::detail::work_stealing_pool_backend::set_on_task_start ( Callback && cb )

inline

Definition at line 686 of file work_stealing_pool_backend.hpp.

◆ set_on_task_start() [2/2]

void threadschedule::detail::work_stealing_pool_backend::set_on_task_start ( task_start_callback cb )

inline

Register a callback invoked just before each task executes.

Parameters

cb	Receives the start time and the worker's `std::thread::id`.

Definition at line 677 of file work_stealing_pool_backend.hpp.

◆ shutdown()

void threadschedule::detail::work_stealing_pool_backend::shutdown ( shutdown_policy_backend policy = shutdown_policy_backend::drain )

inline

Shut the pool down.

Parameters

policy drain (default) finishes all queued tasks; drop_pending discards queued tasks.

Definition at line 192 of file work_stealing_pool_backend.hpp.

References is_current_worker(), and threadschedule::detail::throw_worker_deadlock().

Referenced by ~work_stealing_pool_backend().

◆ shutdown_for()

auto threadschedule::detail::work_stealing_pool_backend::shutdown_for ( std::chrono::milliseconds timeout ) -> bool

inline

Attempt a timed drain: finish as many tasks as possible within timeout, then discard queued work.

New submissions are rejected before the timed wait begins. Running C++ callables cannot be stopped safely, so this function can return after the timeout while an already-running task finishes.

Returns: true if all tasks completed within the deadline, false if the timeout expired first.

Definition at line 216 of file work_stealing_pool_backend.hpp.

References is_current_worker(), threadschedule::detail::shutdown_deadline_after(), and threadschedule::detail::throw_worker_deadlock().

◆ size()

auto threadschedule::detail::work_stealing_pool_backend::size ( ) const -> size_t

inlinenoexcept

Number of worker threads in this pool.

Definition at line 548 of file work_stealing_pool_backend.hpp.

◆ submit()

template<typename F , typename... Args>

auto threadschedule::detail::work_stealing_pool_backend::submit	(	F &&	f,
		Args &&...	args
	)		-> std::future<bind_result_t<F, Args...>>

inline

Submit a task, throwing on shutdown.

Equivalent to try_submit but throws std::runtime_error instead of returning an error code when the pool is shutting down.

Exceptions

std::runtime_error If the pool is shutting down.

Returns: std::future<R> that becomes ready when the task completes.

Definition at line 350 of file work_stealing_pool_backend.hpp.

References threadschedule::expected< T, E >::has_value(), try_submit(), and threadschedule::expected< T, E >::value().

◆ submit_batch()

template<typename Iterator >

auto threadschedule::detail::work_stealing_pool_backend::submit_batch	(	Iterator	begin,
		Iterator	end
	)		-> std::vector<std::future<void>>

inline

Submit a range of void() callables in one go (throwing).

Exceptions

std::runtime_error If the pool is shutting down.

See also: try_submit_batch() for the non-throwing variant.

Definition at line 519 of file work_stealing_pool_backend.hpp.

References threadschedule::expected< T, E >::has_value(), try_submit_batch(), and threadschedule::expected< T, E >::value().

◆ try_post()

template<typename F , typename... Args>

auto threadschedule::detail::work_stealing_pool_backend::try_post	(	F &&	f,
		Args &&...	args
	)		-> expected<void, std::error_code>

inline

Fire-and-forget task submission (non-throwing variant).

Returns: expected<void, std::error_code> – std::errc::operation_canceled on shutdown.

Definition at line 385 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::bind_args(), and threadschedule::detail::make_move_only_function().

Referenced by post().

◆ try_submit()

template<typename F , typename... Args>

auto threadschedule::detail::work_stealing_pool_backend::try_submit	(	F &&	f,
		Args &&...	args
	)		-> expected<std::future<bind_result_t<F, Args...>>, std::error_code>

inline

Submit a task without throwing on shutdown.

Wraps the callable in a std::packaged_task and enqueues it. Returns an expected containing the std::future on success, or std::errc::operation_canceled if the pool is shutting down.

Template Parameters

F	Callable type.
Args	Argument types forwarded to `F`.

Parameters

f	Callable to execute.
args	Arguments forwarded to `f`.

Returns: expected<std::future<R>, std::error_code> where R is the result of invoking the stored, decayed callable and arguments.

See also: submit() for the throwing variant.

Definition at line 286 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::bind_args().

Referenced by submit().

◆ try_submit_batch()

template<typename Iterator >

auto threadschedule::detail::work_stealing_pool_backend::try_submit_batch	(	Iterator	begin,
		Iterator	end
	)		-> expected<std::vector<std::future<void>>, std::error_code>

inline

Submit a range of void() callables in one go (non-throwing).

Acquires the lock once per batch, distributing tasks across worker queues in round-robin fashion. Significantly more efficient than calling submit() in a loop for large batches.

Template Parameters

Iterator Input iterator whose value_type is callable as void().

Returns: expected containing a vector of futures, or std::errc::operation_canceled on shutdown.

Definition at line 448 of file work_stealing_pool_backend.hpp.

References threadschedule::detail::multipass_range_size().

Referenced by submit_batch().

◆ wait_for_tasks()

void threadschedule::detail::work_stealing_pool_backend::wait_for_tasks ( )

inline

Block until all pending and active tasks have completed.

Definition at line 653 of file work_stealing_pool_backend.hpp.

References is_current_worker(), and threadschedule::detail::throw_worker_deadlock().

The documentation for this class was generated from the following file:

work_stealing_pool_backend.hpp

Classes

Public Types

Public Member Functions

Detailed Description

Member Typedef Documentation

◆ queued_task

◆ task_type

Constructor & Destructor Documentation

◆ work_stealing_pool_backend() [1/3]

◆ work_stealing_pool_backend() [2/3]

◆ work_stealing_pool_backend() [3/3]

◆ ~work_stealing_pool_backend()

Member Function Documentation

◆ configure_threads() [1/2]

◆ configure_threads() [2/2]

◆ distribute_across_cpus()

◆ get_statistics()

◆ is_current_worker()

◆ operator=()

◆ parallel_for_each()

◆ pending_tasks()

◆ post()

◆ set_affinity()

◆ set_on_task_end() [1/2]

◆ set_on_task_end() [2/2]

◆ set_on_task_start() [1/2]

◆ set_on_task_start() [2/2]

◆ shutdown()

◆ shutdown_for()

◆ size()

◆ submit()

◆ submit_batch()

◆ try_post()

◆ try_submit()

◆ try_submit_batch()

◆ wait_for_tasks()