tvm_tirx.support.popen_pool.PopenPoolExecutor

tvm_tirx.support.popen_pool.PopenPoolExecutor#

class tvm_tirx.support.popen_pool.PopenPoolExecutor(max_workers=None, timeout=None, initializer=None, initargs=(), maximum_process_uses=None, stdout=None, stderr=None)#

Bases: object

An parallel executor backed by Popen processes.

Parameters:

  • max_worker (int) – Maximum number of workers

  • timeout (float) – Timeout value for each function submit.

  • initializer (callable or None) – A callable initializer, or None

  • initargs (Tuple[object]) – A tuple of args for the initializer

  • maximum_process_uses (Optional[int]) – The maximum number of times each process can be used before being recycled, i.e. killed and restarted. If None, processes will be reused until an operation times out.

  • stdout (Union[None, int, IO[Any]]) – The standard output streams handler specified for the workers in the pool.

  • stderr (Union[None, int, IO[Any]]) – The standard error streams handler specified for the workers in the pool.

Note

If max_workers is NONE then the number returned by os.cpu_count() is used. This method aligns with the behavior of multiprocessing.pool().

shutdown(wait=True)#

Shutdown the executor and clean up resources.

Parameters:

wait (bool) – If True, wait for pending work to complete.

Note

DEADLOCK WARNING: This method can deadlock when called during garbage collection due to exception reference cycles. When exceptions occur, Python creates reference cycles that delay garbage collection. The deadlock happens when: exception creates reference cycle → new pool creates worker → GC cleans old pool → old pool’s __del__ calls shutdown() which tries to acquire locks again.

submit(fn, *args, **kwargs)#

Submit a new function job to the pool

Parameters:

  • fn (function) – The function to be invoked.

  • args (list) – Positional argument.

  • kwargs (dict) – Keyword arguments

Returns:

future (concurrent.futures.Future) – A future that can be used to access the result.

Return type:

Future

map_with_error_catching(fn, iterator)#

Same as map, but catches exceptions and return them instead.

Parameters:

  • fn (function) – The function to be invoked.

  • iterator (Iterator) – Input iterator.

Returns:

out_iter (Iterator[MapResult]) – The result iterator.