Concurrency control

Tune how many processing components can run at the same time with `max_inflight_components` or the `COCOINDEX_MAX_INFLIGHT_COMPONENTS` environment variable, and understand how the limit bounds a hierarchy of components without deadlocking it.

Version
v 1.0.62
Last reviewed
Jul 4, 2026

CocoIndex executes each processing component as a unit of concurrent work. By default, up to 1024 processing components can run at the same time per app. When components perform resource-intensive work (e.g., calling external APIs, running ML models), or each holds a sizeable piece of data (e.g., a file’s contents) while it runs, you may want a tighter limit.

Setting the limit

Set max_inflight_components in AppConfig:

python
app = coco.App(
    coco.AppConfig(name="MyPipeline", max_inflight_components=4),
    app_main,
    sourcedir=pathlib.Path("./data"),
)

With max_inflight_components=4, at most 4 processing components execute at the same time. When a component finishes, the next pending one starts. A mount(), use_mount() or mount_each() call waits while no slot is free, so a parent cannot run ahead of its children without bound: a loop that mounts a child per source item advances only as fast as children finish.

Setting max_inflight_components=1 serializes all components — only one runs at a time.

You can also set the limit via the COCOINDEX_MAX_INFLIGHT_COMPONENTS environment variable:

bash
export COCOINDEX_MAX_INFLIGHT_COMPONENTS=4

Precedence: AppConfig value > environment variable > default (1024).

Nested mounts: a parent lends its slot

A parent component usually waits for its children: use_mount() returns the child’s result, and a component’s own target states are committed only once all its children are done. If a waiting parent simply kept its slot, a small limit could fill up with waiting parents and leave no slot for the children they wait for.

Instead, a parent keeps its slot while it waits and lends it to one child at a time:

  • The first child a parent mounts runs on the parent’s slot; it needs no free slot of its own.
  • Further children take free slots from the pool, and their mount() / use_mount() calls wait while the pool is empty.
  • When the child running on the lent slot finishes, the slot passes to the parent’s next waiting child.

So a parent that is blocked — on its children, or on anything else after mounting one, such as an application-level asyncio.Semaphore — always has one child that can run, and a hierarchy of any depth completes even with max_inflight_components=1. The limit bounds the components that execute at the same time; counting the parents waiting on them, the components in flight are bounded by the limit times the depth of the component hierarchy.

A few related rules:

  • A component that awaits a function call — including a batched one (@coco.fn.as_async(batching=True)) — keeps its slot; only mounting a child lends it.
  • When several parents mount the same component at once (two App.update() calls overlapping, or several owners of a shared component), only the run that executes takes a slot. The others wait for it without a slot and reuse its result.
  • Live components (coco.auto_refresh, mount_each() over a live map) borrow the slot of the component that mounted them for their cycles and item updates, like its other children, for as long as it runs; afterwards they take slots from the pool like any other component.
  • Deleting components (cleaning up a removed subtree, App.drop()) is bounded by the same limit.
PreviousOverview
CocoIndex Docs