Deadlines
An operation without a time budget can outlive the caller that wanted it: the client gave up seconds ago, but the handler is still holding a connection, a bulkhead slot, and a transaction. Forze uses gRPC-style deadlines: a budget is bound once at operation entry, and everything downstream — hooks, the transaction, dispatched operations, resilience strategies, outbound calls — inherits it for free.
Declare the budget on the plan¶
A deadline is a property of the operation, not of the route or the caller. Declare it where the operation is composed:
registry = (
registry
.bind("orders.create")
.with_deadline(timedelta(seconds=5))
.finish()
.freeze()
)
Patch mode sets a default across many operations at once —
registry.patch(selector).with_deadline(timedelta(seconds=10)).finish() gives
every matched operation a budget in one line. Scope it to a namespace with
patch(selector, namespace=ns), or settle it into per-operation plans with
registry.materialize_patches() so it can't leak to a sibling registry on
merge — see Cross-cutting patches.
Merging is restrictive: across patches, explicit plans, and any caller-bound deadline, the tightest budget wins. A layer can shorten an operation's budget, never extend it — to give one operation more room than a broad patch default, narrow the patch selector instead.
Bind a caller budget at the boundary¶
A boundary (HTTP middleware, a worker loop) may additionally bind the caller's own budget:
from forze.application.execution import bind_deadline
with bind_deadline(timeout_s): # None is a no-op passthrough
result = await resolved_op(args)
Binding is task-scoped and tighten-only — a nested bind can shorten the
budget but never extend past the enclosing deadline. bind_deadline(None)
passes through, so an optional per-request timeout forwards without
branching. Ports that want to derive per-call budgets can read
remaining_time() — seconds left, or None when no deadline is bound.
What enforcement looks like¶
- Entry fail-fast — an operation invoked with its budget already spent fails immediately, before any hook runs.
- The whole plan is bounded — the budget covers hooks, the transaction, and dispatch chains, and propagates into dispatched operations.
- Resilience cooperates — the resilience executor gates its entire strategy chain from the outside, and a retry abandons a backoff sleep that would outlive the deadline, surfacing the real error instead.
Expiry raises exc.timeout (code="deadline_exceeded") — 504 at the
FastAPI edge, details suppressed. The kind is deliberately non-retryable:
within the same invocation the budget is spent, and a fresh invocation carries
a fresh deadline.
Deadline ≠ per-attempt timeout
The resilience TimeoutStrategy bounds a single attempt and keeps
raising retryable infrastructure, so a retry can take another shot. The
invocation deadline bounds the whole call and is final. They compose:
per-attempt timeouts inside, the deadline outside.
When no deadline is declared or bound, nothing changes — the unbound path is one ContextVar read, with no timeout machinery.
The budget is visible¶
A declared deadline projects through the operation catalog like every other
plan-derived fact: OperationCatalogEntry.deadline, an x-deadline-seconds
vendor extension plus a "Time budget" line on
generated FastAPI routes, and a
sentence in each MCP tool description — so clients
and agents can set their own timeouts instead of retrying a call that died of
budget exhaustion.
Budgets cross service boundaries¶
When one Forze service calls another, the outbound HTTP
adapter attaches the caller's remaining budget as an
X-Forze-Deadline-Budget header automatically (opt out per service with
HttpServiceConfig(propagate_deadline=False)). The receiving side honors it
only when asked:
app.add_middleware(
InvocationMetadataMiddleware,
ctx_dep=runtime.get_context,
bind_deadline_from_header=True,
)
The header carries a duration, not an instant, so clock skew between hosts doesn't matter; and because binding is tighten-only, a forged or stale value can only shorten the sender's own request — never extend it.
The budget is deliberately not a message-envelope header: deadlines belong to the synchronous call chain. A queued event consumed after a backlog must not inherit its producer's leftover budget.
Cancellation never skips committed work¶
A deadline firing — or a client disconnecting — cancels the operation's task. One window must survive that: after the root transaction commits, the deferred after-commit work (idempotency records, event dispatch) is a cancellation-protected critical section. It runs to completion even while cancellation is pending, and the cancellation re-raises afterwards — so a committed transaction is never left half-announced.
A late cancel is never reported as safe¶
A deadline or a cancel that lands at or after the commit point leaves the outcome unknown. The commit may have landed, so a retry could apply the operation twice. What happens depends on where it lands:
- In the commit or the after-commit work, a deadline or a cancel raises
exc.internalwithcode="commit_ambiguous"instead ofdeadline_exceededor a bare cancellation. The kind is non-retryable. - A deadline after the transaction has closed — in an outer hook or a dispatched operation — raises the same error.
- A cancel after the transaction has closed stays a cancellation, because
server shutdown and task groups rely on it propagating. The operation logs a
warning with
code="commit_ambiguous"andoperation, carrying the invocation's ids, and adds aforze.commit_ambiguousevent to the active span.
What each surface does with that late cancel:
| Surface | What the caller sees | What records it |
|---|---|---|
forze ops call |
a commit_ambiguous error, exit 1 |
the error document and the warning |
| HTTP routes, MCP tools, Socket.IO and WebSocket commands | nothing: the client went away or the server is stopping | the warning and the span event |
| Durable runs, Inngest functions, your own Temporal activities | the runner's cancel outcome; recovery or a retry may run it again | the warning and the span event |
| Saga steps | saga.step_ambiguous, and nothing is compensated |
the saga's error or its journal row |
Reconcile from the warning before running the operation again. Where a client retries, an idempotency key makes the retry safe.
A query's read-only transaction is a commit point only where the backend could have let a write through it:
| Transaction manager | A query interrupted after its read reports |
|---|---|
Postgres, and the mock's journal and strict managers, when the query's read-only transaction is the only one it entered |
deadline_exceeded (CLI exit 6), or a plain cancellation (CLI exit 130 or 143): the store refused writes inside it |
| The same managers, when the query's transaction is nested in one opened outside forze | commit_ambiguous: it became a savepoint, which cannot be read-only |
| The same managers, when a nested scope on another route finished its body | commit_ambiguous: that route opened, and may have committed, a transaction of its own |
The same managers, when the query queued after-commit work (run_or_defer) |
commit_ambiguous: that work runs after the transaction closes and may write |
Postgres, after a detached() statement, another client's bound_connection(), or another client's transaction or statement outside a transaction |
commit_ambiguous: those commit outside the read-only transaction, so even an autocommit read gives up the skip |
Mongo, Firestore, Neo4j, and the mock's none manager |
commit_ambiguous: they accept read_only without enforcing it |
The kernel asks the manager about the transaction it actually opened, through
ReadOnlyAware.read_only_enforced(), and only while the read-only root is the
only route that may have committed and no after-commit work is queued. A manager that does not implement it counts as not
enforcing. Where a query reports commit_ambiguous but wrote nothing, it is
still safe to run again.
A deadline is one of several controls over how an operation behaves under stress; the full set — retries, bulkheads, breakers, hedging — composes in Resilience.