- Workers with different task durations
- Shared checkpoint with explicit versions
- Integrate or report the delay explicitly
Define checkpoints
Specify which tasks must be read together. Distinguish required and optional inputs for each checkpoint.
Progress through events
A finished worker emits event_id, task_id, checkpoint_version and artifact reference. Deduplicate repeated events.
Bound waiting
On timeout explicitly report missing inputs. Do not label a result complete when a mandatory input is missing.
Merge the version
Merge outputs of the same version; route late arrivals to the next review. Record waiting cost after the experiment.
Role responsibilities
- Asynchronous worker
- Produce versioned output events without losing work while awaiting a checkpoint.
- Merger
- Merge ready outputs belonging to the same checkpoint version.
- Straggler checker
- Record missing tasks, retries and timeouts.
Shared memory contract
event_idtask_idcheckpoint_versionartifact_refreceived_atrequiredtimeout_result- Stop condition
- Finish after mandatory inputs are verified and merged. If time expires with missing inputs, deliver an incomplete status.
- Primary failure mode
- Mixing versions creates incorrect synthesis; a global barrier creates dependence on one slow worker. Process repeated messages idempotently.