An agent sends a request to create a service order. The receiving system creates the order, but the response is lost before the agent records the result. When the job restarts, the agent sees an unfinished step and creates another order. Both systems behaved plausibly on their own; the business now has a duplicate.
Reliable continuation requires more than saving a conversation. The application needs durable job state, a stable identity for each intended business action and a way to resolve uncertain outcomes. The important question is what has been confirmed outside the agent, not merely which sentence it last generated.
Separate the job from an attempt to run it
Give a business job an identity that survives worker restarts. A job can contain several operations, and each operation can have several attempts. Keep those identities separate. Starting a new attempt should not accidentally declare a new business request.
In an illustrative service workflow, the job might prepare an order, wait for approval, create the order in a back-office system and notify the requester. Record the input version, approved payload, operation identifier, external record identifier and status of each step. Store references to evidence rather than relying on a conversational summary.
Use states that reflect what the system actually knows. “Pending,” “waiting for approval,” “running,” “confirmed,” “outcome unknown,” “failed” and “cancelled” may have different recovery rules. If the order service timed out, “failed” can be misleading: the order may already exist.
The user interface should expose this distinction. Tell an operator that the request was sent but its result is being checked, instead of inviting them to press a generic retry button. Otherwise a person can unknowingly create the same duplication that the software is trying to avoid.
Understand what a checkpoint does and does not prove
A checkpoint preserves workflow state so work can continue. LangGraph documents checkpoint-based persistence, including state history and resumable execution. The storage backend and durability behaviour still need to suit the application. LangGraph persistence
Temporal uses recorded event history and replay to resume workflow progress. Its documentation also explains the determinism constraints around replay. This provides a useful foundation for long-running coordination, but it does not make an unrelated external service part of one atomic transaction. Temporal workflow execution
The gap matters when an external action succeeds before its result is durably recorded. A checkpoint that says “about to create an order” cannot reveal whether the remote system already created it. The application must use the remote operation's contract, an external reference or a reconciliation process to settle that question.
Temporal specifically recommends idempotent Activities because an Activity may be retried when completion was not reported, even if its external effect already occurred. Apply the same reasoning to whichever workflow foundation you choose. Temporal Activity idempotency
Make retries refer to the same intended action
An idempotency key identifies repeated attempts at the same operation to a service that supports it. Reuse the appropriate key when retrying the same authorised action, and preserve its association with the payload. Generating a new key on every retry defeats that purpose.
Do not assume that every API has the same guarantee. Check which operations accept keys, how long records are retained, what happens when parameters change and how errors are replayed. Stripe's documentation, for example, distinguishes API v1 and v2 behaviour and describes retention and parameter checks for v1. Those are provider-specific contracts, not a rule for all systems. Stripe idempotent requests
For an API without such support, inspect whether it accepts a unique external business reference that can be queried or constrained. If it does not, a local “already sent” flag alone cannot close the crash window between the remote action and the local write. Use a reconciliation step or hold the uncertain operation for a person rather than claiming an exactly-once guarantee.
Concurrent workers add another risk. Use durable coordination so two attempts cannot independently own the same pending operation. Combine that with downstream deduplication or a uniqueness rule where available. A lease or lock alone may expire while a slow remote request is still completing.
Classify failures before deciding to repeat work
| Observed condition | Useful next step | What to avoid |
|---|---|---|
| Required field fails validation | Correct the input and obtain approval if meaning changes | Repeating the same invalid payload indefinitely |
| Service explicitly rejects access | Resolve permission or stop the job | Switching to a more privileged credential automatically |
| Temporary rate limit | Retry within a bounded policy and deadline | Immediate bursts that increase overload |
| Response is lost after submission | Query by operation or business reference | Creating a new business operation without checking |
| Customer changes the approved request | Prepare and review a new version | Reusing approval for the earlier payload |
| Confirmed operation already exists | Continue from its recorded result | Repeating the external action to reconstruct context |
Set a limit and a deadline for automatic retries, with increasing delays where appropriate. The policy should suit the provider response and business urgency. A job that retries forever can occupy capacity while staff assume someone else is handling it.
Cancellation also needs a contract. Cancelling a local task does not necessarily cancel an external action already accepted by another system. Show what has stopped, what remains uncertain and whether a separate reversal is possible. Some actions require a compensating business operation rather than a technical rollback.
Give people a safe recovery path
A recovery screen should show the original request, current version, completed steps, external references and the last meaningful error. Let authorised staff reconcile a result, amend a request or cancel the remaining work. Avoid a button that replays the entire workflow when only one step needs attention.
Keep action history append-only or otherwise auditable within the product's data rules. Record who made a manual decision and which evidence supported it. Preserve the difference between an operator confirming an existing order and the system creating a new one.
Test failure windows deliberately in a non-production environment. Stop a worker before sending, after the remote service accepts the request and before the result is saved. Deliver the same event twice, let two workers compete and resume a job after its approval becomes stale. These cases expose defects that a successful end-to-end demo does not.
Verify the final business state independently. Check for duplicate orders, missing notifications, unintended changes and jobs left without an owner. Then inspect whether the interface gave staff enough information to recover. Reliability includes the person operating the workflow, not only the worker process.
For choosing a workflow foundation, see n8n, LangGraph and custom code. To make these recovery cases part of release acceptance, use evals that check business outcomes.
Talvyn Labs can build your agent around durable jobs, explicit approvals, safe retry contracts and an operations interface. Bring a workflow that gets stuck or creates repeated work. We can map its failure windows, connect the systems and develop a product that can explain what happened and continue from verified results.
Build an agent that can recover without repeating the business action
Let Talvyn Labs map failure windows, connect durable job state and develop explicit approvals, retry contracts and an operations interface for your team.
Build a recoverable business agent