The order appeared twice: idempotency with Redis

The order was already in the system

Some orders were appearing twice in our Nuvemshop app. Sometimes the second entry showed up hours after the first. In other cases, it appeared on another date.

The app had more than 500 active customers. Most were small stores, with at least a hundred medium or large ones. I fixed the problem by creating and handling an idempotency key with Redis: a way to recognize an operation that had already been processed when another attempt arrived.

Putting the volume in context

With 500 stores, traffic depends heavily on which stores are selling, how many orders they receive, and the events the app subscribes to. For a sense of scale, we can use these assumptions:

• 400 small stores with 10 orders per day: 4,000 orders.
• 100 medium or large stores with 150 orders per day: 15,000 orders.
• Three notifications per order, such as creation, payment, and an update: 57,000 notifications per day.

That works out to about 40 notifications per minute, or 0.66 per second, spread across 24 hours. It is a hypothetical scenario, not a market average or a historical measurement of the app. Product and customer events and additional delivery attempts are excluded. Requests can also cluster during sales hours in ways a daily average hides.

Even so, two executions of the same operation are enough to create a duplicate order.

One order can produce several notifications

Nuvemshop has separate events for order creation and payment. Its webhook documentation also states that messages can repeat and be processed out of order. The integration needs to handle those conditions.

One purchase can therefore appear in several legitimate notifications. The import needs to recognize the existing order when that happens.

In our case, the different timestamps alone cannot establish where each duplicate originated. That would require correlating deliveries with internal processing. They do establish a requirement for the solution, though: recognizing a repeat after the first request has finished. A block lasting a few seconds will not cover another attempt the following day.

Choosing what to identify

An operation is idempotent when repeating it preserves the same final effect. For an initial import, that means recognizing the order that has already been created.

A key for that operation could look like this:

nuvemshop:store:123:order:456:import:v1

The store scopes the origin. The order identifies the resource. The import segment names the effect we want to execute once. The identifiers and format are teaching examples, as are the durations below; they do not reproduce the original configuration.

If creation and payment trigger the same import, distinguishing them only by event name would allow two imports. But a key blocking all work on that order would prevent legitimate updates. Updating the payment is a separate operation. Recurring updates may need a version or a specific transition to identify the change being applied.

Generating a random identifier for every incoming delivery would not help either. It would change between attempts, exactly when we need to recognize that they refer to the same business effect.

Two executions competing for the same key

Consider two processes receiving the same operation almost simultaneously. If both perform a GET before either writes the key with SET, both could start the work.

Redis allows the reservation to happen in one atomic operation:

SET nuvemshop:store:123:order:456:import:v1 execution-abc NX EX 120

NX creates the key only when it is absent. EX sets expiration in seconds. An OK response means the reservation was created; when the key already exists, this attempt does not replace it. Details are in the SET command reference.

In this example, one execution gets the reservation and the other finds the key occupied. The 120 seconds represent a processing window. Once the work finishes, we still need enough information to recognize a later delivery. Deleting the key and forgetting the result would allow duplication again.

What if the process stops after writing?

To discuss the flow, it helps to distinguish three situations:

1. The operation has not been recorded, and an execution tries to reserve it.
2. An execution is running, and another attempt should avoid the same effect in parallel.
3. The operation has finished, and a repeat can use the existing result.

These states are a model for explaining failure handling. Finding a key does not prove that the work finished. If execution is still underway, acknowledging success requires a completion guarantee, such as a delivery already stored for later processing.

One failure window deserves particular attention: the order was written, but the process crashed before recording completion in Redis. The next attempt needs to check the persisted result to recognize that the write happened.

To complement the Redis control, I would recommend a unique constraint on platform, store, and external identifier, together with an idempotent write at the destination. That recommendation goes beyond the account of the original fix. It protects order creation even when the temporary key no longer exists.

Expiration is part of the decision

A completion key lasting only a few minutes may already be gone when a repeat arrives on another date. Retention needs to cover the repeat and reprocessing window the application supports. The specific TTL used in that incident is not recorded here.

Memory-pressure eviction and Redis data loss also need consideration. When an effect must remain unique, that rule needs to exist in the final data store.

Reservations have another detail. If one expires and a second process takes over, the first process must not delete the second process’s key when it finishes. The value execution-abc identifies the reservation owner. Releasing it needs to check that value atomically, as described in the Redis lock documentation.

What I would test

I would start by sending the same operation twice at the same time and checking how many orders were written at the destination. Then I would repeat the delivery later to check that the result was still recognized.

I would also interrupt execution before and after the write, simulate reservation expiration, and send a legitimate order update. That update needs to keep working after the initial import.

In logs, I would track the store, order, and operation, with an indication of work in progress or completed. Those records distinguish an attempt that found an occupied reservation from one that recognized an already persisted result.