Skip to content
Blocify
All posts
Commerce5 min read

Killing the nightly CSV export

Overselling during campaigns is almost never a warehouse problem. It is a sixteen-hour-old stock file, and replacing it is mostly about idempotency.

Ask a retailer why they oversold on Black Friday and you will hear about the warehouse. Look at the integration and you will usually find a CSV that leaves the ERP at 02:00 and lands in the storefront at 02:40, describing stock as it was the previous evening.

For most of the year that lag is invisible. During a campaign, an hour of traffic can clear a size run, and every order after that is a refund, a support contact and a customer who does not come back.

Replacing the file with an event stream is the easy part. The hard part is that the storefront now receives thousands of small updates instead of one big one, and any of them can arrive twice, out of order, or not at all.

So the rules matter more than the transport. Every message carries a version and an idempotency key. Applying the same message twice is a no-op. Applying an older message after a newer one is discarded rather than written. A message that fails goes to a dead-letter queue a human can inspect, not into a log nobody reads.

We also keep the nightly file for a while, running as a reconciliation job rather than the source of truth. If the stream and the file disagree at 03:00, somebody gets an alert with the specific SKUs. That parallel period is what makes the switch boring.

The result on one forty-market retailer was stock accuracy going from 'usually fine' to a measurable number, and overselling during campaigns dropping to effectively zero. The storefront did not change at all.

Written by Fitim Hoti — Head of Commerce

Related services

More from the blog