Payment webhooks are the most critical events in your system. A missed payment_intent.succeeded webhook means an order never gets fulfilled. A duplicate webhook means a customer gets charged twice. Unlike a slow page load or a broken image, payment webhook failures cost real money.

This guide covers how to build payment webhook handling that never misses an event and never processes one twice.

Step 1: Understand What Goes Wrong

Scenario 1: Missed webhook, unfulfilled order

A customer pays $200 for a product. The payment processor sends a webhook to confirm the charge. Your server returns a 500 error because of a temporary database issue. The payment processor retries 3 times over the next hour, but your server is still recovering. After exhausting retries, the event gets dropped. The customer paid but never receives their order.

Scenario 2: Duplicate webhook, double fulfillment

A customer pays for one item. The payment webhook arrives and your handler starts processing, but responds too slowly. The payment processor assumes the delivery failed and retries. Now your handler runs twice. The customer receives two shipments and you absorb the cost of the extra one.

Scenario 3: Out-of-order events

A customer starts a subscription, then cancels right away. The subscription.canceled webhook arrives before the subscription.created webhook (network routing is not deterministic). Your handler tries to cancel a subscription that does not yet exist in your database.

Business impact:

Failure	Impact
Missed payment confirmation	Customer paid, order not fulfilled, support ticket filed
Duplicate processing	Double charge or double fulfillment, refund required
Out-of-order events	Inconsistent state, manual intervention required
Slow webhook response	Payment processor marks your endpoint as unhealthy

Step 2: Ensure At-Least-Once Delivery

The first rule: never miss a webhook. Use a task queue as a buffer between the payment processor and your business logic.

// Webhook endpoint - do minimal work, respond fast
app.post('/api/payment-webhook', async (req, res) => {
  // Verify signature first
  if (!verifyPaymentSignature(req)) {
    return res.status(401).json({ error: 'Invalid signature' });
  }

  // Queue the event for reliable processing
  await aq.tasks.create({
    targetUrl: 'https://your-app.com/api/process-payment-event',
    payload: {
      eventId: req.body.id,
      eventType: req.body.type,
      data: req.body.data,
    },
    maxRetries: 5,
    retryBackoff: 'exponential',
  });

  // Respond at once - payment processor sees 200
  res.json({ received: true });
});

This pattern separates receipt from processing. Your webhook endpoint always responds fast (keeping the payment processor satisfied), and the processing runs in a task with retries and persistence.

Why this matters:

If your processing logic fails, the task queue retries on your behalf
The payment processor sees a fast 200 response and skips its own retry logic
You avoid the thundering herd of payment processor retries hitting your server

Step 3: Prevent Duplicate Processing

Payment processors guarantee at-least-once delivery, not exactly-once. You will receive the same event more than once. Your handler must stay idempotent.

app.post('/api/process-payment-event', async (req, res) => {
  const { eventId, eventType, data } = req.body;

  // Deduplicate by event ID
  const existing = await db.processedEvents.findOne({ eventId });
  if (existing) {
    return res.json({ received: true, deduplicated: true });
  }

  // Process based on event type
  switch (eventType) {
    case 'payment_intent.succeeded':
      await fulfillOrder(data.orderId, data.amount);
      break;
    case 'payment_intent.payment_failed':
      await handleFailedPayment(data.orderId);
      break;
    case 'charge.refunded':
      await processRefund(data.chargeId, data.amount);
      break;
  }

  // Record that we processed this event
  await db.processedEvents.insert({
    eventId,
    eventType,
    processedAt: new Date(),
  });

  res.json({ received: true });
});

For financial operations, use database transactions:

await db.transaction(async (tx) => {
  // Check and process atomically
  const order = await tx.orders.findForUpdate(orderId);
  if (order.status === 'paid') return; // Already processed

  await tx.orders.update(orderId, { status: 'paid' });
  await tx.ledger.insert({ orderId, amount, type: 'payment' });
});

Step 4: Handle Out-of-Order Delivery

Events can arrive in any sequence. Design your handlers to tolerate unexpected ordering.

Pattern: status machine with valid transitions

const VALID_TRANSITIONS = {
  pending: ['paid', 'failed', 'canceled'],
  paid: ['refunded', 'shipped'],
  shipped: ['delivered', 'returned'],
  failed: ['pending'], // retry
};

async function updateOrderStatus(orderId, newStatus) {
  const order = await db.orders.findOne({ orderId });

  if (!order) {
    // Event arrived before creation event - store for later
    await db.pendingEvents.insert({ orderId, status: newStatus, receivedAt: new Date() });
    return;
  }

  const allowed = VALID_TRANSITIONS[order.status] || [];
  if (!allowed.includes(newStatus)) {
    console.log(`Ignoring transition ${order.status} -> ${newStatus} for ${orderId}`);
    return;
  }

  await db.orders.update(orderId, { status: newStatus });
}

Pattern: process pending events after creation

// When a new entity is created, check for events that arrived early
async function createOrder(orderId, data) {
  await db.orders.insert({ orderId, ...data, status: 'pending' });

  // Process any events that arrived before creation
  const pending = await db.pendingEvents.find({ orderId });
  for (const event of pending) {
    await updateOrderStatus(orderId, event.status);
  }
  await db.pendingEvents.delete({ orderId });
}

Step 5: Monitor and Alert on Webhook Failures

Do not wait for customer complaints to surface webhook problems.

Track webhook processing metrics:

app.post('/api/process-payment-event', async (req, res) => {
  const start = Date.now();

  try {
    await processEvent(req.body);
    metrics.increment('webhooks.processed', { type: req.body.eventType });
  } catch (error) {
    metrics.increment('webhooks.failed', { type: req.body.eventType });
    throw error;
  } finally {
    metrics.histogram('webhooks.duration', Date.now() - start);
  }

  res.json({ received: true });
});

Alert on these conditions:

Condition	Alert Threshold	Why
Processing failures	> 1% of events	Something is broken
Processing latency	p95 > 5 seconds	Handler runs too slow
Event gap	No events for 30 minutes	Webhook delivery may have stopped
Duplicate rate	> 10% duplicates	Payment processor retries too aggressively
Unknown event types	Any	New event type you are not handling

Reconciliation as a safety net:

Even with perfect webhook handling, run periodic reconciliation:

// Daily: compare your orders against payment processor records
async function reconcilePayments() {
  const recentPayments = await paymentApi.listCharges({ created: { gte: yesterday } });

  for (const charge of recentPayments) {
    const order = await db.orders.findByChargeId(charge.id);
    if (!order) {
      await alertTeam(`Unmatched payment: ${charge.id} for $${charge.amount}`);
    }
  }
}

Step 1: Understand What Goes Wrong

Step 2: Ensure At-Least-Once Delivery

Step 3: Prevent Duplicate Processing

Step 4: Handle Out-of-Order Delivery

Step 5: Monitor and Alert on Webhook Failures

Related Guides