Building reliable email workflows in an application

A welcome email starts with a database change but finishes in another system. No ordinary transaction spans our account record, queue, SES API call and recipient’s mail server. Reliability comes from preserving enough state to recover at each boundary.

Commit the intent with the account

This Rails-style pseudocode assumes PostgreSQL and a unique database constraint on email_outboxes.dedup_key:

ApplicationRecord.transaction do
  account = Account.create!(registration_attributes)
  EmailOutbox.create!(account_id: account.id,
    dedup_key: "welcome:#{account.id}:v1",
    template: 'welcome-v1', state: 'pending',
    next_attempt_at: Time.current)
end

A durable poller scans committed due rows. An after-commit job can reduce latency but isn’t the only way to discover work. If the process dies after commit and before enqueue, the poller still finds the row. If the transaction rolls back, neither account nor intent survives. The unique constraint rejects concurrent duplicate intents; it doesn’t guarantee one provider-side send. Preserve the intended recipient association and template version, with appropriate personal-data retention.

Claim an attempt before calling the provider

The following is precise pseudocode, not a runnable complete worker. Use short transactions; never hold row locks across a network call:

Transaction 1:
  select due pending row FOR UPDATE SKIP LOCKED
  check recipient suppression; mark suppressed and stop if necessary
  assign random lease token and expiry
  insert attempt(started, outbox_id, lease_token, opaque_request_tag)
  mark outbox sending; commit

Outside transaction:
  call SES SendEmail with the application correlation tag

Transaction 2:
  lock row; check lease token still belongs to this attempt
  accepted: save MessageId, mark attempt accepted and outbox submitted
  definitely not accepted, retryable: record reason and bounded backoff
  definitely not accepted, permanent: mark failed
  acceptance unknown: mark ambiguous for reconciliation
  commit

SKIP LOCKED stops workers claiming the same available work simultaneously. A lease token prevents an old worker overwriting a newer claim’s local state. An expired lease is uncertainty, not proof that sending never happened: a stale worker might already have reached SES. A token check can’t undo that call.

Recheck suppression immediately before initiating the call. A complaint arriving during an in-flight send cannot retract it; prevent later sends and keep that race window short.

The timeout local deduplication cannot solve

SES v2 SendEmail doesn’t expose a documented client idempotency token. Retrying an accepted-but-timed-out request can send twice. Local uniqueness, locks and queue deduplication reduce duplicates but cannot promise exactly-once delivery across that boundary. SDK retries have the same risk.

Store ambiguity explicitly. The product may accept occasional duplicate welcome emails or prefer manual reconciliation; neither choice removes the tradeoff between missed and duplicate sends. Apply retry limits, backoff and operational review rather than indefinitely retrying a rejected address.

Authenticate events, then preserve facts

Choose a concrete transport: SES notifications through SNS to an HTTPS endpoint. Verify SNS signatures with a maintained verifier following AWS’s procedure, constrain certificate URLs, and check the expected TopicArn. HTTPS alone doesn’t establish request origin. Validate subscription confirmations before following their URLs.

verify SNS signature and expected topic
parse notification
transaction:
  insert event with UNIQUE(topic_arn, sns_message_id); skip duplicate
  correlate SES mail.messageId to attempt.provider_id
  store unmatched events for reconciliation
  record recipient-level delivery, bounce or complaint evidence
  upsert suppression for permanent bounce or complaint
  commit
acknowledge after durable processing

SNS can redeliver; an event can arrive before the worker saves its provider ID. Persist unmatched events for later reconciliation. Opaque application tags can help when the selected event-publishing configuration includes them; don’t assume every legacy notification includes arbitrary tags.

Avoid “last event wins.” Store server acceptance, permanent failure evidence and complaint time separately. A late delivery event mustn’t undo complaint suppression; acceptance and later failure can both be real facts. Multi-recipient attempts need per-recipient outcomes.

Every future send consults application suppression and the provider’s applicable suppression behavior. Permanently invalid addresses and complaints stop future sends; temporary delays use bounded retries. Reactivation requires a deliberate rule.

Log opaque outbox and attempt IDs, provider IDs, stage, time, classified error and retry decision. Avoid credentials, full message bodies and unnecessary recipient data. Track pending age, ambiguous attempts, failures and suppression separately: successful API calls alone hide later problems.

References

• Rails transaction callbacks.
• PostgreSQL locking clauses.
• AWS transactional outbox pattern.
• SES v2 SendEmail.
• SNS signature verification.
• SES notification contents and suppression list.