The problem: bursty, expensive, must-not-drop work
PDF rendering with a headless browser is CPU- and memory-heavy, and orders arrive in bursts. Doing it inline in the request would either time out the customer or crash the box under a spike. And you can never silently drop an order someone paid for. That combination — expensive, bursty, exactly-once — is what a queue is for.
Queue first, render second
The order API does almost nothing at request time: it validates the order, computes a content hash of the render inputs, and enqueues a job. The customer gets an immediate response; the heavy work happens on a worker that pulls from the queue at a rate the hardware can actually sustain.
// API: enqueue, don't render inline
const key = contentHash(order.renderInputs); // stable per unique book
await renderQueue.add('book', { orderId: order.id, key }, {
jobId: key, // dedupe identical jobs
attempts: 3,
backoff: { type: 'exponential', delay: 5000 },
removeOnComplete: 1000,
});Content-hash caching: don't render the same thing twice
Two orders for the same personalized book with the same inputs produce byte-identical PDFs. Hashing the render inputs and using that hash as both the cache key and the BullMQ jobId means identical work is deduplicated: if the PDF already exists in object storage under that hash, the worker skips rendering entirely and reuses it. On a catalog with popular templates this removes a large fraction of render load for free.
// Worker: cache hit -> skip the expensive render
const cached = await r2.head(`pdf/${job.data.key}.pdf`);
if (cached) return { url: r2.url(job.data.key), cached: true };
const pdf = await renderWithPuppeteer(job.data); // heavy
await qpdf.linearize(pdf); // print-ready
await r2.put(`pdf/${job.data.key}.pdf`, pdf);Storage and fulfillment
Rendered PDFs land in Cloudflare R2 (cheap egress, S3-compatible). Fulfillment is a print partner (Gelato) reached over a webhook. The direction that matters for security is the callback: the partner tells us a job printed/shipped, and a forged callback there would mark real product as fulfilled — so that endpoint verifies an HMAC signature before it trusts anything.
Rule of thumb: sign every inbound webhook that changes state. A payment or fulfillment callback an attacker can forge is a way to get free product or corrupt your order state.
What the queue buys you operationally
- Backpressure: a traffic spike queues instead of crashing the renderer.
- Retries with backoff: a transient failure (font load, memory blip) recovers on its own.
- Observability: queue depth and job age are a single, honest health metric.
- Idempotency: the content-hash jobId means a retried enqueue can't produce a duplicate render.
Key takeaways
- Move expensive, bursty work off the request path onto a queue keyed for idempotency.
- Hash the inputs: identical work should never run twice.
- Sign inbound webhooks that mutate order/fulfillment state.
- Cache the artifact in cheap object storage and let the worker short-circuit on a hit.