Building SDF Server: Fastify, BullMQ, and Multi-Tenant Architecture

@etapsky/sdf-server-core is the production-grade server that powers the Etapsky SaaS platform. It handles document ingestion, async validation, multi-tenant storage, ERP webhook delivery, and the REST API that developer-facing SDKs talk to. This is a walkthrough of the architectural decisions we made and the problems we ran into.

Why Fastify, not Express

Express is the default answer for Node.js HTTP servers, but it has a structural problem for high-throughput document APIs: it was designed before async/await existed, and error handling in async route handlers requires wrapping every handler or using a plugin. Unhandled promise rejections in Express routes crash silently or, in newer Node.js versions, terminate the process.

Fastify handles async routes natively. A route handler that throws or rejects automatically produces a 500 response with a structured JSON error body — no wrapper needed. Fastify also serializes response payloads using a JSON Schema-based fast serializer (fast-json-stringify) that is 2–3× faster than JSON.stringify for known response shapes.

The performance difference matters at scale. Fastify processes roughly 75,000 requests/second on a single core for a typical JSON route. Express is around 40,000. For a validation API that processes thousands of SDF documents per hour, this headroom means fewer instances, lower infrastructure cost.

import Fastify from 'fastify'
import multipart from '@fastify/multipart'
import rateLimit from '@fastify/rate-limit'

const app = Fastify({
  logger: { level: process.env.LOG_LEVEL ?? 'info' },
  ajv: { customOptions: { strict: false } },
})

await app.register(multipart, { limits: { fileSize: 50 * 1024 * 1024 } })
await app.register(rateLimit, { redis, keyGenerator: tenantKeyGenerator })

Multi-tenancy architecture

Every API request to sdf-server-core is authenticated by an API key. The key is hashed with BLAKE2b and looked up in a Redis hash. The lookup result is a TenantContext object cached for 60 seconds:

interface TenantContext {
  tenantId:    string
  plan:        'free' | 'pro' | 'enterprise'
  storageRoot: string       // S3 prefix: tenants/{tenantId}/
  rateLimit:   number       // requests per minute
  signingKey?: string       // base64 DER public key
  webhooks:    WebhookConfig[]
}

Isolation between tenants is enforced at three layers:

• Storage Every document is stored under a tenant-prefixed S3 path. There is no shared prefix. A bug that accidentally reads the wrong prefix would read an empty key, not another tenant's data.
• Rate limiting Per-tenant rate limits are enforced by @fastify/rate-limit with a Redis backend. The key generator uses the tenantId, not the IP address — an enterprise tenant with 100 servers behind a NAT isn't penalized for a single IP limit.
• Queue isolation BullMQ validation jobs are enqueued with the tenantId as a job attribute. Workers read the tenantId from the job and apply tenant-specific schema registry lookups and signing key checks.

Async validation with BullMQ

SDF document validation is not instantaneous. A document may reference schemas that need to be fetched and cached, have signatures that need verification, or trigger ERP webhook delivery that may retry on failure. Running all of this synchronously in the HTTP handler would mean holding a connection open for seconds.

The upload endpoint returns immediately with a 202 Accepted and a jobId. The actual work happens in a BullMQ worker pool:

app.post('/documents', async (req, reply) => {
  const { tenantId } = req.tenant

  const file = await req.file()
  const s3Key = `tenants/${tenantId}/inbox/${ulid()}.sdf`
  await s3.put(s3Key, file.file)

  const job = await validationQueue.add('validate', {
    tenantId,
    s3Key,
    webhooks: req.tenant.webhooks,
  }, { attempts: 3, backoff: { type: 'exponential', delay: 2000 } })

  return reply.code(202).send({ jobId: job.id, status: 'queued' })
})

Clients poll GET /documents/:jobId/status or receive a webhook when the job completes. The status endpoint reads directly from BullMQ's Redis state — no database query needed.

S3 without the AWS SDK

The AWS JavaScript SDK v3 is a large dependency tree. For the operations we need — PutObject, GetObject, DeleteObject, HeadObject, ListObjectsV2 — a minimal S3 client built on the AWS Signature Version 4 algorithm is about 200 lines of code.

We use the native fetch API (available in Node.js 18+) and compute SigV4 signatures manually. This also means zero changes for MinIO compatibility — MinIO implements the S3 API, and our client works identically against both by changing the endpoint URL and region.

interface S3Client {
  put(key: string, body: ReadableStream | Buffer): Promise<void>
  get(key: string): Promise<ReadableStream>
  head(key: string): Promise<{ size: number; lastModified: Date }>
  del(key: string): Promise<void>
  list(prefix: string): AsyncIterable<{ key: string; size: number }>
}

The implementation swaps seamlessly between AWS S3 and local MinIO depending on the STORAGE_ENDPOINT environment variable. In development, a Docker Compose MinIO instance provides local S3 semantics with the exact same code path that runs in production.

ERP connectors

The most common enterprise integration request we've received: "can it push to SAP?" The answer is yes, via a connector layer in sdf-server-core. When a validation job completes successfully, the worker checks the tenant's webhook configuration for registered connectors and fires them asynchronously.

Each connector is a small function that receives the validated data.json payload and the tenant context, and returns a structured result. Currently implemented:

• SAP Business One: REST API (Service Layer). Upserts Business Partner and creates AP/AR Invoice. Handles document type mapping (SDF invoice → SAP OINV/OPCH).
• SAP S/4HANA: OData v4 endpoints. Creates Supplier Invoice via /API_SUPPLIERINVOICE_PROCESS_SRV. Supports cost center and profit center assignment from SDF extended fields.
• Generic webhook: Sends the SDF document as a multipart/form-data POST to any URL. Includes X-Sdf-Signature header for recipient verification. Retries with exponential backoff via BullMQ.
• Email delivery: Attaches the .sdf file to an email and sends via SMTP or the Resend API. Used for counterparties that cannot receive webhooks.

Schema registry integration

Every SDF document includes a schema.json that defines the structure of its data.json. On the server side, the validation worker doesn't trust the schema bundled in the uploaded document — it cross-references it against the tenant's schema registry to ensure the schema hasn't been tampered with.

The schema registry is built on @etapsky/sdf-schema-registry, backed by a Redis sorted set (schema versions as scores) and the S3 schema store. Schema lookup is O(log n) and typically cached in the worker's process memory for the duration of a request batch.

const worker = new Worker('validation', async (job) => {
  const { tenantId, s3Key } = job.data

  const buf = await s3.get(s3Key)
  const doc = await SdfKit.read(buf)

  const registrySchema = await registry.resolve(
    tenantId,
    doc.meta.schemaId,
    doc.meta.schemaVersion
  )

  const result = await SdfKit.validate(doc, { schema: registrySchema })
  if (!result.valid) throw new ValidationError(result.errors)

  await deliverWebhooks(tenantId, doc, job.data.webhooks)
  return { documentId: doc.meta.id, valid: true }
}, { connection: redisConnection })

Observability

Every request in sdf-server-core is traced with OpenTelemetry. The Fastify request context carries a trace ID that propagates into BullMQ job attributes and S3 request headers. When a validation job fails 20 minutes after the upload, you can trace the full execution path: HTTP request → queue enqueue → worker dequeue → S3 fetch → validation → webhook delivery attempt.

Fastify's built-in pino logger emits structured JSON. In production, these go to a log aggregator. In development, pino-pretty formats them for terminal readability. The logger is injected into every route handler via Fastify's req.log, ensuring the trace ID is present on every log line.

Production deployment

The server and worker run as separate processes — both from the same Docker image, but with different start commands:

services:
  api:
    image: etapsky/sdf-server:latest
    command: node dist/server.js
    scale: 2
    # Stateless — can scale horizontally

  worker:
    image: etapsky/sdf-server:latest
    command: node dist/worker.js
    scale: 4
    # CPU-bound validation — scale independently

  redis:
    image: redis:7-alpine
    command: redis-server --save "" --appendonly no
    # Queue + cache only — persistence not required

The API is stateless: no local file system, no in-process cache that can't be rebuilt. Any API instance can handle any request. Workers are the CPU-intensive side — they decompress ZIP archives, parse JSON, run ajv validators — and scale independently. In practice, we run 2 API instances and 4 workers per 1,000 active tenants.

Redis is configured without persistence. It is used for BullMQ job queues and rate limiting counters — data that is ephemeral by nature. If Redis restarts, queued jobs that weren't yet claimed by a worker are lost. Uploaded documents are always durably stored in S3 before the job is enqueued, so a lost queue entry means the client retries the upload — not that the document is lost.

The 0.1.x release of sdf-server-core is the foundation. Upcoming work: a GraphQL subscription endpoint for real-time job status, a tenant management UI, and the connection between the server and the SaaS billing system.