The single sharpest FHIR concurrency failure mode is background jobs competing with interactive requests on the same connection pool. Interactive requests need pool slots for milliseconds; background jobs hold slots for seconds or minutes. When the two share a pool, the interactive workload experiences the background job's latency as if it were its own.
Naming the split is the discipline that keeps morning rounds from stalling behind a nightly export. For related walkthroughs, the FHIR learning path collects the surrounding material.
The Shape of the Contention
Interactive traffic arrives in bursts and drains fast. A clinician opens a chart, fires a burst of ten reads, and moves on. Each read holds the pool for maybe fifty milliseconds. The pool utilization is momentary and low.
Background jobs are the opposite. A Bulk Data export walks the archive and holds pool slots for seconds per query for minutes on end. Utilization is sustained and high. When interactive requests hit that pool during the export, they wait behind the export's slots.
A pass through the site's concurrency p99 predictor surfaces the interaction by taking mixed workload assumptions and projecting the shared-pool behavior.
Two Solutions That Work
Two solutions solve the contention:
- Split the pool. Give background jobs their own connection pool with a fixed slot count. Interactive traffic gets the primary pool. Neither can starve the other.
- Rate-limit background jobs. Cap background job pool usage to a percentage of total capacity. Interactive traffic gets guaranteed headroom.
Both work. The split-pool pattern is more surgical; the rate-limit pattern is easier to deploy. Most mature deployments end up using both.
The Bulk Export Case
Bulk Data exports are the most common background workload. They read from history tables, walk large ranges, and hold pool slots long enough for interactive traffic to notice. For the database-side story, concurrency limits that come from the database, not the API covers the shape.
Every deployment that supports Bulk Data should treat exports as a first-class background workload with its own capacity budget.
Subscription Cascades Are Sneaky
FHIR subscriptions fire notifications on writes. Each notification may trigger downstream work that hits the pool. If subscription dispatchers share the same pool as interactive requests, a burst of writes can produce a cascade that sits in the pool for extended time.
Subscription workloads should have their own pool or a strict rate limit. Sharing with interactive traffic invites the same failure mode as the export case.
Nightly Reports Are Deceptive
Reporting jobs that run overnight look safe because the interactive workload is quiet. The trap is that reports written for the nightly window sometimes get scheduled during the day when someone needs a fresh view. On that day, the report hits the pool during interactive hours and the trouble starts.
Reports should carry a scheduled window and a rate limit that assumes they might run outside the window. For the throttling framing, throttling policies that keep FHIR fair under contention covers the patterns.
Monitoring the Split
The dashboard that keeps the split honest shows pool utilization by workload class. Interactive traffic, background jobs, subscriptions, and reporting each contribute a share; the shares should be visible.
Reports that only show total utilization miss the mix and mask the contention until it fires as an incident. For the wider prediction framing, predicting concurrency for a FHIR API before you have real users covers how to size against the mix.
Every FHIR deployment eventually has this conversation. Having it before an incident is much cheaper than having it after.
Sources
- HL7 FHIR Bulk Data Access IG covering export workloads that - HL7 FHIR Bulk Data Access IG covering export workloads that contend with interactive traffic
Related Posts
