You are asked to size a FHIR API before it has any real users. The traffic is hypothetical, the workflows are being redesigned as you plan, and the partner integrations exist only as slides. That is a hard exercise, and the shortcut most teams reach for is to guess a total RPS number and pad it. The number that survives longer than the guess is a concurrency prediction rather than a peak RPS number.
Working from concurrency instead of RPS is the discipline that keeps sizing honest. For related walkthroughs, the FHIR fundamentals hub collects the surrounding material.
Concurrency Is Different From RPS
RPS measures how many requests arrive per second. Concurrency measures how many requests are in flight at any moment. A FHIR API that handles two hundred RPS with an average latency of fifty milliseconds carries roughly ten concurrent requests. The same RPS with two-second latency carries four hundred.
The number that determines whether the server survives is the concurrent one. Queue depth, connection pool utilization, and thread pool saturation all scale with concurrency, not with RPS alone. A pass through the site's concurrency p99 predictor turns a target RPS and engine choice into utilization and p99 projections.
Little's Law Is the Cheat Code
Little's Law says in-flight requests equal throughput times average latency. For a FHIR API:
- L (in-flight) = λ (RPS) × W (mean latency)
Working the equation backward gives useful bounds. If your target latency is one hundred milliseconds and your target RPS is five hundred, expected in-flight is fifty. If the connection pool has thirty slots, that number is a problem before anyone deploys.
Little's Law is not exact under bursts, but it is precise enough for planning before real users arrive.
Active Users Do Not Equal Concurrent Requests
Every plan meeting asks how many users the system will have. That is the wrong question for sizing. The right question is how many requests those users will fire concurrently.
An active user during a chart review fires a request every ten seconds. Two hundred active users produce twenty concurrent requests, not two hundred. For the specific split, why active users are not the same as concurrent requests covers the shape.
The Queue Is the First Thing to Watch
Once concurrent requests exceed a system's capacity, the queue grows. Queue depth is the leading indicator of trouble because latency lags behind it. Sizing that only watches latency notices trouble later than sizing that watches queue depth.
Every FHIR API should have queue depth on the dashboard from day one. For the operational shape, queue depth as the leading indicator of trouble covers the pattern.
Bracket the Prediction With a Confidence Range
A single concurrency number is a lie in a hopeful disguise. A bracket is honest. Predict a low, a mid, and a high concurrency number for the target RPS, name the assumptions behind each, and defend the bracket rather than the point estimate.
The bracket tightens as real usage data arrives. Predictions that never tightened were never useful.
Run the Scenarios Before Real Users
The predictions become usable through synthetic load tests that mimic realistic workload shapes. Baseline, spike, sustained, and mixed workloads each exercise different parts of the system. For the specific scenarios, concurrency test scenarios worth running before go-live is the accompanying reference.
Testing catches the sizing errors that spreadsheet math cannot. Testing without prediction produces numbers that describe the test setup, not the future workload. The two work together.
Predicting concurrency before real users is uncomfortable and useful in equal measure. Every FHIR API that lands well starts with a bracket, tests against it, and tightens over the first weeks of real traffic.
Sources
- HL7 FHIR core specification of HTTP interactions covering - HL7 FHIR core specification of HTTP interactions covering read/write semantics behind concurrency planning
Related Posts
