FHIR Server & Mapper
SPICE stores clinical data as FHIR R4 resources. Two services are involved:
FHIR server
A HAPI FHIR JPA server (HAPI 8.x starter project) backed by PostgreSQL, with SPICE additions: token validation, extra search parameters, database migrations and optional administrative-area partitioning. The REST API is served under /fhir.
8080 (set with FHIR_SERVER_PORT)
FHIR mapper
Translates the platform's internal data into FHIR bundles and posts them to the FHIR server. It also registers SPICE's custom search parameters on start-up.
8092
Deploy the FHIR server first, then the FHIR mapper. The other backend services (authentication, user, admin, offline sync, clinical workflow) are described in Backend Services.
Prerequisites
A PostgreSQL database for the FHIR server, managed or self-hosted. The reference compose file uses PostgreSQL 16. Keep it separate from the platform database used by the other services.
The authentication service reachable from the FHIR server (it validates tokens).
For the FHIR mapper: the platform PostgreSQL database, Redis, and the authentication, user, admin, clinical workflow and CQL services (see Connect the mapper).
Java 25 and Maven to build the FHIR server image from source (the Dockerfile uses a Java 25 runtime image). The mapper image is built from its own Dockerfile.
FHIR server
Build and run
The container image packages target/ROOT.war, so build the WAR first, then the image:
cd fhir-server
mvn clean package spring-boot:repackage -Pboot -DskipTests
docker build -t fhir-server .For a quick local trial the repository includes a docker-compose.yml that starts the server and a PostgreSQL container (the credentials in that file are examples only):
On first start the server applies its Flyway migrations (src/main/resources/db/migration). They create the schema, load SPICE's custom search parameters and set up partition support. Hibernate never changes the schema itself: HIBERNATE_HBM2DDL_AUTO defaults to validate, so the server refuses to start if the schema has drifted from what it expects. Leave this setting alone.
Required settings
Set these as environment variables. Use your own secret store for the password.
FHIR_DATASOURCE_URL
JDBC URL of the FHIR PostgreSQL database, for example jdbc:postgresql://<db-host>:5432/<fhir-db>
none (required)
FHIR_DATASOURCE_USERNAME / FHIR_DATASOURCE_PASSWORD
Database credentials. The user must be able to run migrations (create and alter tables).
none (required)
FHIR_SERVER_PORT
HTTP port
none (set it, for example 8080)
APP_VALIDATE_TOKEN_API
URL of the authentication service endpoint that validates bearer tokens
none (required)
FHIR_SERVER_AUTH_ENABLED
Set to false to switch authentication off. For local development only.
true
INTERNAL_ASSERTION_PUBLIC_KEY
Public key (base64 X.509 DER, ES256) used to verify the internal assertion header that the authentication service issues and the mapper forwards. Leave blank to skip.
blank
HAPI_FHIR_CORS_ALLOWED_ORIGIN
Allowed CORS origin. Restrict it in production.
*
HAPI_FHIR_CORS_ALLOW_CREDENTIALS
CORS credentials
false
FHIR_PARTITION_ENABLED, FHIR_PARTITION_REQUEST_TENANT, FHIR_PARTITION_PER_DISTRICT_RES_ID
Partitioning mode. See Administrative-area partitioning.
true, true, true
Authentication works as follows. Every request to /fhir must carry either a valid internal assertion, or a bearer token plus a client header that the authentication service accepts. Anything else gets 401. Do not expose the FHIR server directly to the internet. Only the SPICE services and, optionally, trusted integrations should reach it.
Tuning settings you may need later (all optional):
FHIR_DB_MAX_POOL_SIZE, FHIR_DB_MIN_IDLE
Database connection pool size
100, 10
TOMCAT_THREADS_MAX, TOMCAT_ACCEPT_COUNT, TOMCAT_MAX_CONNECTIONS
Web server threads and queues
200, 100, 2000
HAPI_SEARCH_COORD_CORE, HAPI_SEARCH_COORD_MAX, HAPI_SEARCH_COORD_QUEUE
Search coordinator thread pool
20, 60, 500
HAPI_RETAIN_CACHED_SEARCHES_MINS
How long HAPI keeps cached search results
5
HAPI_REUSE_CACHED_SEARCH_MILLIS
Cached search reuse window. Keep at 0: HAPI does not invalidate a cached search when a resource is written, so a non-zero value can return stale results right after a write.
0
HAPI_FHIR_ADVANCED_LUCENE_INDEXING
Advanced full-text indexing (needs a search backend)
false
The defaults above come from application.yaml. They are starting points, not sizing advice. See Sizing & Operations.
Search parameters and implementation guides
SPICE relies on a few custom FHIR search parameters, for example a national-ID search on Patient and RelatedPerson, an enrolment-location search and an active-diagnosis search on Condition. They are loaded in two ways, and you do not need to load them by hand:
The FHIR server's migrations insert the ones needed for SPICE's own searches.
On start-up, the FHIR mapper registers the search parameters bundled with it (
fhir/searchparams/*.json) through the FHIR API. If it creates any new ones, it asks the server to mark all resources for reindexing. Re-running is safe: parameters that already exist are skipped.
Runtime implementation guide upload is switched off (ig_runtime_upload_enabled: false), and no implementation guides are loaded by default.
Administrative-area partitioning
The FHIR server can split clinical data into partitions, one per administrative area (for example one per district). Resources shared by everyone (practitioners, organisations, code systems and similar) live in a shared partition 0. Clinical resources (households, members, encounters, observations and so on) live in the partition of their area, and requests are routed by a URL path segment such as /fhir/<area-code>/Patient. The partition identifier is the numeric code of the area in the administrative hierarchy. Each area also gets its own block of resource ids, which prepares the ground for moving areas to separate database servers later without renumbering.
Pick one of two modes before the first start against an empty database, and before any data is loaded:
Use when
The programme covers many areas, and you want per-area partitions, partition pruning on large tables, and a path to sharding by area
One national or pilot deployment where you do not need per-area routing
FHIR_PARTITION_ENABLED
true
true
FHIR_PARTITION_REQUEST_TENANT
true
false
FHIR_PARTITION_PER_DISTRICT_RES_ID
true
false
Mapper FHIR_PARTITION_MODE_ENABLED
true
false
Extra work
Onboard each area before it writes (below)
None. Every operation is pinned to the default partition 0.
Why the decision comes first:
The schema is always created in its partitioned form.
partition_idis a mandatory part of the primary keys, so do not setFHIR_PARTITION_ENABLED=false: every write fails. Single-partition mode is selected withFHIR_PARTITION_REQUEST_TENANT=false, not by disabling partitioning.FHIR_PARTITION_REQUEST_TENANTis also read by the database migrations, which decide whether to keep or remove the example area partition seeded by earlier migrations. It must have its final value the first time the server starts.Data written to partitions cannot be moved to a different mode simply by flipping a variable. Changing mode on a database that already holds data is not a supported operation.
The mapper's
FHIR_PARTITION_MODE_ENABLEDmust match: with it on, the mapper posts each bundle to the partition for the facility's area; with it off, it posts to the plain/fhirbase URL. The same switch exists in the CQL service. The reporting ETL has a partition id setting (REPORT_ETL_FHIR_PARTITION_ID) that a single-partition install sets to0.
Onboarding an area (multi-area mode). Before facilities in a new area start writing data, an administrator creates that area's partition and its block of resource ids. The repository provides partitioning/onboard_district.sql, which registers the partition, creates the next resource-id sequence and records the mapping. Edit the area code and name at the top of the script, then run it:
Run it once per area, and before that area's first write. Rows for an area without a partition land in a default catch-all partition that gets no pruning, and moving them out later needs a full table scan and an exclusive lock. Treat rows in the catch-all partition as an alert. The script header also describes registering the partition through the REST operation when the server is already running.
Reader services (clinical workflow, offline sync, CQL) and the reporting ETL also need to know about partitioning. See each service's configuration for its partition setting.
Health checks
The FHIR server exposes Spring Boot actuator endpoints on its HTTP port, outside /fhir and without authentication:
Probe endpoints are enabled, so /actuator/health/liveness and /actuator/health/readiness are available for container orchestration. To check the FHIR API itself, request the capability statement with credentials:
A 401 here with a healthy actuator endpoint means the server is up and authentication is working. A Prometheus metrics endpoint is also enabled in the configuration.
Upgrading HAPI FHIR
The FHIR server tracks the upstream starter project. CUSTOMIZATIONS.md in the repository lists every file SPICE adds or changes. In summary:
Add the upstream project as a git remote, fetch the target release tag and merge it.
Expect merge conflicts only in the files
CUSTOMIZATIONS.mdlists as modified (the application configuration,pom.xml, the Dockerfile, the Helm chart and the partition configuration class). Keep the Flyway dependencies, the authentication interceptor registration and SPICE's partition code.Compare the upstream Dockerfile for version bumps of the build image and the OpenTelemetry agent.
Build and run the tests. Smoke-test that an unauthenticated request returns
401, and boot once in each partitioning mode.Back up the database first. Never edit a migration that has already run anywhere. Add a new one. A major HAPI upgrade can change the database schema.
Backups
Back up the FHIR database with your normal PostgreSQL tooling (see Sizing & Operations). It is the only copy of the clinical record, so treat it as the most important database in the platform. A logical dump looks like this:
Test restores regularly. Restore into an empty database and start the FHIR server against it. If you use native partitioning or per-area resource-id sequences, a restore must include the sequences and the resource-id block mapping table.
FHIR mapper
Required settings
DATABASE_URL, DATABASE_USERNAME, DATABASE_PASSWORD
Platform PostgreSQL database (the same one the other backend services use)
REDIS_HOST, REDIS_PORT
Redis
HAPI_SERVER_URL
Base URL of the FHIR API with a trailing slash, for example http://<fhir-server-host>:8080/fhir/
HAPI_IDENTIFIER_URL
Namespace URL used to build identifier systems (national ID, village ID and so on) and the custom search parameter URLs. Include the trailing slash. Choose it once, before loading any data. Changing it later makes existing identifiers unreachable by lookup. The mapper refuses to start if it is blank.
AUTH_SERVICE, ADMIN_SERVICE, USER_SERVICE, SPICE_SERVICE, CQL_SERVICE
Base URLs of the other platform services
JOB_USER_NAME, JOB_USER_PASSWORD
A service account the mapper signs in with to register search parameters. It must exist in the platform before the mapper starts.
FHIR_PARTITION_MODE_ENABLED
Must match the FHIR server's mode (default false). See above.
INTERNAL_ASSERTION_PUBLIC_KEY
Public key to verify the internal assertion header on inbound calls. Leave blank to skip.
SMART_ANC_ENABLED, LAB_TEST_INTENSIFICATION_LIST, PRESCRIPTION_PREDICTION_DAYS, TB_CONFIRMATION_LAB_TEST_ID
Clinical behaviour settings the configuration requires.
Optional settings: DB_MAX_POOL_SIZE (default 30), SERVER_MAX_HTTP_HEADER_SIZE (default 64 KB; the internal assertion header can be large for users scoped to many facilities), SWAGGER_ENABLED (default false) and the outbound client limits FHIR_CLIENT_MAX_CONCURRENT_REQUESTS (default 80) and FHIR_CLIENT_RETRY_MAX_ATTEMPTS (default 6).
The container image runs the JVM with -XX:MaxRAMPercentage=75 and exits on out-of-memory, so set a container memory limit and let your orchestrator restart it. A heap dump is written to /tmp.
Connect the mapper to the FHIR server and platform services
Start the authentication, user and admin services and the FHIR server first.
Create the job service account in the platform (a user the mapper can sign in as) and set
JOB_USER_NAMEandJOB_USER_PASSWORD.Start the mapper. On start-up it signs in, then checks and registers the custom search parameters. If the FHIR server or authentication service is not ready yet, it retries with exponential backoff (up to 10 attempts by default, roughly three minutes). If all attempts fail, it logs an error and carries on, so check the logs and restart the mapper once the dependencies are healthy.
Confirm the mapper can reach the FHIR server by checking the logs for
Found <n> SearchParameter filesfollowed by either a reindex message orNo new SearchParameters created.
Health check
Common problems
FHIR server fails on start with a schema validation error
The database schema does not match the application. Do not change HIBERNATE_HBM2DDL_AUTO. Check that migrations ran against the right database and that the database user can alter tables.
FHIR server fails on start with a Flyway history error after moving from an older HAPI release
The configuration already ignores history rows whose script is missing (ignore-migration-patterns: "*:missing"). Check that you did not override it.
Every write fails with a not-null violation on partition_id
FHIR_PARTITION_ENABLED was set to false. Set it back to true and use FHIR_PARTITION_REQUEST_TENANT=false for single-partition mode.
Server fails on start in single-partition mode with an error about resource-id blocks
FHIR_PARTITION_PER_DISTRICT_RES_ID is still true. Set it to false.
All FHIR calls return 401
The token validation URL is wrong or unreachable, the client header is missing, or the token has expired. Check APP_VALIDATE_TOKEN_API.
A resource that was just created does not appear in a search
A cached search is being reused. Check HAPI_REUSE_CACHED_SEARCH_MILLIS is 0.
Mapper refuses to start with an identifier base error
HAPI_IDENTIFIER_URL is blank.
Custom searches (such as national ID) return nothing
Search parameters were not registered because the mapper could not reach the FHIR server or sign in. Check the mapper logs, fix connectivity or the job user, and restart the mapper. Reindexing may take time on a large database.
Mapper logs Failed to connect during a FHIR server restart
The mapper retries connection failures for about 15 seconds. A longer restart window needs a higher FHIR_CLIENT_RETRY_MAX_ATTEMPTS, or start the services in order.
Mapper returns Request header is too large
An internal assertion header exceeded the limit. Raise SERVER_MAX_HTTP_HEADER_SIZE on the mapper.
Mapper restarts repeatedly
Out-of-memory under concurrent bundle processing. Increase the container memory limit or lower FHIR_CLIENT_MAX_CONCURRENT_REQUESTS.
FHIR server runs out of database connections
The combined pool sizes of all services exceed PostgreSQL max_connections. Lower the pool sizes or add a connection pooler. See Sizing & Operations.
Last updated