> For the complete documentation index, see [llms.txt](https://spice.docs.medtroniclabs.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://spice.docs.medtroniclabs.org/deploy/deployment-guide/server.md).

# Backend Services

This page explains how to build, package and run the SPICE backend. The backend is a set of Java (Spring Boot) microservices that share a PostgreSQL database, a Redis cache and, for the offline sync service, a RabbitMQ broker. For the full list of environment variables, see [Configuration](/deploy/deployment-guide/configuration.md). For a ready-made single-host stack, see [Docker Compose](/deploy/deployment-guide/docker-compose.md).

## Services

| Service                | Role                                                                              | Default port |
| ---------------------- | --------------------------------------------------------------------------------- | ------------ |
| `auth-service`         | Login, session and token issuing; the only service that signs internal assertions | 8089         |
| `user-service`         | Users, organisations and locations; owns the database schema (Flyway)             | 8085         |
| `admin-service`        | Administrative and configuration APIs                                             | 8086         |
| `spice-service`        | Core clinical APIs (patients, assessments, static data)                           | 8087         |
| `cql-service`          | Clinical Quality Language rules and value sets                                    | 8094         |
| `notification-service` | E-mail and SMS notifications                                                      | 8084         |
| `offline-service`      | Offline synchronisation for the mobile app                                        | 8093         |
| `fhir-mapper`          | Translates platform data to and from FHIR                                         | 8092         |
| `cfr-service`          | Report and registry APIs (reads the reporting database)                           | 8050         |
| `common-service`       | Shared library used by all of the above. It is not a runnable service             | n/a          |

The FHIR server (HAPI FHIR) is a separate project; see [FHIR Server and Mapper](/deploy/deployment-guide/fhir-server-and-mapper.md). The reference reverse proxy expects it on port 8091.

Each service listens on the port above with the servlet context path `/`. The proxy adds the per-service URL prefix (for example `/spice-service/`) and strips it before forwarding.

## Prerequisites

| Requirement                           | Version                                                                                    | Notes                                                                             |
| ------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------- |
| JDK                                   | 25                                                                                         | `java.version` is 25 in every `pom.xml`, and the runtime images use a Java 25 JRE |
| Apache Maven                          | 3.9 or later                                                                               | `offline-service` and `fhir-mapper` also include a Maven wrapper (`mvnw`)         |
| Spring Boot                           | 3.5.16                                                                                     | Managed by the POMs; nothing to install                                           |
| PostgreSQL                            | See [Components and Requirements](/deploy/deployment-guide/components-and-requirements.md) | Three databases are used (see [Databases](#databases))                            |
| Redis                                 | See [Components and Requirements](/deploy/deployment-guide/components-and-requirements.md) | Used by every service                                                             |
| RabbitMQ                              | See [Components and Requirements](/deploy/deployment-guide/components-and-requirements.md) | Required by `offline-service`                                                     |
| Docker (or another OCI image builder) | Current stable                                                                             | Only needed to produce container images                                           |

> Some service READMEs still say Java 21. The build files and Dockerfiles are the reference: use JDK 25.

## Get the source

Clone the backend repository and change into it. All commands below run from the `backend` directory, which contains the aggregator `pom.xml` and one directory per service.

```bash
git clone <your-backend-repository-url> backend
cd backend
```

## Build

The shared library `common-service` must be built and installed into your local Maven repository before any other service, because every service depends on it (`com.mdtlabs.coreplatform:common-service:0.0.2-SNAPSHOT`).

1. Build and install the common library:

   ```bash
   mvn -pl common-service clean install -DskipTests
   ```
2. Build the remaining services. The aggregator `pom.xml` lists every module (`common-service`, `user-service`, `spice-service`, `notification-service`, `offline-service`, `fhir-mapper`, `cfr-service`), so a single command builds all of them in dependency order:

   ```bash
   mvn clean install -DskipTests
   ```

   To build selected modules only (`-am` also builds `common-service`), for example the core service group:

   ```bash
   mvn -pl spice-service/admin-service,spice-service/spice-service,spice-service/cql-service -am clean install -DskipTests
   ```
3. Each runnable module produces a single executable JAR in its own `target/` directory:

   | Module directory              | JAR location   |
   | ----------------------------- | -------------- |
   | `user-service/auth-service`   | `target/*.jar` |
   | `user-service/user-service`   | `target/*.jar` |
   | `spice-service/admin-service` | `target/*.jar` |
   | `spice-service/spice-service` | `target/*.jar` |
   | `spice-service/cql-service`   | `target/*.jar` |
   | `notification-service`        | `target/*.jar` |
   | `offline-service`             | `target/*.jar` |
   | `fhir-mapper`                 | `target/*.jar` |
   | `cfr-service`                 | `target/*.jar` |

Run the unit tests with `mvn test` (or `mvn verify` for coverage reports) before you release a build; remove `-DskipTests` from the commands above to run them as part of the build.

## Build container images

Every runnable module has a `Dockerfile` that copies `target/*.jar` into an `eclipse-temurin:25-jre-jammy` image, so build the JARs first. The build context is the directory that holds the `Dockerfile`, and each image exposes port 80 in its metadata while the application listens on its own port from the table above (publish that port).

```bash
docker build -t spice/auth-service:<tag>         user-service/auth-service
docker build -t spice/user-service:<tag>         user-service/user-service
docker build -t spice/admin-service:<tag>        spice-service/admin-service
docker build -t spice/spice-service:<tag>        spice-service/spice-service
docker build -t spice/cql-service:<tag>          spice-service/cql-service
docker build -t spice/notification-service:<tag> notification-service
docker build -t spice/offline-service:<tag>      offline-service
docker build -t spice/fhir-mapper:<tag>          fhir-mapper
docker build -t spice/cfr-service:<tag>          cfr-service
```

Optionally pass `--build-arg GIT_SHA=<commit>` to stamp the image. The value is returned by `GET /actuator/info`, which is useful for confirming which build is running. Without it the value is `unknown`.

Push the images to a registry that your cluster or hosts can pull from. Use a unique tag for every build: re-using a tag usually means the orchestrator does not pull the new image.

`fhir-mapper` starts its JVM with `-XX:MaxRAMPercentage=75` and exits on out-of-memory so that the orchestrator restarts it. Give it a container memory limit (the reference stack uses 3 GB) and let the JVM size its heap from that limit.

## Databases

The services use three logical PostgreSQL databases. Create them before the first start. The reference stack names the FHIR and report databases `fhir_db` and `report_db`; you can choose any names as long as the connection settings match.

| Database          | Used by                                                           | Schema is created by                                                |
| ----------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------- |
| Platform database | auth, user, admin, spice, cql, notification, offline, fhir-mapper | Flyway migrations that run inside `user-service`                    |
| `fhir_db`         | FHIR server                                                       | The FHIR server itself, on start-up                                 |
| `report_db`       | `cfr-service` (schemas `core` and `report`)                       | The reporting ETL schema scripts. `cfr-service` has Flyway disabled |

```bash
createdb -h <db-host> -U <db-owner> <platform-database>
createdb -h <db-host> -U <db-owner> <fhir-database>
createdb -h <db-host> -U <db-owner> <report-database>
```

## Database migrations and initial data

Only `user-service` runs Flyway. On start-up it applies every script under `db/migration` (bundled in the JAR) to the platform database, using `DATABASE_URL`, `DATABASE_USERNAME` and `DATABASE_PASSWORD`. All other services expect the schema to exist and do not create tables, so **start `user-service` first on a new database and wait until it is healthy**.

Things to know:

* The first migration creates the `pgcrypto` extension. The database user must be allowed to create it, or a database administrator must run `CREATE EXTENSION IF NOT EXISTS pgcrypto;` beforehand.
* Flyway is configured with `baseline-on-migrate` and `out-of-order` enabled, so it can adopt an existing database and apply migrations that were added out of sequence.
* The baseline migration inserts the `SUPER_USER` and `JOB_USER` roles, a bootstrap super-user account and its role assignment. **Change this account's credentials (or replace the account) before exposing the system.** Do not use the seeded values in a real deployment.
* The job user is a technical account used by background work. Set `JOB_USER_NAME` and `JOB_USER_PASSWORD` on `offline-service`, `fhir-mapper` and `spice-service` to match the job user that exists in your database.
* Some migrations load reference data (locations, forms, menus, roles and permissions). Review them against your own programme before going live.

For loading your own facility, user and patient data, see [Data Import and Migration](/deploy/deployment-guide/data-import-and-migration.md).

## Runtime configuration

Configure every service through environment variables; the full tables are in [Configuration](/deploy/deployment-guide/configuration.md). The settings below are the ones that must be correct for the services to work together.

**All services**

* `DATABASE_URL`, `DATABASE_USERNAME`, `DATABASE_PASSWORD`: JDBC URL (for example `jdbc:postgresql://<db-host>:5432/<platform-database>`) and credentials.
* `REDIS_HOST`, `REDIS_PORT`: Redis endpoint. `cfr-service` also reads `REDIS_PASSWORD`; provide a value (it may be empty if Redis has no password).
* Service URLs: `AUTH_SERVICE`, `USER_SERVICE`, `ADMIN_SERVICE`, `SPICE_SERVICE`, `NOTIFICATION_SERVICE`, `OFFLINE_SERVICE`, `CQL_SERVICE`, `CFR_SERVICE`, `FHIR_SERVICE`. Each is the base URL (for example `http://<host>:<port>`) that the other services use to reach it. Set only the ones a service calls; the reference `docker-compose.yml` shows which service needs which.
* `INTERNAL_ASSERTION_PRIVATE_KEY` (auth-service only) and `INTERNAL_ASSERTION_PUBLIC_KEY` (every other service): a matching ES256 key pair, base64-encoded DER (private key PKCS#8, public key X.509). Services pass a short-lived signed assertion to each other, so a missing or mismatched public key causes failures between services. Generate your own pair and keep the private key in a secret store.
* `ALLOWED_ORIGINS`, `DOMAIN_NAME`, `ENVIRONMENT`: see [Reverse proxy, CORS and cookies](#reverse-proxy-cors-and-cookies).
* `AUTH_TOKEN_TIME_LIMIT_IN_MINUTES`: session lifetime; defaults to 240 in `user-service` and `auth-service`.
* `SWAGGER_ENABLED`: API documentation is off by default. Leave it off in production.

**Service-specific**

| Service                | Must set                                                                                                                                                                                                                                                            |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `user-service`         | `EMAIL_APP_URL` (links in e-mails), `SMS_RESET_PASSWORD_URL`, `APP_VERSION`; password hashing flags are off by default and are described in the service's `docs/PASSWORD_HASHING_ROLLOUT.md`                                                                        |
| `spice-service`        | `HAPI_SERVER_URL`/`FHIR_SERVICE` reachability, `VERSION_CODE` (minimum accepted mobile app version code, see [Mobile App](/deploy/deployment-guide/android.md)), `APP_VERSION`, `JOB_USER_NAME`, object-storage settings if you use uploads                         |
| `admin-service`        | Map-service API key if map features are used                                                                                                                                                                                                                        |
| `offline-service`      | `JOB_USER_NAME`, `JOB_USER_PASSWORD`; message queue: set `IS_RABBITMQ_SQS_ENABLED=true` and the `RABBIT_MQ_*` settings; blob storage: `OFFLINE_STORAGE_PROVIDER` (a local-filesystem mode with `OFFLINE_STORAGE_LOCAL_PATH` is available, otherwise object storage) |
| `notification-service` | `MAIL_HOST`, `MAIL_USER`, `MAIL_PASSWORD`, `MAIL_FROM`, `MAIL_SEND`; enable exactly the SMS provider you use                                                                                                                                                        |
| `fhir-mapper`          | `HAPI_SERVER_URL`, `HAPI_IDENTIFIER_URL`, `JOB_USER_NAME`, `JOB_USER_PASSWORD`                                                                                                                                                                                      |
| `cql-service`          | `HAPI_SERVER_URL`, `CQL_PATH` and `VALUESET_PATH` (directories holding the CQL and value-set files; the reference stack mounts the bundled `anc/cql` and `anc/valueset` resources)                                                                                  |
| `cfr-service`          | `REPORT_DB_URL`, `REPORT_DB_USERNAME`, `REPORT_DB_PASSWORD`; `API_PERMISSION_DB_*` (defaults to the platform database)                                                                                                                                              |

Keep secrets (database passwords, mail credentials, key material) out of images and source control. Inject them from your orchestrator's secret mechanism.

## Start-up order

1. PostgreSQL, Redis and RabbitMQ.
2. FHIR server (HAPI FHIR). `spice-service`, `cql-service` and `fhir-mapper` call it.
3. `user-service` (applies the Flyway migrations; wait until its health check passes).
4. `auth-service`.
5. `admin-service`, `spice-service`, `cql-service`, `notification-service`, `fhir-mapper`, `offline-service` and `cfr-service`, in any order.
6. The reverse proxy, once every upstream resolves (nginx refuses to start if an upstream host name cannot be resolved).

Starting in this order avoids connection errors on a first boot.

### Run a service from a JAR

For a quick check without containers, export the variables and run the JAR:

```bash
export DATABASE_URL=jdbc:postgresql://<db-host>:5432/<platform-database>
export DATABASE_USERNAME=<db-user>
export DATABASE_PASSWORD=<change-me>
export REDIS_HOST=<redis-host>
export REDIS_PORT=6379
# ...plus the service-specific variables above
java -jar user-service/user-service/target/user-service.jar
```

### Run as containers

```bash
docker run -d --name user-service -p 8085:8085 \
  --env-file ./user-service.env \
  spice/user-service:<tag>
```

Create one env file per service from the variables in [Configuration](/deploy/deployment-guide/configuration.md). In Kubernetes, map the same variables to a ConfigMap and a Secret and expose each service through a `ClusterIP` Service.

## Health checks

Every service exposes two unauthenticated actuator endpoints: `/actuator/health` and `/actuator/info`. Other actuator endpoints are not exposed.

```bash
curl -s http://<service-host>:8085/actuator/health
curl -s http://<service-host>:8085/actuator/info
```

`health` reports whether the service is up. `info` returns the commit SHA stamped at image build time. Use `/actuator/health` for container or Kubernetes readiness and liveness probes. Through the reference proxy the same endpoints are available at `/<service-name>/actuator/health`, for example `/user-service/actuator/health`.

## Reverse proxy, CORS and cookies

The web applications and the mobile app reach all services through one public host name. The proxy routes by path prefix and removes the prefix before forwarding.

| Public path prefix       | Upstream                        |
| ------------------------ | ------------------------------- |
| `/auth-service/`         | auth-service, port 8089         |
| `/user-service/`         | user-service, port 8085         |
| `/admin-service/`        | admin-service, port 8086        |
| `/spice-service/`        | spice-service, port 8087        |
| `/cql-service/`          | cql-service, port 8094          |
| `/notification-service/` | notification-service, port 8084 |
| `/offline-service/`      | offline-service, port 8093      |
| `/fhir-mapper/`          | fhir-mapper, port 8092          |
| `/fhir-server/`          | FHIR server, port 8091          |
| `/cfr-service/`          | cfr-service, port 8050          |

Example nginx location (repeat for each service, changing the prefix and upstream):

```nginx
location /spice-service/ {
    rewrite ^/spice-service/(.*)$ /$1 break;
    proxy_pass http://spice_service_upstream;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
}
```

Settings that matter:

* **Timeouts and body size.** The reference configuration uses `proxy_connect_timeout`, `proxy_send_timeout` and `proxy_read_timeout` of 300 s and `client_max_body_size 150M`, because sync and import requests can be large and slow.
* **Response header buffers.** The login response carries large token headers. The reference configuration sets `proxy_buffer_size 16k`, `proxy_buffers 8 16k` and `proxy_busy_buffers_size 32k`; without this nginx returns `502 Bad Gateway` on login.
* **FHIR links.** For `/fhir-server/` also send `X-Forwarded-Host`, `X-Forwarded-Proto`, `X-Forwarded-Port` and `X-Forwarded-Prefix /fhir-server`, otherwise the paging links returned by the FHIR server omit the prefix and clients cannot follow them.
* **TLS.** Terminate HTTPS at the proxy or load balancer. The web login cookie is `Secure`, so the web applications only work over HTTPS.
* **Cookie and CORS.** For web clients `auth-service` sets an `HttpOnly`, `Secure` cookie named `AuthCookie`, scoped to the domain in `DOMAIN_NAME`, with `SameSite=Strict`. It allows credentialed cross-origin requests from the origins listed in `ALLOWED_ORIGINS` (a comma-separated list of exact origins, for example `https://admin.example.org,https://reports.example.org`). Therefore:
  * Serve the web applications and the API from sub-domains of one registrable domain and set `DOMAIN_NAME` to that domain.
  * List every web application origin in `ALLOWED_ORIGINS`. Avoid `*` in production.
  * Setting `ENVIRONMENT` to `dev` switches the cookie to `SameSite=None`; use that only for local development.
* **Upstreams.** In a container network, use service names as upstream hosts, and make sure every upstream is resolvable when nginx starts.

## Verify the deployment

1. Check every service's `/actuator/health` returns an `UP` status.
2. Confirm `user-service` applied its migrations: its log ends the Flyway step without errors, and the platform database contains the Flyway history table (`flyway_schema_history`).
3. From outside the proxy, call a health endpoint through a prefix, for example `https://<api-host>/user-service/actuator/health`.
4. Sign in through the [Web Applications](/deploy/deployment-guide/admin-portal.md) with the account you set up, and confirm the location and menu data loads.
5. Check the logs of `offline-service` and `fhir-mapper` for connection errors to RabbitMQ and the FHIR server.
