Integration Styles

Hohpe and Woolf's Enterprise Integration Patterns define four fundamental styles. File transfer: one application writes a file and another reads it later, usually in batches over SFTP or object storage. Shared database: applications integrate by reading and writing the same tables. Remote procedure invocation, today usually a REST or gRPC API: one application calls another and waits for a response. Messaging: applications exchange messages asynchronously through a broker such as Kafka, RabbitMQ or a JMS provider. They differ in timeliness, coupling and failure behaviour, and each solved problems the previous one created. None is obsolete; real landscapes mix all four, plus hybrids such as change data capture and claim check.

The main drawback is latency: data moves in batches, often nightly or hourly, so receivers work on stale data and anything that needs near-real-time updates cannot use it. Errors also surface late. A malformed record, a missing file or a half-written file is discovered only when the import runs, possibly hours after the source produced it, and the batch may fail partially. Contracts such as CSV layouts are often undocumented, and duplicates or reprocessing need explicit handling. The strengths are real: simple, decoupled in technology and time, and ideal for bulk volumes and partners with limited capabilities. So the drawback matters when timeliness or early error feedback is a requirement.

Because the schema becomes a public contract between all applications using it, but nobody manages it as one. Any change, a column rename, a new constraint, even an index, can break or slow other applications, so the schema freezes and every change needs cross-team coordination. Business rules are not encapsulated: each application writes the tables directly, so validation and invariants are duplicated or bypassed, and it is unclear who owns the data. Runtime coupling is high too: one application's heavy query or lock causes contention for all of them, as in this note's outage example. Read-only views or replicas reduce the damage; APIs, events or CDC are the long-term exit.

An API is better when the caller needs an immediate answer to continue: a price quote, a stock check, a payment authorisation or a validation result, or when a user is waiting for a read. Request-response over REST or gRPC gives the answer directly, with simple error handling and debugging, whereas request-reply over a broker adds correlation ids, reply queues and latency for no benefit. APIs also suit queries over data another service owns when copying it is not justified. The price is temporal coupling: the provider must be available and fast, so callers need timeouts, retries for idempotent calls and circuit breakers. For propagation to many consumers, messaging wins.

Messaging adds temporal decoupling: the sender hands a message to the broker and continues, and the consumer processes it when it is available, so an outage or slowdown of the consumer does not fail the sender. The broker buffers load spikes, letting consumers work at their own pace and scale out with partitions or competing consumers. It provides reliable delivery through persistence, acknowledgements, retries and dead-letter queues. Publish-subscribe makes fan-out easy: new consumers subscribe without any change to the producer. The costs are eventual consistency, at-least-once delivery that requires idempotent consumers, ordering concerns and harder end-to-end debugging, so trace context must travel in message headers.

Change data capture reads a database's own change log, the PostgreSQL WAL or the MySQL binlog, and turns committed inserts, updates and deletes into an event stream, typically with Debezium publishing to Kafka. It is a hybrid of database integration and messaging: the source application keeps writing to its database unchanged, while consumers receive events asynchronously instead of querying shared tables. It fits legacy systems you cannot modify, cache and search-index synchronisation, and strangler migrations. Combined with an outbox table, it gives reliable event publishing without dual writes. Be careful with raw table events: they expose the internal schema, so publish curated domain events through the outbox where possible.

I make every step explicit and recoverable. The producer writes to a temporary name and renames atomically when the file is complete, then writes a completion marker or manifest containing the record count and a SHA-256 checksum. The consumer processes only files with a marker, verifies the checksum and count, and validates the content against a documented schema before applying anything. Imports are idempotent, keyed by file name or business key, so reruns are harmless, and destructive inferences such as deleting missing products require a complete, validated file. Processed files move to an archive folder and rejected ones to an error folder, with alerts. Spring Integration or Spring Batch implements these patterns well.

I start with an inventory of integration needs rather than a preferred technology. For each integration I record the timeliness required, whether the caller needs an immediate response, how many consumers there are, data volume, coupling tolerance, failure behaviour and any partner constraints. Then I choose per integration: APIs for immediate queries and commands, messaging for propagation and fan-out, files for bulk exchange and constrained partners, and CDC or read-only views as bridges for legacy databases, while avoiding new shared-database integrations. Each choice is captured in an ADR. Finally I standardise one supported platform per style, with contracts in OpenAPI and AsyncAPI, so teams do not reinvent tooling.