CONSULT CIRCLE | OPERATIONS
Integrations fail quietly. What to measure, how to instrument, and the playbooks that turn an alert into a resolution.
Integration failures are unusually costly because they are unusually quiet. A web server that stops responding is noticed in minutes. A nightly file transfer that silently stops is noticed when someone asks why the figures look wrong, and by then you have days of missing data and no clear idea when it stopped.
The distinction that matters: monitoring tells you a known thing has gone wrong; observability lets you work out why something you did not anticipate is happening. Integration estates need both.
The KPIs worth measuring
| KPI | What it tells you | Why it matters |
|---|---|---|
| Throughput — messages or records per interval | Volume moving through the flow | A drop to zero is the clearest signal of silent failure |
| Latency — end to end | How long a record takes from source to destination | Rising latency precedes failure and is visible earlier |
| Error rate | Proportion of records failing | Distinguishes a broken flow from a data quality problem |
| Queue depth or backlog | Work waiting to be processed | The earliest indicator that consumption cannot keep pace |
| Success rate against expectation | Actual runs against scheduled runs | Catches the flow that did not start at all, which throughput alone can miss |
| Dead letter volume | Records that failed and were set aside | Frequently accumulates unnoticed until someone asks about missing data |
| Reconciliation variance | Source record count against destination count | The definitive check that data arrived, and complete |
| Time since last successful run | Freshness | The single most useful alert for scheduled and batch flows |
Table 1 — Integration KPIs.
If you only implement one thing: Alert on time since last successful run, per flow. It catches the silent stoppage, which is the failure mode that does the most damage precisely because nothing is generating errors.
Instrumenting properly
Three signals, and one identifier that makes them useful together.
- Metrics. Numeric series over time — throughput, latency, error counts, queue depth. Cheap to store and what you alert on.
- Logs. Structured event records with enough context to explain what happened to a specific record. Structured, so they can be queried rather than read.
- Traces. The path of a single transaction across systems, showing where time was spent and where it failed. Invaluable in multi-hop flows.
- Correlation identifier. A single identifier attached at ingestion and carried through every system and log line. Without it, investigating a specific failed record across five systems is manual archaeology.
The retrofit problem: Correlation identifiers are cheap to design in and expensive to add later, because every component in the chain has to change. If you are building or replacing an integration, add one now even if nothing consumes it yet.
Tooling categories
| Category | What it covers | Consider when |
|---|---|---|
| Platform-native monitoring | Whatever your integration platform provides out of the box | Always start here; it is included and understands the platform |
| Metrics and time-series platforms | Collection, storage, dashboards and alerting on numeric signals | You need trend visibility and alerting across many flows |
| Log aggregation | Centralised, searchable, structured logs | Investigation currently means logging into several systems |
| Distributed tracing | End-to-end transaction paths across services | Flows cross several systems and latency problems are hard to localise |
| Synthetic transactions | A test record pushed through the flow on a schedule | You need to know the flow works even when no real traffic is passing |
| Business-level reconciliation | Counts and totals compared between source and destination | Correctness matters as much as availability, which for finance data it does |
Table 2 — Tooling categories for integration observability.
Synthetic transactions deserve particular attention for low-volume flows. If a flow runs twice a day, absence of errors tells you very little; a synthetic record proves the path still works.
Playbooks
An alert without a playbook produces a person searching for context at three in the morning. Each playbook should be short enough to follow under pressure.
Playbook: flow has stopped
- Confirm scope — one flow or several? Several suggests a shared dependency rather than a flow problem.
- Check the source: is it producing? A stopped flow is often an upstream system that has nothing to send.
- Check connectivity and credentials. Expired certificates and rotated passwords are the most common causes.
- Check the destination is accepting. Full disks, locked accounts and schema changes all present as a stopped flow.
- Check for a backlog. If work has queued, plan the catch-up before restarting, or you may overwhelm the destination.
- Restart, then verify with a synthetic transaction rather than assuming.
- Reconcile what was missed during the outage and replay it deliberately.
Playbook: error rate rising
- Determine whether errors are concentrated in one record type or spread evenly. Concentration points to data; spread points to infrastructure.
- Sample the failing records and read the actual error rather than the summary count.
- Check for a recent change at either end — schema, validation rules, a release.
- Decide whether to stop the flow. Continuing with a high error rate can propagate bad data faster than stopping does damage.
- Isolate failed records to the dead letter destination so good data continues to flow.
- Fix, replay the failed records, and reconcile.
Playbook: latency increasing
- Establish whether it is one stage or the whole path. Tracing answers this in seconds if instrumented.
- Check queue depth. Rising latency with rising backlog means consumption is behind production.
- Check whether volume has increased or capacity has decreased.
- Look for a slow dependency — an external service or a database query is the usual answer.
- Decide whether to scale, throttle upstream, or accept and monitor.
Common mistakes
- Monitoring only for errors, so the flow that stops silently is invisible.
- Alerting on every anomaly until the alerts are ignored, which is worse than not alerting.
- No correlation identifier, making cross-system investigation manual.
- Dead letter queues nobody reviews, quietly accumulating lost records.
- Dashboards without playbooks, so alerts lead to searching rather than to action.
- No business-level reconciliation, so a flow that runs successfully but delivers incomplete data looks healthy.
Related reading
- Managed services vs self-managed in the cloud
- RFP template and evaluation criteria for managed IT services
- Monitoring as a Service
Frequently asked questions
What is the difference between monitoring and observability?
Monitoring tells you whether known conditions are met — is it up, is the error rate above a threshold. Observability is the ability to investigate questions you did not anticipate, using metrics, logs and traces together. Integration estates need both.
What should we alert on for integration flows?
Time since last successful run is the highest-value alert, because it catches silent stoppage. Add error rate above a threshold, queue depth growing consistently, and reconciliation variance.
How do we monitor a flow that runs infrequently?
Use synthetic transactions. Push a test record through on a schedule so you learn the path is broken before the next real run rather than after it.
What is a correlation ID and why does it matter?
A unique identifier attached to a transaction at ingestion and carried through every system it touches. It lets you trace one record end to end. Cheap to design in, expensive to retrofit.
What is a dead letter queue?
A destination for records that could not be processed, so one bad record does not block the flow. It needs active review, because unreviewed dead letter queues are where data quietly disappears.
Talk to Consult Circle
Our Monitoring as a Service covers integration flows as well as infrastructure, including KPI definition, alerting and playbooks.
Book a free 30-minute call - 0203 916 5593 - info@consultcircle.com