Technology — Cross-stream ML Pipeline Leak Detection

End-to-end data flow

Field sensor → cloud ML → control-room alert.

Pressure is sampled at 100 Hz on a Raspberry-Pi-based field unit at each upstream and downstream valve. The data streams over cellular to AWS, where a dedicated ML model per pipeline segment continuously compares predicted downstream pressure against actual. Deviation outside the model's learned envelope becomes an alert with magnitude, duration, and (when both endpoints see it) a GPS estimate.

┌─────────────────────────────────────────────────────────────┐ │ FIELD LAYER │ │ Raspberry Pi + Keller 100 Hz digital pressure transducer │ │ GPS · vibration sensor · cellular (Verizon / AT&T) │ └──────────────────────────────┬──────────────────────────────┘ │ MQTT (TLS) ▼ ┌─────────────────────────────────────────────────────────────┐ │ CLOUD INGESTION (AWS, us-east-1) │ │ Amazon MQ (TLS MQTT broker) │ │ mqttio service → RDS leakmon-raw (TiDB-grade time series) │ └──────────────────────────────┬──────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ MODEL CLUSTER (Docker Swarm) │ │ One dedicated ML service per pipeline segment │ │ TensorFlow inference of EI Combined v3 model │ │ predicts downstream pressure given upstream stream │ │ delta > learned threshold → analyst alert published │ └──────────────────────────────┬──────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ ALERT & INTEGRATION LAYER │ │ Dashboard (Traefik/ALB) · Modbus TCP · OPC UA · REST │ │ email / SMS notification · SCADA register mapping │ └─────────────────────────────────────────────────────────────┘

Sampling

100 Hz pressure. Two orders of magnitude above most CPM systems.

Most volume-balance and acoustic systems sample pressure at 1 Hz or slower. That is fast enough to see a steady-state leak — and far too slow to see a fast pressure-drop signature before the wave has propagated past your measurement window.

EI samples at 100 Hz using the Keller digital transducer, giving each ML model 100 datapoints to fit per second per sensor. With 145+ valve streams in the rolling 42-day retention window, the platform ingests 1.28 billion pressure samples a day and the model has enough temporal resolution to characterize the full waveform of every transient — not just its peak.

Pressure waveform showing the negative pressure wave following a leak

ML architecture

EI Combined v3 — cross-stream prediction, not single-point thresholding.

Endpoint Integrity's model architecture is purpose-built for pipeline pressure dynamics: a deep encoder-decoder that takes both upstream and downstream pressure streams as input and learns the pipeline's transfer function — how a pressure wave introduced at one end propagates to the other.

⤺

Cross-stream input

Both upstream and downstream pressure feed the model — so it learns the relationship between sensors, including delay, attenuation, and reflection characteristics specific to this pipe.

∿

Time-delay Embedding Differential (TiED)

The TiED layer analyzes the time-shifted relationship between upstream and downstream waveforms. Genuine leaks produce a characteristic asymmetric signature that operational transients do not.

⛞

Per-segment model

Each pipeline segment gets a dedicated model trained on its own flow history. There is no generic, one-size-fits-all classifier — every model has learned the pump curve, viscosity, and geometry of its own line.

Leak localization

From timestamp to GPS coordinate.

When a release occurs, the negative pressure wave (NPW) it generates propagates both upstream and downstream at close to the speed of sound in the fluid. EI records the precise NPW arrival timestamp at each station. The time-difference-of-arrival, combined with the known fluid wave speed and pipeline length, locates the leak.

Pipeline: [station A]═══════════════════════════════[station B] ▲ release point NPW arrives at A at t_A NPW arrives at B at t_B Distance from A = ( L + (t_A - t_B) · v ) / 2 L = total length v = wave speed (product-specific) Wave speeds used: Natural gas: ~350 m/s NGL / Y-grade: ~1000 m/s Crude oil: ~1200 m/s Refined products: ~1100 m/s

Estimated leak coordinates are rendered directly on the operator's pipeline map alongside the alert. Routes are loaded from KMZ files — a typical line like the GT Pipeline has 852 coordinate points along its centerline for accurate visual placement.

EI Sentinel hardware

Customer-installable. No mod/demob. No site visit for software.

The EI Sentinel (successor to the EI Sentinel) field unit ships as a sealed enclosure ready to bolt onto an existing measurement-point manifold. Flash the SD card image, plug in sensors, approve the device in the dashboard. Data flows immediately.

100 Hz Keller digital pressure transducer — USB, drop-in calibration
Raspberry Pi compute core — leakmon-2026 image, auto-provisioning
GPS + vibration sensors — environmental + location context
Verizon or AT&T cellular — automatic carrier failover
12–24 VDC input — direct off existing solar/battery rails
Small footprint — fits inside standard valve cabinets
OLED status display — uptime, signal, GPS-fix at a glance
Self-provisioning — boot → discover sensors → dashboard approval
Store-and-forward (roadmap) — local buffer through cellular outages
Remote management (roadmap) — reboot + config update over MQTT

Integration

Talk to the SCADA you already operate.

Endpoint Integrity is a CPM system, not a replacement console. Every alert and pressure value is exposed on the standard industrial protocols your control room already polls. Drop our register map into your existing Modbus or OPC UA configuration and the alarms appear in your existing screens.

Modbus TCP — standard holding registers for alarm state, pressure, pipeline health
OPC UA — modern OPC Unified Architecture server for newer SCADA stacks
REST API — JSON over HTTPS for custom integrations and historians
Email / SMS — analyst-configurable destination list per alert type
Webhook — drive PagerDuty, Slack, Opsgenie on PCR alerts
CSV / Parquet export — for audit binders + retroactive analytics

Pressure waves, interpreted by ML.