Almost every routing system I've worked on has the same hidden failure point: the distance/time matrix. A single route request is cheap. But a VRP solver asks for the travel time between every pair of stops — an N×N problem that grows quadratically. 200 stops is 40,000 cells. It's where the pretty demo meets real dispatch and quietly falls over.
1. Use the engine's native table service
Don't compute a matrix by firing N² individual route requests. OSRM's /table
endpoint computes the whole grid in a single, highly optimized call. This one change is
usually the difference between seconds and milliseconds.
GET /table/v1/driving/{coordinates}?sources=all&destinations=all&annotations=duration,distance
2. Cap and tile large matrices
Beyond a few hundred points, split the problem: compute sub-matrices in parallel and stitch them together. It bounds the worst-case latency of any single call and lets you spread the work across replicas instead of hammering one.
3. Cache the boring parts
In logistics, a huge fraction of origin/destination pairs repeat day over day — the same depots, the same delivery clusters. A cache keyed on snapped coordinates (rounded to the road network) turns repeat lookups into instant hits and takes real load off the engine.
4. Autoscale on the right signal
Matrix workloads are spiky — quiet, then a dispatch run lights everything up. CPU alone
is a laggy signal. I scale on request concurrency / queue depth with Kubernetes HPA, keep
a warm baseline of replicas, and set sane requests/limits so the
scheduler can actually pack pods.
5. Load test like it's Black Friday
If you haven't load-tested your matrix at 3× peak, you don't know your ceiling — you know your last good day.
Replay real dispatch traffic, watch p99 (not the average), and find the knee in the curve before your customers do.
Put those five together and a self-hosted matrix stays sub-second under load that would bankrupt you on a metered API. Want help getting there? Let's talk.