latency

Why routing is the real bottleneck in model serving

Routing, not model execution, is the main constraint in modern model serving.