AI infrastructure
The three numbers that decide what your inference costs
Cost per thousand requests is an SLI. Batching, utilisation and the tail of your latency distribution set it, and most teams are only watching one of them.
Author
Long NGH
Dev/builder https://github.com/pierreneter. Founder of @p10node, Managing Partner at @transflox. And author of Localally https://amazon.com/dp/B0HHXZ62Q7
3 articles
Cloud cost
Cutting cloud spend, in the order that actually works
Most cost programmes start with reserved instances and stall. The savings are real but they are last, not first. Here is the sequence we use and why.
Reliability
What a reliability assessment actually looks at
Two weeks inside a production system, and the eight questions that decide the score. Most of them are not about the infrastructure.
Want this looked at in your own systems?
Two weeks inside your platform, a scored report, and a roadmap your team could run without us.