Waymo

ML System Design: Inference Serving with Back-of-Envelope Capacity Planning

Members only

Onsite ML system design for a runtime / optimization-flavored MLE loop: design an inference serving system for a model with ~100M DAU and reason from first principles about memory footprint, bandwidth...