xAI

Mock-LLM Inference Engine — Dynamic Batching

Members only

Implement a simplified inference-engine loop on top of a `model(batch) → next_tokens` API. The bar is dynamic batching: as sequences finish (max-tokens or stop-token), free their slots and fill them w...