Dynamic Batching for Autoregressive Decoding

Implement a dynamic batching scheduler for autoregressive generation with a simulated language model.

The model interface is:

next_tokens(b...

Unlock to view complete problem details

and practice with sample input/output

Was this article helpful?

View Test Cases & Run Code requires membership

Standard Input

Execution Result: