Cheat sheets › Serving Large Language Models: The Numbers That Decide Speed and Cost

Serving Large Language Models: The Numbers That Decide Speed and Cost

Back-of-the-envelope methods for memory, latency and throughput, with a worked example you can check by hand.

• LLM inference
• GPU memory
• Batching
• Quantisation

Open this cheat sheetMake your own

What is inside

• Two phases, two bottlenecks
• Worked example: memory
• Worked example: speed
• The techniques, and what each one buys
• Rules of thumb

This is a ready-made, printable cheat sheet that fits 2 pages. Open it in CramSheet to read it, change anything, pick a layout and paper size, then print it or save it as a PDF. It opens as your own copy and works in your browser.