4 ms·
This article describes what is usually called a Pareto frontier: the best known achievable trade-offs between two (or more) optimization goals. "Efficient front
by entrope 1mo ago
This article describes what is usually called a Pareto frontier: the best known achievable trade-offs between two (or more) optimization goals. "Efficient frontier" in common usage seems to be specifically a Pareto frontier for financial risk versus return of an investment portfolio. Even outside of finance, points on a Pareto frontier are called Pareto optimal or Pareto efficient. A Pareto frontier is sometimes shown with more than two dimensions, although usually people will pick just two for simplicity.
Within LLMs, and even inference naturally, there are many other potential parameters that one might optimize: Unsloth typically shows a Pareto frontier for size of a quantized model versus KL divergence. Others trade total concurrent tok/s against single-stream tok/s. KV cache size, context length and context coherency are other trade-offs that are closely related to inference. Total intelligence is usually a defining characteristic of a "frontier model", with cost (per token or task) as a salient trade-off. Cost is one parameter that is implicitly fixed by the "throughput versus latency" analysis: using a GB300 versus Radeon R9700 moves the curve enormously and probably changes the shape of it. Lots of threads here argue over local vs cloud inference regarding cost efficiency, often with privacy and control as competing objectives.