Best local LLM for 16 GB
For RTX 4060 Ti (16 GB), RTX 4080 Super (16 GB), RTX 5060 Ti (16 GB), RTX 5070 Ti (16 GB), RTX 5080 (16 GB), Radeon RX 9070 XT (16 GB), Mac, 24 GB and Mac, 32 GB. Method
Behind the frontier23 monthsFirst reached by o1, Dec 2024
Models that fit in 16 GB
Top 10#ModelIndexFormatMemoryCheapest APILowest output price per 1M.
By memory
Index, highest firstMemorySmartest fitIndexFormatBehindAPI equivalent
Method
- Models: ranked models with open weights, an official Hugging Face repository and medium or high confidence. One line per set of weights.
- Memory need: the weights at the format's bits per weight (4-bit 4.85, 8-bit 8.5, 16-bit 16), plus 5% and 1 GB for a short context.
- Format: the most bits that fit. The index is for the full model; a 4-bit copy can score a little lower.
- Shared memory (Mac, DGX Spark, Ryzen AI Max): about two thirds goes to the model up to 36 GB, three quarters above.
- Behind the frontier: months from the first release with the same index or more, among models with medium or high confidence.
- Same by API: the lowest blended price (3 input : 1 output) for that index or more, per 1M tokens, among the same models.