Mistral

Nov 10, 20265 min read

A 1-Trillion-Parameter Model on Rented A100s: Mistral Large 4 with llama.cpp

1.05T

parameters, running on eight A100s

16.5 tok/s

for one user, faster than most people read