Fp8

Nov 10, 20266 min read

Ripple-65B on One H200: 43 Tokens per Second for One User, 1,197 at 64

43 tok/s

one request, a 65B dense model on one H200

1,197 tok/s

in total at 64 concurrent requests