I get 40 tok/sec on my M3 Max on various 34B models, I gather a desktop 4090 wou... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

brandall10 on Dec 11, 2023 | parent | context | favorite | on: Mistral: Our first AI endpoints are available in e...

I get 40 tok/sec on my M3 Max on various 34B models, I gather a desktop 4090 would be at least 80?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact