3.5B per weight with no quality loss is state of the art - that's an awesome opt... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

buildbot on June 10, 2024 | parent | context | favorite | on: Apple's On-Device and Server Foundation Models

3.5B per weight with no quality loss is state of the art - that's an awesome optimization result (a mix of 2b and 4b weights).

Hugsun on June 11, 2024 [–]

I would like to see their method compared quantitatively to the best llama.cpp methods. IQ3_S has a similar bpw and pretty high quality.

I wonder if they didn't stretch the truth using the phrase "without loss in accuracy".

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact