Sure, but larger models that fit in that 512gb memory are going to take a long t... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		bigyabai on March 5, 2025 \| parent \| context \| favorite \| on: Apple M3 Ultra Sure, but larger models that fit in that 512gb memory are going to take a long time to tokenize/detokenize without hardware-accelerated BLAS.

microtonal on March 5, 2025 | [–]

Why would you need BLAS for tokenization/detokenization? Pretty much everyone still uses BBPE which amounts to iteratively applying merges.

(Maybe I'm missing something here.)

ryao on March 5, 2025 | [–]

Tokenization/detokenization does not use BLAS.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact