You can offload the vision projector to CPU/sysRAM | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		7speter 3 days ago \| parent \| context \| favorite \| on: Benchmarking Qwen3.8 27B quantizations: 4-bit hold... You can offload the vision projector to CPU/sysRAM
		help

slim 3 days ago [–]

Awesome. How would you setup that ?

skirmish 3 days ago | [–]

  --no-mmproj-offload

See the documentation: https://github.com/ggml-org/llama.cpp/blob/master/docs/multi...

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact