Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

If you have to come up with a custom format anyway, why not just make it a draft extension to GGUF layout definitions (something like "coalesced expert fetch" or the like) and submit it for inclusion in the standard? Then future models could be autoconverted to such a format.


This is a good suggestion.

I will consider to do this after I gather enough experience to determine which is the best layout and when I will have enough benchmark data to support that.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: