Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Shoot sorry about that. There's a few ways to keep the model loaded in memory:

1. If using `ollama run`: `ollama run llama3 --keepalive -1`

2. If running ollama serve directly, use `OLLAMA_KEEP_ALIVE=-1` ollama serve

3. If using the api, there's a `keep_alive` parameter you can set to -1



Yeah I tried all those things. Especially the middle one. Because that's how I use it mostly. I added the environment variable to the systemd service. But it still removes it after 5 minutes. Very weird.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: