6·
5 hours agoYou can. I’m running a 14B deepseek model on mine. It achieves 28 t/s.
The Hobbyist
Just a stranger trying things.
- 0 Posts
- 2 Comments
Joined 2 years ago
Cake day: July 16th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
Ollama, latest version. I have it setup with Open-WebUI (though that shouldn’t matter). The 14B is around 9GB, which easily fits in the 12GB.
I’m repeating the 28 t/s from memory, but even if I’m wrong it’s easily above 20.
Specifically, I’m running this model: https://ollama.com/library/deepseek-r1:14b-qwen-distill-q4_K_M