Open-source Large Language Models (LLMs) have become much easier to run. But one thing is often misunderstood: Running a model locally is not the same as serving a model efficiently.
I switched from ollama to llama.cpp and love it. For such an article, it frustrates me that they don’t include the common and popular option in the comparison.
Ollama gained traction by being the first easy llama.cpp wrapper, then spent years dodging attribution, misleading users, and pivoting to cloud, all while riding VC money earned on someone else’s engine.
He shares that the people behind llama.cpp don’t act poorly (at least in those regards). My experience confirms llama.cpp can run just about any gguf while ollama can only run those that have been customized for ollama, and llama.cpp seems faster (no evidence, just anecdotal).
I switched from ollama to llama.cpp and love it. For such an article, it frustrates me that they don’t include the common and popular option in the comparison.
What’s the difference between the two?
Here the article that spurred my change: Friends don’t let friends use Ollama
He shares that the people behind llama.cpp don’t act poorly (at least in those regards). My experience confirms llama.cpp can run just about any gguf while ollama can only run those that have been customized for ollama, and llama.cpp seems faster (no evidence, just anecdotal).