How to run a 100% local, fully private LLM with llama.cpp 🔥 | LinkedIn
Discovered: Jun 17, 2024 18:28 How to run a 100% local, fully private LLM with llama.cpp 🔥
¦ LinkedIn –> QUOTE: How to run a 100% local, fully private LLM with llama.cpp 🔥 ... 2 lines of code, OpenAI compatible! ... Step 1: brew install llama.cpp ...Step 2: llama-server --hf-repo microsoft/Phi-3-mini-4k-instruct-gguf --hf-file Phi-3-mini-4k-instruct-q4.gguf ... Step 3: curl 8080/v1/chat/completions Also ` Step 1: Run the above instructions on your local computer Step 2: npm i holesail -g Step 3: holesail –live 8080 –host localhost and python -m llama_cpp.server –model XXX is way simpler`