diff --git a/README.md b/README.md index e69de29..0944f09 100644 --- a/README.md +++ b/README.md @@ -0,0 +1,25 @@ +# Qwen3-0.6B + +Run [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B-GGUF) locally with ollama or llama.cpp. + +## Setup + +```sh +nix develop +``` + +## Usage + +### ollama + +```sh +ollama serve & +ollama run qwen3:0.6b +``` + +### llama.cpp + +```sh +# Download a GGUF from https://huggingface.co/Qwen/Qwen3-0.6B-GGUF +llama-cli -m Qwen3-0.6B-Q4_K_M.gguf -p "Hello" -cnv +```