25 lines
378 B
Markdown
25 lines
378 B
Markdown
# Qwen3-0.6B
|
|
|
|
Run [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B-GGUF) locally with ollama or llama.cpp.
|
|
|
|
## Setup
|
|
|
|
```sh
|
|
nix develop
|
|
```
|
|
|
|
## Usage
|
|
|
|
### ollama
|
|
|
|
```sh
|
|
ollama serve &
|
|
ollama run qwen3:0.6b
|
|
```
|
|
|
|
### llama.cpp
|
|
|
|
```sh
|
|
curl -L -O https://huggingface.co/Qwen/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q8_0.gguf
|
|
llama-cli -m Qwen3-0.6B-Q8_0.gguf -p "Hello" -cnv
|
|
```
|