25 lines
358 B
Markdown
25 lines
358 B
Markdown
# Qwen3-0.6B
|
|
|
|
Run [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B-GGUF) locally with ollama or llama.cpp.
|
|
|
|
## Setup
|
|
|
|
```sh
|
|
nix develop
|
|
```
|
|
|
|
## Usage
|
|
|
|
### ollama
|
|
|
|
```sh
|
|
ollama serve &
|
|
ollama run qwen3:0.6b
|
|
```
|
|
|
|
### llama.cpp
|
|
|
|
```sh
|
|
# Download a GGUF from https://huggingface.co/Qwen/Qwen3-0.6B-GGUF
|
|
llama-cli -m Qwen3-0.6B-Q4_K_M.gguf -p "Hello" -cnv
|
|
```
|